WO2019037259A1 - Electronic device, method and system for categorizing invoices, and computer-readable storage medium - Google Patents

Abstract

An electronic device, a method and system for categorizing invoices, and a computer-readable storage medium. The electronic device comprises a memory (11) and a processor (12) connected to the memory (11). The memory (11) is stored with an identification system operable by the processor (12). The identification system, when executed by the processor (12), performs the following steps: upon receiving images of invoices to be processed, performing tilt correction on the images of the invoices using a preset correction rule (S1); and performing category identification on the tilt-corrected images of the invoices using a pre-trained and generated invoice image identification model and outputting a result of category identification (S2). The electronic device can perform fast angle correction and categorization on batches of invoice images, improving the effect of processing.

Description

Electronic device, method and system for invoicing, and computer readable storage medium

Priority claim

This application is based on the priority of the Chinese Patent Application entitled "Electronic Device, Method of Invoice Classification and Computer-Readable Storage Media", filed on August 20, 2017, with the application number of CN 201710715451.7, which is filed on Aug. 20, 2017. The entire content is incorporated herein by reference.

Technical field

The present application relates to the field of communications technologies, and in particular, to an electronic device, a method and system for invoicing an invoice, and a computer readable storage medium.

Background technique

At present, for operations that require centralized financial data processing, such as life insurance claims, employee expense reimbursement, etc., before the centralized processing of batch uploading invoice images, it is usually necessary to manually correct the invoice image and correct the invoice. The images are classified for centralized business processing. The centralized business processing includes invoice accounting, invoice information entry, etc. The existing manual correction and classification of batch uploading invoice images is time-consuming and laborious, and inefficient.

Summary of the invention

The purpose of the present application is to provide an electronic device, a method and system for invoice classification, and a computer readable storage medium, which are intended to quickly correct and classify batch invoice images and improve processing efficiency.

To achieve the above object, the present application provides an electronic device including a memory and a processor coupled to the memory, wherein the memory stores an identification system operable on the processor, the identification The system implements the following steps when executed by the processor:

S1, after receiving the invoice image to be processed, performing tilt correction on the invoice image by using a predetermined correction rule;

S2, using the invoice image recognition model generated by the pre-training to classify the tilt corrected invoice image, and output the category recognition result;

The predetermined correction rules include:

Separating a short straight line segment of the invoice picture that is less than or equal to the first preset length by using a probability algorithm of Hough transform Hough;

Dividing each of the separated short straight line segments into several classes based on the x coordinate or the y coordinate of each of the separated short straight line segments;

Using a short straight line segment belonging to the same class as a target class straight line, and using a least squares method to obtain a long straight line segment similar to each target class straight line;

Calculate the slope of each long straight line segment and the median and mean of the slope of each long straight line segment, The median and mean of the calculated slope are compared to determine the smaller one, and the inclination of the invoice image is adjusted based on the smaller one determined.

To achieve the above object, the present application also provides a method for invoice classification, and the method for invoice classification includes:

S1, after receiving the invoice image to be processed, performing tilt correction on the invoice image by using a predetermined correction rule;

S2, using the invoice image recognition model generated by the pre-training to classify the tilt corrected invoice image, and output the category recognition result;

The predetermined correction rules include:

Separating a short straight line segment of the invoice picture that is less than or equal to the first preset length by using a probability algorithm of Hough transform Hough;

Dividing each of the separated short straight line segments into several classes based on the x coordinate or the y coordinate of each of the separated short straight line segments;

Using a short straight line segment belonging to the same class as a target class straight line, and using a least squares method to obtain a long straight line segment similar to each target class straight line;

Calculate the slope of each long straight line segment, and the median and mean of the slope of each long straight line segment, compare the calculated median and mean of the slope to determine the smaller one, and according to the smaller one determined Adjust the inclination of the invoice image.

To achieve the above object, the present application further provides an identification system, the identification system comprising:

a correction module, configured to perform a tilt correction on the invoice image by using a predetermined correction rule after receiving the invoice image to be processed;

The identification module is configured to perform category identification on the indented image after the tilt correction by using the invoice image recognition model generated by the pre-training, and output the category recognition result;

The predetermined correction rules include:

Separating a short straight line segment of the invoice picture that is less than or equal to the first preset length by using a probability algorithm of Hough transform Hough;

Dividing each of the separated short straight line segments into several classes based on the x coordinate or the y coordinate of each of the separated short straight line segments;

Using a short straight line segment belonging to the same class as a target class straight line, and using a least squares method to obtain a long straight line segment similar to each target class straight line;

Calculate the slope of each long straight line segment, and the median and mean of the slope of each long straight line segment, compare the calculated median and mean of the slope to determine the smaller one, and according to the smaller one determined Adjust the inclination of the invoice image.

The application further provides a computer readable storage medium having an identification system stored thereon, the implementation system implementing the steps when the recognition system is executed by the processor:

S1, after receiving the invoice image to be processed, performing tilt correction on the invoice image by using a predetermined correction rule;

S2, using the invoice image recognition model generated by the pre-training to classify the tilt corrected invoice image, and output the category recognition result;

The predetermined correction rules include:

Separating a short straight line segment of the invoice picture that is less than or equal to the first preset length by using a probability algorithm of Hough transform Hough;

Dividing each of the separated short straight line segments into several classes based on the x coordinate or the y coordinate of each of the separated short straight line segments;

Using a short straight line segment belonging to the same class as a target class straight line, and using a least squares method to obtain a long straight line segment similar to each target class straight line;

Calculate the slope of each long straight line segment, and the median and mean of the slope of each long straight line segment, compare the calculated median and mean of the slope to determine the smaller one, and according to the smaller one determined Adjust the inclination of the invoice image.

The beneficial effects of the present application are as follows: after receiving the invoice image to be processed, the present application performs a tilt correction on the invoice image by using a predetermined correction rule, and then adopts a pre-trained invoice image recognition model to correct the invoice image after the tilt correction. Perform category identification and output category recognition results. Compared with the existing manual correction and classification of batch uploading invoice images, this application can quickly correct and classify batch invoice images, saving time. Save effort and improve the efficiency of business processing.

DRAWINGS

1 is a schematic diagram of an optional application environment of each embodiment of the present application;

2 is a schematic flow chart of an embodiment of a method for classifying an invoice according to the present application;

Figure 3 is a schematic illustration of the predetermined correction rule shown in Figure 2.

Detailed ways

In order to make the objects, technical solutions, and advantages of the present application more comprehensible, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It is understood that the specific embodiments described herein are merely illustrative of the application and are not intended to be limiting. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments of the present application without departing from the inventive scope are the scope of the present application.

It should be noted that the descriptions of "first", "second" and the like in the present application are for the purpose of description only, and are not to be construed as indicating or implying their relative importance or implicitly indicating the number of technical features indicated. . Thus, features defining "first" and "second" may include at least one of the features, either explicitly or implicitly. In addition, the technical solutions between the various embodiments may be combined with each other, but must be based on the realization of those skilled in the art, and when the combination of the technical solutions is contradictory or impossible to implement, it should be considered that the combination of the technical solutions does not exist. Nor is it within the scope of protection required by this application.

Referring to FIG. 1, it is a schematic diagram of an application environment of a preferred embodiment of the method for invoicing an invoice of the present application. The application environment diagram includes an electronic device 1 and a terminal device 2. Electronic device 1 can pass A suitable technology such as a network or a near field communication technology performs data interaction with the terminal device 2.

The terminal device 2 includes, but is not limited to, any electronic product that can interact with a user through a keyboard, a mouse, a remote controller, a touch panel, or a voice control device, for example, a personal computer, a tablet computer, or a smart phone. , Personal Digital Assistant (PDA), game consoles, Internet Protocol Television (IPTV), smart wearable devices, navigation devices, etc., or mobile devices such as digital TVs, desktop computers, Fixed terminal for notebooks, servers, etc.

The electronic device 1 is an apparatus capable of automatically performing numerical calculation and/or information processing in accordance with an instruction set or stored in advance. The electronic device 1 may be a computer, a single network server, a server group composed of multiple network servers, or a cloud-based cloud composed of a large number of hosts or network servers, where cloud computing is a type of distributed computing. A super virtual computer consisting of a group of loosely coupled computers.

In the present embodiment, the electronic device 1 may include, but is not limited to, a memory 11 communicably connected to each other through a system bus, a processor 12, and a network interface 13, and the memory 11 stores an identification system operable on the processor 12. It should be noted that FIG. 1 only shows the electronic device 1 having the components 11-13, but it should be understood that not all illustrated components may be implemented, and more or fewer components may be implemented instead.

The storage device 11 includes a memory and at least one type of readable storage medium. The memory provides a cache for the operation of the electronic device 1; the readable storage medium may be, for example, a flash memory, a hard disk, a multimedia card, a card type memory (eg, SD or DX memory, etc.), a random access memory (RAM), a static random access memory (SRAM). A non-volatile storage medium such as a read only memory (ROM), an electrically erasable programmable read only memory (EEPROM), a programmable read only memory (PROM), a magnetic memory, a magnetic disk, an optical disk, or the like. In some embodiments, the readable storage medium may be an internal storage unit of the electronic device 1, such as a hard disk of the electronic device 1; in other embodiments, the non-volatile storage medium may also be external to the electronic device 1. A storage device, such as a plug-in hard disk equipped with an electronic device 1, a smart memory card (SMC), a Secure Digital (SD) card, a flash card, or the like. In this embodiment, the readable storage medium of the storage device 11 is generally used to store an operating system installed in the electronic device 1 and various types of application software, such as program codes of the identification system in an embodiment of the present application. Further, the storage device 11 can also be used to temporarily store various types of data that have been output or are to be output.

The processor 12 may be a Central Processing Unit (CPU), controller, microcontroller, microprocessor, or other data processing chip in some embodiments. The processor 12 is typically used to control the overall operation of the electronic device 1, such as performing control and processing related to data interaction or communication with the terminal device 2. In this embodiment, the processor 12 is configured to run program code or process data stored in the memory 11, such as running an identification system or the like.

The network interface 13 may comprise a wireless network interface or a wired network interface, which is typically used to establish a communication connection between the electronic device 1 and other electronic devices. In this embodiment, the network interface 13 is mainly used to connect the electronic device 1 with one or more terminal devices 2, in the electronic A device 1 establishes a data transmission channel and a communication connection with one or more terminal devices 2.

The identification system is stored in the memory 11 and includes at least one computer readable instruction stored in the memory 11, the at least one computer readable instruction being executable by the processor 12 to implement the methods of various embodiments of the present application; The at least one computer readable instruction can be classified into different logic modules according to different functions implemented by the various parts thereof. The embodiment includes a correction module and an identification module.

In an embodiment, the above identification system is implemented by the processor 12 to implement the following steps:

Step S1, after receiving the invoice image to be processed, performing tilt correction on the invoice image by using a predetermined correction rule;

In this embodiment, after receiving the batch of the invoice image to be processed, the invoice image is tilt corrected by using a predetermined correction rule, wherein the predetermined correction rule includes a plurality of types:

In an embodiment, the predetermined correction rule may be: obtaining an angle at which the invoice image is tilted, and correcting the invoice image based on the tilted angle;

In another embodiment, in order to correct the invoice picture more accurately, the predetermined correction rule may be: using a probability algorithm of Hough transform to separate the short length of the invoice picture that is less than or equal to the first preset length. A straight line segment, wherein the Hough transform probability algorithm is capable of detecting a straight line (line segment) from a black and white image of the invoice picture, the first preset length being, for example, 3 mm, and the separated short straight line segments being as many as possible. Dividing each of the separated short straight line segments into several categories based on the x coordinate or the y coordinate of each of the separated short straight line segments, specifically, determining that the horizontal tilt angle is less than or equal to a preset angle from the separated short straight line segments (eg, preset A short straight line segment with an angle of 5 degrees or 10 degrees, and a short straight line segment in which the determined x coordinate values differ by less than or equal to a preset threshold (for example, a preset threshold of 0.5 mm) is divided into one class until all The separated short straight line segments are divided into several categories, or the shortest straight line segments whose determined horizontal tilt angle is less than or equal to the preset angle are short enough that the difference of the y coordinate values is less than or equal to a preset threshold (for example, a preset threshold of 0.5 mm). The straight segments are divided into one category until all the separated short straight segments are divided into several categories. A short straight line segment belonging to the same class is used as a target class straight line, and a long straight line segment similar to each target class straight line is obtained by least square method, wherein the least square method finds the most straight line of each target class by minimizing the square sum of errors Good function matching (ie long straight line segments). Calculate the slope of each long straight line segment, and the median and mean of the slope of each long straight line segment, and compare the median and mean of the calculated slope to determine the slope with a small median and mean. And adjusting the inclination of the invoice picture according to the determined smaller slope. In other embodiments, the slope corresponding to the median minimum or the slope corresponding to the minimum value may be determined, and the slope corresponding to the median minimum or the mean is the smallest. The corresponding slope adjusts the inclination of the invoice picture. In addition, when adjusting the inclination of the invoice picture according to the slope, the angle of the inclination corresponding to the slope is obtained, and then the angle of the inclination is adjusted in the opposite direction of the invoice picture.

In step S2, the invoice image recognition model generated by the pre-training is used to classify the obliquely corrected invoice image, and the category recognition result is output.

In this embodiment, the invoice picture recognition model generated by the pre-training is a deep convolutional neural network model, wherein when the depth-convolution neural network model is used to classify the tilt-corrected invoice picture, preferably, the use can be performed in CaffeNet. Deep convolutional neural network The target detection algorithm classifies the invoice image after the tilt correction. Of course, other algorithms can also be used to identify the invoice image after the tilt correction, which is not limited here. In addition, there are many types of invoices. For example, the types of invoices for hospitals include outpatient invoices and hospital invoices. After classifying the invoice images after the tilt correction, the category corresponding to each invoice image is output.

Preferably, the target detection algorithm includes an infrastructure and an auxiliary architecture, specifically, including one input layer, 13 convolution layers, 5 pooling layers, 2 fully connected layers, and 1 sorting layer, as shown in Table 1 below. :

Layer Name	Batch Size	Kernel Size	Stride Size	Pad Size
Input	128	N/A	N/A	N/A
Conv1	128	3	1	1
Conv2	128	3	1	1
MaxPool1	128	2	2	0
Conv3	128	3	1	1
Conv4	128	3	1	1
MaxPool2	128	2	2	0
Conv5	128	3	1	1
Conv6	128	3	1	1
Conv7	128	3	1	1
MaxPool3	128	2	2	0
Conv8	128	3	1	1
Conv9	128	3	1	1
Conv10	128	3	1	1
MaxPool4	128	2	2	0
Conv11	128	3	1	1
Conv12	128	3	1	1
Conv13	128	3	1	1
MaxPool5	128	2	2	0
Fc1	4096	1	1	0
Fc2	2048	1	1	0
Softmax	3	N/A	N/A	N/A

Table 1

The Layer Name column indicates the name of each layer, the Batch Size indicates the number of input images of the current layer, and the Kernel Size indicates the scale of the current layer convolution kernel (for example, the Kernel Size can be equal to 3, indicating that the scale of the convolution kernel is 3x 3 Stride Size indicates the moving step size of the convolution kernel, that is, the distance moved to the next convolution position after one convolution, and the Pad Size indicates the size of the image padding in the current network layer. Input represents the input layer, Conv represents the convolutional layer, Conv1 represents the first convolutional layer, MaxPool represents the maximum pooled layer, MaxPool1 represents the first maximum pooled layer, Fc represents the fully connected layer, and Fc1 represents the first Fully connected layer, Softmax represents the Softmax classifier.

Compared with the prior art, the embodiment uses the predetermined after receiving the invoice image to be processed. The correction rule performs the tilt correction on the invoice image, and then uses the pre-trained invoice picture recognition model to classify the indented image after the tilt correction, and outputs the category recognition result, compared to the existing manual method for batch uploading The invoice picture is used for angle correction and classification. This embodiment can quickly correct and classify batch invoice pictures, save time and effort, and improve the efficiency of business processing.

In a preferred embodiment, based on the above embodiment, before the identification system is executed by the processor 12, the following steps are further implemented:

Preparing, for each preset invoice picture category, a first preset number of invoice picture samples marked with corresponding categories, and dividing the invoice picture sample into a first proportion of the training subset and a second proportion of the verification subset The preset invoice image category includes a plurality of, for example, an outpatient invoice and an inpatient invoice, and the first preset number is, for example, 1000 sheets, the first ratio is, for example, 75%, and the second ratio is, for example, 25%. Wherein, the sum of the first ratio and the second ratio is less than or equal to 1.

Obtaining a second preset number of certificate image samples corresponding to each preset invoice picture category, and dividing the certificate picture sample into a first proportion of the training subset and a second proportion of the verification subset, wherein each preset The sample of the certificate picture corresponding to the invoice picture category is the standard invoice picture corresponding to the invoice picture category, and the invoice picture of the standard is the invoice picture with the label and the information not having the problem, and the second preset quantity is, for example, 1000 sheets. The first ratio is, for example, 75%, and the second ratio is, for example, 25%, wherein the sum of the first ratio and the second ratio is less than or equal to 1.

Mixing picture samples from all training subsets to obtain a training set, mixing picture samples in all verification subsets to obtain a verification set, and training the deep convolutional neural network model with the training set, using the verification set Verify the accuracy of the deep convolutional neural network model after training. If the accuracy is greater than or equal to the preset accuracy (preset accuracy is, for example, 0.98), the training ends, and the post-training deep convolutional neural network model As the invoice picture recognition model in the step S2, or if the accuracy rate is less than the preset accuracy rate, the number of the certificate picture samples corresponding to each preset invoice picture category is increased to re-train.

In a preferred embodiment, on the basis of the above embodiments, in order to improve the efficiency of training the deep convolutional neural network model, the recognition system is implemented by the processor 12 before performing the training of the deep convolutional neural network model. The following steps:

Before training the deep convolutional neural network model, analyzing the image data of the training set and the annotation information of the image sample of the verification set, and cleaning the image sample with the wrong information;

According to the aspect ratio information of the invoice picture and the position of the seal, the transposition of the remaining picture samples after the cleaning is analyzed, and the transposition is reversed.

Before training the deep convolutional neural network model, analyzing the image information of the training set and the annotation information of the image sample of the verification set, for example, analyzing whether the key position information of the image sample is missing or exceeding the entire image range, and the seal labeling Whether the location is in the center of the invoice and other data marked with errors. If the image sample of the above problem occurs, it will be cleaned or discarded to ensure that the annotation information of the image sample is accurate.

For the remaining image samples after cleaning, determine the transposition of the image samples according to their aspect ratio information and the position of the seal, and make a flip adjustment for the transposed image samples: when the aspect ratio is greater than 1, the invoice image is high. The width is reversed. If the stamp position is on the left side of the image sample, the image sample is rotated clockwise by ninety degrees. If the stamp position is on the right side of the invoice image, the invoice image is rotated counterclockwise by ninety degrees; When the width ratio is less than 1, the height and width of the image sample are not reversed. If the stamp position is on the lower side of the invoice image, the invoice image is rotated clockwise by one hundred and eighty degrees. If the stamp position is on the upper side of the invoice image, the label is not made. deal with.

In addition, the label data of the image sample subjected to the flipping adjustment is corrected, and the label data of each image sample refers to the position information of the rectangular frame of the image sample, and the coordinates of the upper left corner of the rectangular frame (xmin, ymin) are used. And the lower right corner coordinates (xmax, ymax) are represented by four numbers. If xmax < xmin, the positions of the two are reversed, and the same processing is performed for the y coordinates to ensure that max>min.

As shown in FIG. 2, FIG. 2 is a schematic flowchart of a method for classifying an invoice according to an application, and the method for classifying the invoice includes the following steps:

Step S1, after receiving the invoice image to be processed, performing tilt correction on the invoice image by using a predetermined correction rule;

In this embodiment, after receiving the batch of the invoice image to be processed, the invoice image is tilt corrected by using a predetermined correction rule, wherein the predetermined correction rule includes a plurality of types:

In an embodiment, the predetermined correction rule may be: obtaining an angle at which the invoice image is tilted, and correcting the invoice image based on the tilted angle;

In another embodiment, in order to correct the invoice picture more accurately, referring to FIG. 3, the predetermined correction rule may be: separating the invoice image by the probability algorithm using the Hough transform Hough to be equal to or less than the first preset. A short straight line segment of length, wherein the Hough transform probability algorithm is capable of detecting a straight line (line segment) from a black and white image of the invoice picture, the first predetermined length being, for example, 3 mm, and the separated short straight line segments being as many as possible. Dividing each of the separated short straight line segments into several categories based on the x coordinate or the y coordinate of each of the separated short straight line segments, specifically, determining that the horizontal tilt angle is less than or equal to a preset angle from the separated short straight line segments (eg, preset A short straight line segment with an angle of 5 degrees or 10 degrees, and a short straight line segment in which the determined x coordinate values differ by less than or equal to a preset threshold (for example, a preset threshold of 0.5 mm) is divided into one class until all The separated short straight line segments are divided into several categories, or the shortest straight line segments whose determined horizontal tilt angle is less than or equal to the preset angle are short enough that the difference of the y coordinate values is less than or equal to a preset threshold (for example, a preset threshold of 0.5 mm). The straight segments are divided into one category until all the separated short straight segments are divided into several categories. A short straight line segment belonging to the same class is used as a target class straight line, and a long straight line segment similar to each target class straight line is obtained by least square method, wherein the least square method finds the most straight line of each target class by minimizing the square sum of errors Good function matching (ie long straight line segments). Calculate the slope of each long straight line segment, and the median and mean of the slope of each long straight line segment, and compare the median and mean of the calculated slope to determine the slope with a small median and mean. And adjusting the inclination of the invoice picture according to the determined smaller slope. In other embodiments, the slope corresponding to the median minimum or the slope corresponding to the minimum value may be determined, and the slope corresponding to the median minimum or the mean is the smallest. The corresponding slope adjusts the inclination of the invoice picture, in addition, according to the slope When adjusting the inclination of the invoice picture, obtain the angle of the inclination corresponding to the slope, and then adjust the angle of the inclination in the opposite direction of the invoice picture.

In step S2, the invoice image recognition model generated by the pre-training is used to classify the obliquely corrected invoice image, and the category recognition result is output.

In this embodiment, the invoice picture recognition model generated by the pre-training is a deep convolutional neural network model, wherein when the depth-convolution neural network model is used to classify the tilt-corrected invoice picture, preferably, the use can be performed in CaffeNet. The object detection algorithm based on deep convolutional neural network selected in the environment classifies the invoice image after tilt correction. Of course, other algorithms can also be used to identify the invoice image after tilt correction. . In addition, there are many types of invoices. For example, the types of invoices for hospitals include outpatient invoices and hospital invoices. After classifying the invoice images after the tilt correction, the category corresponding to each invoice image is output.

Preferably, the target detection algorithm includes an infrastructure and an auxiliary architecture, specifically, including one input layer, 13 convolution layers, 5 pooling layers, 2 fully connected layers, and one sorting layer, as shown in Table 1 above. Show, no longer repeat here.

The Layer Name column indicates the name of each layer, the Batch Size indicates the number of input images of the current layer, and the Kernel Size indicates the scale of the current layer convolution kernel (for example, the Kernel Size can be equal to 3, indicating that the scale of the convolution kernel is 3x 3 Stride Size indicates the moving step size of the convolution kernel, that is, the distance moved to the next convolution position after one convolution, and the Pad Size indicates the size of the image padding in the current network layer. Input represents the input layer, Conv represents the convolutional layer, Conv1 represents the first convolutional layer, MaxPool represents the maximum pooled layer, MaxPool1 represents the first maximum pooled layer, Fc represents the fully connected layer, and Fc1 represents the first Fully connected layer, Softmax represents the Softmax classifier.

After receiving the invoice image to be processed, the embodiment corrects the invoice image by using a predetermined correction rule, and then uses the pre-trained invoice image recognition model to identify the invoice image after the tilt correction, and outputs the invoice image. Compared with the existing manual correction method for manually correcting the batch invoice image, the present embodiment can quickly correct and classify batch invoice images, save time and effort, and improve business processing. s efficiency.

In a preferred embodiment, based on the foregoing embodiment of FIG. 2, before the step S2, the method further includes:

Preparing, for each preset invoice picture category, a first preset number of invoice picture samples marked with corresponding categories, and dividing the invoice picture sample into a first proportion of the training subset and a second proportion of the verification subset The preset invoice image category includes a plurality of, for example, an outpatient invoice and an inpatient invoice, and the first preset number is, for example, 1000 sheets, the first ratio is, for example, 75%, and the second ratio is, for example, 25%. Wherein, the sum of the first ratio and the second ratio is less than or equal to 1.

Obtaining a second preset number of certificate image samples corresponding to each preset invoice picture category, and dividing the certificate picture sample into a first proportion of the training subset and a second proportion of the verification subset, wherein each preset The sample of the certificate picture corresponding to the invoice picture category is the standard corresponding to the invoice picture category. The invoice picture, the standard invoice picture is an upright, invoice picture with no problem with the label information, the second preset quantity is, for example, 1000 sheets, the first ratio is, for example, 75%, and the second ratio is, for example, 25%, wherein The sum of the first ratio and the second ratio is less than or equal to 1.

Mixing picture samples from all training subsets to obtain a training set, mixing picture samples in all verification subsets to obtain a verification set, and training the deep convolutional neural network model with the training set, using the verification set Verify the accuracy of the deep convolutional neural network model after training. If the accuracy is greater than or equal to the preset accuracy (preset accuracy is, for example, 0.98), the training ends, and the post-training deep convolutional neural network model As the invoice picture recognition model in the step S2, or if the accuracy rate is less than the preset accuracy rate, the number of the certificate picture samples corresponding to each preset invoice picture category is increased to re-train.

In a preferred embodiment, based on the foregoing embodiment, in order to improve the efficiency of training the deep convolutional neural network model, the method for invoice classification further includes:

Before training the deep convolutional neural network model, analyzing the image data of the training set and the annotation information of the image sample of the verification set, and cleaning the image sample with the wrong information;

According to the aspect ratio information of the invoice picture and the position of the seal, the transposition of the remaining picture samples after the cleaning is analyzed, and the transposition is reversed.

Before training the deep convolutional neural network model, analyzing the image information of the training set and the annotation information of the image sample of the verification set, for example, analyzing whether the key position information of the image sample is missing or exceeding the entire image range, and the seal labeling Whether the location is in the center of the invoice and other data marked with errors. If the image sample of the above problem occurs, it will be cleaned or discarded to ensure that the annotation information of the image sample is accurate.

For the remaining image samples after cleaning, determine the transposition of the image samples according to their aspect ratio information and the position of the seal, and make a flip adjustment for the transposed image samples: when the aspect ratio is greater than 1, the invoice image is high. The width is reversed. If the stamp position is on the left side of the image sample, the image sample is rotated clockwise by ninety degrees. If the stamp position is on the right side of the invoice image, the invoice image is rotated counterclockwise by ninety degrees; When the width ratio is less than 1, the height and width of the image sample are not reversed. If the stamp position is on the lower side of the invoice image, the invoice image is rotated clockwise by one hundred and eighty degrees. If the stamp position is on the upper side of the invoice image, the label is not made. deal with.

In addition, the label data of the image sample subjected to the flipping adjustment is corrected, and the label data of each image sample refers to the position information of the rectangular frame of the image sample, and the coordinates of the upper left corner of the rectangular frame (xmin, ymin) are used. And the lower right corner coordinates (xmax, ymax) are represented by four numbers. If xmax < xmin, the positions of the two are reversed, and the same processing is performed for the y coordinates to ensure that max>min.

The present application also provides a computer readable storage medium having stored thereon an identification system, the steps of which are implemented by the processor to implement the method of invoicing the invoice described above.

The serial numbers of the embodiments of the present application are merely for the description, and do not represent the advantages and disadvantages of the embodiments.

Through the description of the above embodiments, those skilled in the art can clearly understand that the foregoing embodiment method can be implemented by means of software plus a necessary general hardware platform, and of course, Hardware, but in many cases the former is a better implementation. Based on such understanding, the technical solution of the present application, which is essential or contributes to the prior art, may be embodied in the form of a software product stored in a storage medium (such as ROM/RAM, disk, The optical disc includes a number of instructions for causing a terminal device (which may be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to perform the methods described in various embodiments of the present application.

The above is only a preferred embodiment of the present application, and is not intended to limit the scope of the patent application, and the equivalent structure or equivalent process transformations made by the specification and the drawings of the present application, or directly or indirectly applied to other related technical fields. The same is included in the scope of patent protection of this application.

Claims (20)

1. An electronic device, comprising: a memory and a processor coupled to the memory, wherein the memory stores an identification system operable on the processor, the identification system being The processor implements the following steps when it executes:

   S1, after receiving the invoice image to be processed, performing tilt correction on the invoice image by using a predetermined correction rule;

   S2, using the invoice image recognition model generated by the pre-training to classify the tilt corrected invoice image, and output the category recognition result;

   The predetermined correction rules include:

   Separating a short straight line segment of the invoice picture that is less than or equal to the first preset length by using a probability algorithm of Hough transform Hough;

   Dividing each of the separated short straight line segments into several classes based on the x coordinate or the y coordinate of each of the separated short straight line segments;

   Using a short straight line segment belonging to the same class as a target class straight line, and using a least squares method to obtain a long straight line segment similar to each target class straight line;

   Calculate the slope of each long straight line segment, and the median and mean of the slope of each long straight line segment, compare the calculated median and mean of the slope to determine the smaller one, and according to the smaller one determined Adjust the inclination of the invoice image.
2. The electronic device according to claim 1, wherein the invoice picture recognition model is a deep convolutional neural network model, and the deep convolutional neural network model comprises a target detection algorithm.
3. The electronic device according to claim 2, wherein the deep convolutional neural network model comprises one input layer, 13 convolution layers, 5 pooling layers, 2 fully connected layers, and 1 sorting layer. .
4. The electronic device according to claim 2, wherein when the identification system is executed by the processor, the following steps are further implemented:

   Preparing, for each preset invoice picture category, a first preset number of invoice picture samples marked with corresponding categories, and dividing the invoice picture sample into a first proportion of the training subset and a second proportion of the verification subset ;

   Obtaining a second preset number of certificate picture samples corresponding to each preset invoice picture category, and dividing the certificate picture sample into a first proportion of the training subset and a second proportion of the verification subset;

   Mixing the image samples of all training subsets to obtain a training set, and mixing the image samples of all the verification subsets to obtain a verification set;

   Training the deep convolutional neural network model with the training set;

   Using the verification set to verify the accuracy of the trained deep convolutional neural network model;

   If the accuracy rate is greater than or equal to the preset accuracy rate, the training ends, and the trained deep convolutional neural network model is used as the invoice picture recognition model in the step S2, or if the accuracy rate is less than the preset accuracy Rate, then increase the image of the certificate corresponding to each preset invoice image category. The number of this to re-train.
5. The electronic device according to claim 4, wherein when the identification system is executed by the processor, the following steps are further implemented:

   Before training the deep convolutional neural network model, analyzing the image data of the training set and the annotation information of the image sample of the verification set, and cleaning the image sample with the wrong information;

   According to the aspect ratio information of the invoice picture and the position of the seal, the transposition of the remaining picture samples after the cleaning is analyzed, and the transposition is reversed.
6. A method for classifying invoices, characterized in that the method for classifying invoices comprises:

   S1, after receiving the invoice image to be processed, performing tilt correction on the invoice image by using a predetermined correction rule;

   S2, using the invoice image recognition model generated by the pre-training to classify the tilt corrected invoice image, and output the category recognition result;

   The predetermined correction rules include:

   Separating a short straight line segment of the invoice picture that is less than or equal to the first preset length by using a probability algorithm of Hough transform Hough;

   Dividing each of the separated short straight line segments into several classes based on the x coordinate or the y coordinate of each of the separated short straight line segments;

   Using a short straight line segment belonging to the same class as a target class straight line, and using a least squares method to obtain a long straight line segment similar to each target class straight line;

   Calculate the slope of each long straight line segment, and the median and mean of the slope of each long straight line segment, compare the calculated median and mean of the slope to determine the smaller one, and according to the smaller one determined Adjust the inclination of the invoice image.
7. The method of invoice classification according to claim 6, wherein the invoice picture recognition model is a deep convolutional neural network model, and the deep convolutional neural network model comprises a target detection algorithm.
8. The method for invoice classification according to claim 7, wherein the deep convolutional neural network model comprises an input layer, 13 convolution layers, 5 pooling layers, 2 fully connected layers, and 1 Classification layer.
9. The method of invoice classification according to claim 7, wherein the step S2 further comprises:

   Preparing, for each preset invoice picture category, a first preset number of invoice picture samples marked with corresponding categories, and dividing the invoice picture sample into a first proportion of the training subset and a second proportion of the verification subset ;

   Obtaining a second preset number of certificate picture samples corresponding to each preset invoice picture category, and dividing the certificate picture sample into a first proportion of the training subset and a second proportion of the verification subset;

   Mixing the image samples of all training subsets to obtain a training set, and mixing the image samples of all the verification subsets to obtain a verification set;

   Training the deep convolutional neural network model with the training set;

   Using the verification set to verify the accuracy of the trained deep convolutional neural network model;

   If the accuracy rate is greater than or equal to the preset accuracy rate, the training ends, and the trained deep convolutional neural network model is used as the invoice picture recognition model in the step S2, or if the accuracy rate is less than the preset accuracy Rate, then increase the number of sample picture samples corresponding to each preset invoice picture category to re-train.
10. The method of invoice classification according to claim 9, wherein the method for invoice classification further comprises:

    Before training the deep convolutional neural network model, analyzing the image data of the training set and the annotation information of the image sample of the verification set, and cleaning the image sample with the wrong information;

    According to the aspect ratio information of the invoice picture and the position of the seal, the transposition of the remaining picture samples after the cleaning is analyzed, and the transposition is reversed.
11. An identification system, wherein the identification system comprises:

    a correction module, configured to perform a tilt correction on the invoice image by using a predetermined correction rule after receiving the invoice image to be processed;

    The identification module is configured to perform category identification on the indented image after the tilt correction by using the invoice image recognition model generated by the pre-training, and output the category recognition result;

    The predetermined correction rules include:

    Separating a short straight line segment of the invoice picture that is less than or equal to the first preset length by using a probability algorithm of Hough transform Hough;

    Dividing each of the separated short straight line segments into several classes based on the x coordinate or the y coordinate of each of the separated short straight line segments;

    Using a short straight line segment belonging to the same class as a target class straight line, and using a least squares method to obtain a long straight line segment similar to each target class straight line;

    Calculate the slope of each long straight line segment, and the median and mean of the slope of each long straight line segment, compare the calculated median and mean of the slope to determine the smaller one, and according to the smaller one determined Adjust the inclination of the invoice image.
12. The identification system according to claim 11, wherein said invoice picture recognition model is a deep convolutional neural network model, and said deep convolutional neural network model comprises a target detection algorithm.
13. The identification system according to claim 12, wherein said deep convolutional neural network model comprises an input layer, 13 convolution layers, 5 pooling layers, 2 fully connected layers, and 1 sorting layer. .
14. The identification system according to claim 12, wherein the identification system further comprises a training module, configured to prepare a first preset number of invoice images marked with corresponding categories for each preset invoice picture category. a sample, the invoice picture sample is divided into a first proportion of the training subset and the second proportion of the verification subset; obtaining a second preset number of certificate picture samples corresponding to each preset invoice picture category, the document is The picture sample is divided into a first proportion of the training subset and the second proportion of the verification subset; the picture samples of all the training subsets are mixed to obtain a training set, and the picture samples of all the verification subsets are mixed to obtain a verification set; Training the deep convolutional neural network model with the training set; verifying the trained deep convolutional neural network model by using the verification set The accuracy rate; if the accuracy rate is greater than or equal to the preset accuracy rate, the training ends, and the trained deep convolutional neural network model is used as the invoice picture recognition model in the identification module, or if the accuracy rate is If it is less than the preset accuracy rate, the number of the certificate picture samples corresponding to each preset invoice picture category is increased to re-train.
15. The identification system of claim 14, wherein the identification system further comprises:

    An analysis module, configured to analyze the image data of the training set and the annotation information of the image sample of the verification set before training the deep convolutional neural network model, and clean the image sample with the wrong information;

    The adjustment module is configured to analyze the transposition condition of the remaining image samples after the cleaning according to the aspect ratio information of the invoice picture and the position of the seal, and perform the inversion adjustment of the transposition.
16. A computer readable storage medium, wherein the computer readable storage medium stores an identification system, and when the identification system is executed by the processor, the steps are:

    S1, after receiving the invoice image to be processed, performing tilt correction on the invoice image by using a predetermined correction rule;

    S2, using the invoice image recognition model generated by the pre-training to classify the tilt corrected invoice image, and output the category recognition result;

    The predetermined correction rules include:

    Separating a short straight line segment of the invoice picture that is less than or equal to the first preset length by using a probability algorithm of Hough transform Hough;

    Dividing each of the separated short straight line segments into several classes based on the x coordinate or the y coordinate of each of the separated short straight line segments;

    Using a short straight line segment belonging to the same class as a target class straight line, and using a least squares method to obtain a long straight line segment similar to each target class straight line;

    Calculate the slope of each long straight line segment, and the median and mean of the slope of each long straight line segment, compare the calculated median and mean of the slope to determine the smaller one, and according to the smaller one determined Adjust the inclination of the invoice image.
17. The computer readable storage medium of claim 16, wherein the invoice picture recognition model is a deep convolutional neural network model, and the deep convolutional neural network model comprises a target detection algorithm.
18. The computer readable storage medium according to claim 17, wherein said deep convolutional neural network model comprises an input layer, 13 convolutional layers, 5 pooling layers, 2 fully connected layers, and 1 Classification layer.
19. The computer readable storage medium according to claim 17, wherein when said identification system is executed by said processor, the following steps are further implemented:

    Preparing, for each preset invoice picture category, a first preset number of invoice picture samples marked with corresponding categories, and dividing the invoice picture sample into a first proportion of the training subset and a second proportion of the verification subset ;

    Obtaining a second preset number of certificate image samples corresponding to each preset invoice image category, The sample picture of the document is divided into a training subset of the first ratio and a verification subset of the second ratio;

    Mixing the image samples of all training subsets to obtain a training set, and mixing the image samples of all the verification subsets to obtain a verification set;

    Training the deep convolutional neural network model with the training set;

    Using the verification set to verify the accuracy of the trained deep convolutional neural network model;

    If the accuracy rate is greater than or equal to the preset accuracy rate, the training ends, and the trained deep convolutional neural network model is used as the invoice picture recognition model in the step S2, or if the accuracy rate is less than the preset accuracy Rate, then increase the number of sample picture samples corresponding to each preset invoice picture category to re-train.
20. The computer readable storage medium according to claim 19, wherein when said identification system is executed by said processor, the following steps are further implemented:

    Before training the deep convolutional neural network model, analyzing the image data of the training set and the annotation information of the image sample of the verification set, and cleaning the image sample with the wrong information;

    According to the aspect ratio information of the invoice picture and the position of the seal, the transposition of the remaining picture samples after the cleaning is analyzed, and the transposition is reversed.

Images