WO2019071662A1 - Electronic device, bill information identification method, and computer readable storage medium - Google Patents

Abstract

The present application discloses an electronic device, a bill information identification method, and a storage medium. The method comprises: upon receiving a bill image to be processed, identifying a bill type of the bill image by means of a pre-trained bill image identification model; using a predetermined correction rule to perform skew correction on the bill image; determining fields to be identified corresponding to the identified bill type; determining a first identification model corresponding to the fields to be identified, and calling the corresponding first identification model to perform region identification on a character line region of the bill image having undergone skew correction, so as to identify a target character line region containing character information of the fields to be identified; and determining a second identification model corresponding to the fields to be identified, and calling the corresponding second identification model to perform character identification, so as to identify character information contained in the target character line region of the fields to be identified. The technical solution of the present application achieves accurate and highly efficient automatic identification of text information in a bill image.

Description

Electronic device, ticket information identification method, and computer readable storage medium

The present application is based on the priority of the Chinese Patent Application entitled "Electronic Device, Bill Information Identification Method and Computer Readable Storage Medium", which is filed on October 9, 2017, with the application number of CN201710929629.8, which is filed on October 9, 2017. The overall content is incorporated herein by reference.

Technical field

The present application relates to the field of data identification technologies, and in particular, to an electronic device, a ticket information identification method, and a computer readable storage medium.

Background technique

Nowadays, with the development of the economy and the improvement of people's living standards, more and more people choose to purchase medical, commercial, financial and other insurance. In order to improve the user's insurance claims experience and improve the efficiency of insurance claims, some insurance companies have launched self-service claims services. For example, in the process of medical insurance claims, users only need to upload photos of outpatient or hospital bills to the insurance company system, insurance companies. The salesperson will enter the information on the picture uploaded by the user into the claim system for the next step. This self-service settlement method greatly facilitates the user's process of claim settlement; however, this self-service settlement method brings convenience. At the same time of the claims process, it increases the work pressure of the insurance company's business personnel. The problem is mainly caused by the need to spend a lot of manpower to process the image uploaded by the user, which is inefficient and the error rate of data entry is high.

Therefore, the automatic identification technology of bill information applied to the insurance self-service claims business is becoming more and more important and urgent. How to propose a scheme for automatically and accurately identifying the text information in the bill pictures uploaded by users has become an urgent problem to be solved.

Summary of the invention

The main purpose of the present application is to provide an electronic device, a ticket information identification method, and a computer readable storage medium, which are intended to accurately and efficiently realize automatic identification of text information in a ticket picture uploaded by a user.

A first aspect of the present application provides an electronic device including a memory, a processor, and a memory information recognition system operable on the processor, where the ticket information identification system is executed by the processor Implement the following steps:

After receiving the picture of the bill to be processed, the pre-trained bill picture recognition model is used to identify the bill type in the received bill picture, and output the category identification result of the bill;

Observing the received bill image with a predetermined correction rule;

Determining, according to a predetermined mapping relationship between the ticket category and the to-be-identified field, a field to be identified corresponding to the identified ticket category;

Determining, according to a predetermined mapping relationship between the to-be-identified field and the first recognition model, a first recognition model corresponding to each of the to-be-identified fields, and calling, for each of the to-be-identified fields, a first recognition model for tilt correction Performing area identification on the line character area of the ticket picture to respectively identify the target line character area including the character information of each of the to-be-identified fields;

Determining, according to a predetermined mapping relationship between the to-be-identified field and the second recognition model, a second recognition model corresponding to each of the to-be-identified fields, and calling a corresponding second recognition model for each of the target line character regions of the to-be-identified field The character recognition is performed to identify the character information included in the target line character region of each of the to-be-identified fields, and the character information of each of the identified to-be-identified fields is associated with the ticket image.

A second aspect of the present application provides a ticket information identification method, where the ticket information identification method includes the following steps:

After receiving the picture of the bill to be processed, the pre-trained bill picture recognition model is used to identify the bill type in the received bill picture, and output the category identification result of the bill;

Observing the received bill image with a predetermined correction rule;

Determining, according to a predetermined mapping relationship between the ticket category and the to-be-identified field, a field to be identified corresponding to the identified ticket category;

Determining, according to a predetermined mapping relationship between the to-be-identified field and the first recognition model, a first recognition model corresponding to each of the to-be-identified fields, and calling, for each of the to-be-identified fields, a first recognition model for tilt correction Performing area identification on the line character area of the ticket picture to respectively identify the target line character area including the character information of each of the to-be-identified fields;

Determining, according to a predetermined mapping relationship between the to-be-identified field and the second recognition model, a second recognition model corresponding to each of the to-be-identified fields, and calling a corresponding second recognition model for each of the target line character regions of the to-be-identified field The character recognition is performed to identify the character information included in the target line character region of each of the to-be-identified fields, and the character information of each of the identified to-be-identified fields is associated with the ticket image.

A third aspect of the present application provides a computer readable storage medium storing a ticket information identification system, the ticket information identification system being executable by at least one processor to cause the at least one processor Perform the following steps:

After receiving the picture of the bill to be processed, the pre-trained bill picture recognition model is used to identify the bill type in the received bill picture, and output the category identification result of the bill;

Observing the received bill image with a predetermined correction rule;

Determining, according to a predetermined mapping relationship between the ticket category and the to-be-identified field, a field to be identified corresponding to the identified ticket category;

Determining, according to a predetermined mapping relationship between the to-be-identified field and the first recognition model, a first recognition model corresponding to each of the to-be-identified fields, and calling, for each of the to-be-identified fields, a first recognition model for tilt correction Performing area identification on the line character area of the ticket picture to respectively identify the target line character area including the character information of each of the to-be-identified fields;

Determining, according to a predetermined mapping relationship between the to-be-identified field and the second recognition model, a second recognition model corresponding to each of the to-be-identified fields, and calling a corresponding second recognition model for each of the target line character regions of the to-be-identified field The character recognition is performed to identify the character information included in the target line character region of each of the to-be-identified fields, and the character information of each of the identified to-be-identified fields is associated with the ticket image.

In the technical solution of the present application, the ticket type in the received bill picture is first identified by the pre-trained bill picture recognition model, and the received bill picture is tilt corrected by a predetermined correction rule; Identifying the mapping relationship of the field, determining the to-be-identified field in the currently received bill image; and determining the first recognition model corresponding to each of the to-be-identified fields according to the mapping relationship between the to-be-identified field and the first recognition model, Identifying a target line character region of each field to be identified; finally, determining a second recognition model corresponding to each field to be identified according to a mapping relationship between the field to be identified and the second recognition model, and identifying each of the respective groups according to each second recognition model The character information included in the target line character area of the identification field is described, and the identified individual character information is associated with the current ticket picture, so that the automatic identification of the text information in the ticket picture uploaded by the user is realized accurately and efficiently. .

DRAWINGS

1 is a schematic flow chart of an embodiment of a method for identifying a ticket information according to the present application;

2 is a flowchart of a training process of a bill picture recognition model in an embodiment of the bill information identification method of the present application;

3 is a flowchart of a training process of a first identification model in an embodiment of a ticket information identification method of the present application;

4 is a flowchart of a training process of a second identification model in an embodiment of the ticket information identification method of the present application;

5 is a schematic diagram of an operating environment of an embodiment of a ticket information identification system of the present application;

FIG. 6 is a program block diagram of an embodiment of a ticket information identification system of the present application.

Detailed ways

The principles and features of the present application are described in the following with reference to the accompanying drawings, which are only used to explain the present application and are not intended to limit the scope of the application.

As shown in FIG. 1 , FIG. 1 is a schematic flowchart of an embodiment of a method for identifying a bill information according to the present application.

In this embodiment, the method for identifying the ticket information includes:

Step S10, after receiving the picture of the bill to be processed, using the pre-trained bill picture recognition model to identify the bill type in the received bill picture, and output the category identification result of the bill;

After receiving the picture of the bill to be processed, the system uses the pre-trained bill picture recognition model to identify the received bill picture, identify the category of the bill and output the category identification result; for example, the bill to be processed received by the system The picture is a picture of the medical bill, and the category of the medical bill includes the outpatient bill, the hospital bill, the surgical bill, etc., then the system uses the bill picture recognition model system to identify the received medical bill picture, and outputs the category identification result of the medical bill: the clinic Bills, hospital bills, surgical bills, etc.

Step S20, performing tilt correction on the received bill image by using a predetermined correction rule;

The system has predetermined correction rules; since the picture of the ticket uploaded by the user to the system (that is, the picture of the ticket received by the system) usually has a certain skew, the system will tilt the received picture of the ticket with a predetermined correction rule. Correction to ensure the system's success rate of identification of ticket information. Preferably, in the embodiment, the predetermined correction rule is: first, using a probability algorithm of Hough transform to find as many small straight lines as possible in the image; and then determining all the horizontal levels from the found small straight line. a straight line, and the straight lines in which the x coordinate values differ by less than the first preset difference (for example, 0.2 cm) are sequentially connected in the order of the corresponding y coordinate values, and are classified into several classes according to the size of the x coordinate value. Alternatively, the straight lines in which the determined y coordinate values differ by less than the second preset difference value (for example, 0.3 cm) are sequentially connected in the order of the magnitude of the corresponding x coordinate value, and are classified into several classes according to the size of the y coordinate value; All horizontal lines belonging to a class are regarded as a target class line, and the longest line closest to each target class line is found by least square method; finally, the slope of each long line is calculated, and the slope of each long line is calculated. The number of bits and the mean, the median and mean of the calculated slope are compared to determine the smaller one, and the image tilt is adjusted based on the smaller one determined. Of course, in other embodiments, other correction rules may also be employed.

Step S30: Determine, according to a mapping relationship between the predetermined ticket category and the to-be-identified field, a field to be identified corresponding to the identified ticket category;

After obtaining the ticket category of the ticket picture, the system may determine the to-be-identified field corresponding to the ticket category of the received ticket picture according to the mapping relationship between the predetermined ticket category and the to-be-identified field, and the ticket category corresponding to the identifier to be identified The number of fields may be one or more.

Step S40: determining, according to a predetermined mapping relationship between the to-be-identified field and the first recognition model, And the first recognition model corresponding to the to-be-identified field, for each of the to-be-identified fields, calling a corresponding first recognition model to perform region recognition on the line character region of the obliquely corrected bill image to identify each of the included a target line character area of the character information of the identified field;

The system has a first mapping relationship table between the to-be-identified field and the first identification model; after the system determines each of the to-be-identified fields of the ticket image, the system can find each of the to-be-finished by searching the first mapping relationship table. Identifying a first recognition model corresponding to each of the fields; the system, for each field to be identified, calling the first recognition model corresponding to the to-be-identified field to perform area recognition on the line character region of the obliquely corrected ticket image, thereby respectively identifying the ticket The image contains the target line character area of the character information of each field to be recognized.

Step S50: Determine, according to a predetermined mapping relationship between the to-be-identified field and the second recognition model, a second recognition model corresponding to each of the to-be-identified fields, and call a corresponding number for each of the target line character regions of the to-be-identified field. The second recognition model performs character recognition to respectively identify the character information included in the target line character region of each of the to-be-identified fields, and associates the identified character information of each of the to-be-identified fields with the ticket image.

The system further has a second mapping relationship table between the to-be-identified field and the second identification model; the system identifies the target line character area of the character information of each to-be-identified field after the ticket picture is identified, a second mapping relationship table, first finding a second recognition model corresponding to each of the to-be-identified fields; and then, for each target character region of the to-be-identified field, calling a corresponding second recognition model for character recognition, thereby Each of the corresponding second recognition models identifies the character information included in the target line character region of each of the to-be-identified fields, and then associates the recognized character information of each of the to-be-identified fields with the ticket image to establish an association. Mapping relations.

In the technical solution of the embodiment, the ticket type in the received ticket picture is first identified by the pre-trained ticket picture recognition model, and the received ticket picture is tilt corrected by a predetermined correction rule; Determining, by the mapping relationship of the to-be-identified field, the to-be-identified field in the currently-received bill image; and determining, according to the mapping relationship between the to-be-identified field and the first recognition model, respectively, the first recognition model corresponding to each of the to-be-identified fields, The target line character region of each field to be identified is identified; finally, according to the mapping relationship between the field to be identified and the second recognition model, a second recognition model corresponding to each field to be identified is determined, and each of the second recognition models is respectively identified according to each second recognition model. The character information included in the target line character area of the to-be-identified field is associated with the current ticket image, so that the text information in the bill image uploaded by the user is automatically and accurately implemented. Identification.

As shown in FIG. 2, in this embodiment, the training process of the ticket picture recognition model is as follows:

Step S1, preparing a preset number of ticket picture samples marked with corresponding picture categories for each preset ticket picture category;

For each preset ticket picture category, a preset number (for example, 1000 sheets) of coupon picture samples with corresponding picture categories are prepared; for example, there are two types of preset ticket picture categories, namely, outpatient tickets and hospitalization. The ticket prepares a preset number of ticket picture samples with the outpatient ticket and a preset number of ticket picture samples marked with the hospital ticket.

Step S2, dividing the ticket picture sample corresponding to each preset ticket picture category into a training subset of the first ratio and a verification subset of the second ratio, and mixing the ticket picture samples in each training subset to obtain a training set. And mixing the sample of the bill pictures in each verification subset to obtain a verification set;

For each of the preset ticket picture categories, the ticket picture samples are divided into a first proportion (for example, 80%) of the training subset and a second ratio (for example, 20%) of the verification subset, and then each of the obtained subsets A sample of the bill pictures in the training subset is mixed to obtain a training set (for example, a training set of bills by the outpatient bill) 80% of the sample of the tablet is mixed with 80% of the sample of the ticket image of the hospitalized ticket), and the obtained sample of the ticket image in each verification subset is mixed to obtain a verification set (for example, the verification set is sampled by the ticket picture of the outpatient ticket) 20% is formed by mixing 20% of the sample picture of the hospital bill.

Step S3, using the training set to train the ticket picture recognition model, and using the verification set to verify the accuracy of the ticket picture recognition model after the training set is completed;

After obtaining the training set and the verification set, the ticket picture recognition model is trained by using the obtained training set, and after the training of the ticket picture recognition model is completed by using the training set, the obtained verification set is used again. Verifying the accuracy of the ticket picture recognition model;

Step S4, if the accuracy rate is greater than or equal to the preset accuracy rate, the training ends;

The verification threshold of the accuracy rate (ie, the preset accuracy rate, for example, 98.5%) is preset in the system, and is used to check the training effect of the ticket picture recognition model; if the ticket image is used by the verification set If the accuracy of the recognition of the recognition model is greater than the preset accuracy, then the training of the ticket image recognition model reaches a preset standard, and the model training is ended.

In step S5, if the accuracy rate is less than the preset accuracy rate, the number of ticket picture samples corresponding to each preset ticket picture category is increased, and steps S2 and S3 are performed again.

If the accuracy of the verification of the bill image recognition model by the verification set is less than or equal to the preset accuracy rate, it indicates that the training of the bill image recognition model has not reached the preset standard, and may be the number of training sets. Not enough or the number of verification sets is not enough, so in this case, increase the number of ticket picture samples corresponding to each preset ticket picture category (for example, increase the fixed number each time or increase the random number each time), and then here Based on this, the above steps S2 and S3 are re-executed, and the loop is executed until the requirement of step S4 is reached, and the model training is ended.

In this embodiment, the ticket picture recognition model is preferably a deep convolutional neural network (for example, the deep convolutional neural network may be a SSD (Single Shot MultiBox Detector) algorithm selected in a CaffeNet environment. Model), the deep convolutional neural network model used in this application consists of one input layer, 13 convolutional layers, 5 pooling layers, 2 fully connected layers, and 1 sorting layer. The detailed structure of the deep convolutional neural network model is shown in the following table:

Among them: Layer Name column indicates the name of each layer, Input indicates the input layer, Conv indicates the convolution layer of the model, Conv1 indicates the first convolution layer of the model, MaxPool indicates the maximum pooling layer of the model, and MaxPool1 indicates the model. The first maximum pooling layer, Fc represents the fully connected layer in the model, Fc1 represents the first fully connected layer in the model, Softmax represents the Softmax classifier; Batch Size represents the number of input images of the current layer; Kernel Size represents the current layer The scale of the convolution kernel (for example, the Kernel Size can be equal to 3, indicating that the scale of the convolution kernel is 3x3); the Stride Size indicates the moving step size of the convolution kernel, that is, moving to the next convolution position after completing one convolution Distance; Pad Size indicates the size of the image fill in the current network layer.

In this embodiment, before the training process of the ticket picture recognition model, the ticket picture sample may be processed as follows:

According to the aspect ratio information and the position of the seal, the transposition of the bill picture is judged and the flip adjustment is made; when the aspect ratio is greater than 1, the height and width of the bill picture are reversed, and if the stamp position is on the left side of the bill picture, the bill is The image is rotated clockwise by ninety degrees. If the stamp position is on the right side of the bill image, the bill image is rotated counterclockwise by ninety degrees. When the aspect ratio is less than 1, the bill image height and width are not reversed. The position is on the lower side of the ticket picture, and the ticket image is rotated clockwise by one hundred and eighty degrees.

Find out the sample of the bill image with serious problems (for example, the key position information is missing or beyond the entire image range, and the stamp mark position is in the center of the bill, such as the bill sample sample that is obviously wrong), remove the bill image samples to ensure the bill image The accuracy of the sample ensures the training effect.

Further, as shown in FIG. 3, in the embodiment, the first recognition model is a convolutional neural network model, and the training process for the first recognition model corresponding to a field to be identified is as follows:

Step C1: Obtain a preset number of bill picture samples for the to-be-identified field;

The preset number (for example, 100,000) of the ticket picture samples are randomly obtained, wherein the partial ticket picture sample contains the character information of the to-be-identified field, and the partial ticket picture sample does not include the character information of the to-be-identified field.

Step C2, the ticket picture sample containing the character information of the to-be-identified field is classified into the first training set, and the ticket picture sample not containing the character information of the to-be-identified field is classified into the second training set;

From the obtained ticket picture sample, the ticket picture sample containing the character information of the to-be-identified field is separated from the ticket picture sample not containing the character information of the to-be-identified field, and the ticket picture including the character information of the to-be-identified field is included The sample is classified into the first training set, and the ticket picture sample that does not contain the character information of the to-be-identified field is classified into the second training set.

Step C3: Extracting, from the first training set and the second training set, the first preset ratio of the ticket picture samples as the sample picture to be trained, and taking the remaining ticket picture samples in the first training set and the second training set as Verified sample image;

Extracting, from the first training set and the second training set, a first preset ratio (for example, 80%) of bill picture samples as sample pictures to be trained, and remaining the first training set and the second training set The ticket picture sample is used as a sample picture with verification, so that the sample picture of the ticket to be trained and the sample picture to be verified are included in the sample picture with and without the character information of the to-be-identified field, and the included and not included The sample image of the character information of the field is sampled and to be sampled in the sample to be trained The proportions in the verified sample images are consistent.

Step C4: performing model training using the extracted sample images to be trained to generate the first recognition model, and verifying the generated first recognition model by using each sample image to be verified;

Performing model training on the preset type model by using the extracted sample images to be trained, thereby obtaining the first recognition model, and then verifying the obtained first recognition model with each sample image to be verified, and obtaining The verification pass rate of the first recognition model.

Step C5, if the verification pass rate is greater than or equal to the preset threshold, the training is completed;

The verification threshold of the verification pass rate (ie, the preset threshold, for example, 98%) is preset in the system, and is used to check the training effect of the first recognition model; If the verification pass rate obtained by the first identification model verification is greater than the preset threshold, then the training of the first recognition model reaches the expected standard, and the model training is ended.

In step C6, if the verification pass rate is less than the preset threshold, the number of ticket picture samples is increased, and steps C2, C3, and C4 are repeatedly executed.

If the verification pass rate obtained by verifying the first recognition model by each of the sample images to be verified is less than or equal to the preset threshold, it indicates that the training of the first recognition model has not reached the expected standard, which may be The number of sample pictures to be trained is insufficient or the number of sample pictures to be verified is insufficient, so in this case, the number of sample pictures of the ticket is increased (for example, each time a fixed number is added or a random number is added each time), and then On this basis, the above steps C2, C3 and C4 are re-executed, and the loop is executed until the requirement of step C5 is reached, and the model training is ended.

Further, as shown in FIG. 4, in the embodiment, the second recognition model is a Long-Short Term Memory (LSTM), and the training process for the second recognition model corresponding to a field to be identified is as follows: :

Step D1: Obtain a preset number of ticket picture samples for the to-be-identified field, where each ticket picture sample contains only one line of character information of the to-be-identified field, the font is black, the background is white, and each ticket picture sample is The name is named as the character information of the to-be-identified field contained therein;

Obtaining a preset number (for example, 100,000) of ticket picture samples; in the obtained ticket picture sample, each ticket picture sample includes and only contains one line of character information of the to-be-identified field, and the name of each ticket picture sample Named as the character information of the to-be-identified field contained therein; when the ticket picture sample is used for model training, the model can recognize the position of the character information according to the font color and the background color of the character information, thereby acquiring the Character information.

Step D2, dividing the ticket picture sample into a first data set and a second data set according to a ratio of X:Y, the number of picture samples in the first data set is greater than the number of picture samples in the second data set, and the first data set is used as training. Set, the second data set as a test set;

All the acquired ticket picture samples are divided into a first data set and a second data set according to a ratio of a preset ratio X:Y (X and Y are greater than 0), wherein the number of picture samples in the first data set is smaller than the second data set. The number of picture samples is large, that is, X is greater than Y (for example, X is 4, Y is 1); the first data set is used as a training set for training the model; and the second data set is used as a test set for training the test model. effect.

In step D3, the image samples in the first data set are sent to the time recurrent neural network model for model training, and the second data set is tested on the model every preset time to evaluate the effect of the current training model; The trained model performs character information recognition on the picture samples in the second data set, and compares with the names of the tested picture samples to calculate the recognition result and the error of the name of the picture sample.

Training the model with image samples from the first dataset, each preset time during the training process (for example, every 1000 iterations) or at a preset frequency, the model is tested using the second data set to detect the effect of the currently trained model; specifically, during the test, the model obtained by the current training is used in the second data set. The picture sample performs character information recognition, and compares the recognized result with the name of the tested picture sample, thereby calculating the error between the recognition result and the name of the picture sample. In this embodiment, the error calculation uses the edit distance as the calculation standard. .

Step D4, if the error of the model identification of the image sample diverges, the training parameters are adjusted and retrained;

If the model discards the error of the image sample recognition, the model training does not meet the requirements. At this time, the model is trained according to the preset rules or randomly adjusted training parameters, so that the error of the recognition of the bill image by the model during training can be convergence.

In step D5, if the error of the model identification on the image sample converges, the model training is ended, and the generated model is used as the final second recognition model corresponding to the to-be-identified field.

When the model converges on the error of the picture sample recognition, if the trained model meets the requirements, the model training is ended, and the generated model (ie, the current trained model) is used as the final second recognition model corresponding to the to-be-identified field.

In addition, the present application also proposes a ticket information identification system.

Please refer to FIG. 5 , which is a schematic diagram of an operating environment of a preferred embodiment of the ticket information identification system 10 of the present application.

In the present embodiment, the ticket information identification system 10 is installed and operated in the electronic device 1. The electronic device 1 may be a computing device such as a desktop computer, a notebook, a palmtop computer, and a server. The electronic device 1 may include, but is not limited to, a memory 11, a processor 12, and a display 13. Figure 5 shows only the electronic device 1 with components 11-13, but it should be understood that not all illustrated components may be implemented, and more or fewer components may be implemented instead.

The memory 11 may be an internal storage unit of the electronic device 1 in some embodiments, such as a hard disk or memory of the electronic device 1. The memory 11 may also be an external storage device of the electronic device 1 in other embodiments, such as a plug-in hard disk equipped on the electronic device 1, a smart memory card (SMC), and a secure digital (SD). Card, flash card, etc. Further, the memory 11 may also include both an internal storage unit of the electronic device 1 and an external storage device. The memory 11 is used to store application software and various types of data, such as program codes of the ticket information recognition system 10, installed in the electronic device 1. The memory 11 can also be used to temporarily store data that has been output or is about to be output.

The processor 12, in some embodiments, may be a Central Processing Unit (CPU), microprocessor or other data processing chip for running program code or processing data stored in the memory 11, such as performing ticket information identification. System 10 and so on.

The display 13 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch sensor, or the like in some embodiments. The display 13 is for displaying information processed in the electronic device 1 and a user interface for displaying visualization, such as a business customization interface or the like. The components 11-13 of the electronic device 1 communicate with one another via a system bus.

Please refer to FIG. 6, which is a program module diagram of a preferred embodiment of the ticket information identification system 10 of the present application. In the present embodiment, the ticket information identification system 10 can be divided into one or more modules, one or more modules are stored in the memory 11, and by one or more processors (the processor 12 in this embodiment) Executed to complete the application. For example, in FIG. 6, the ticket information identification system 10 can be divided into a first identification module 101, a correction module 102, a determination module 103, a second identification module 104, and a third identification module 105. The module referred to in the present application refers to a series of computer program instruction segments capable of performing a specific function, and is more suitable than the program for describing the execution process of the ticket information recognition system 10 in the electronic device 1. in:

The first identification module 101 is configured to: after receiving the picture of the ticket to be processed, identify the type of the ticket in the received ticket picture by using the pre-trained ticket picture recognition model, and output the category identification result of the ticket;

After receiving the picture of the bill to be processed, the system uses the pre-trained bill picture recognition model to identify the received bill picture, identify the category of the bill and output the category identification result; for example, the bill to be processed received by the system The picture is a picture of the medical bill, and the category of the medical bill includes the outpatient bill, the hospital bill, the surgical bill, etc., then the system uses the bill picture recognition model system to identify the received medical bill picture, and outputs the category identification result of the medical bill: the clinic Bills, hospital bills, surgical bills, etc.

The correction module 102 is configured to perform tilt correction on the received bill image by using a predetermined correction rule;

The system has predetermined correction rules; since the picture of the ticket uploaded by the user to the system (that is, the picture of the ticket received by the system) usually has a certain skew, the system will tilt the received picture of the ticket with a predetermined correction rule. Correction to ensure the system's success rate of identification of ticket information. Preferably, in the embodiment, the predetermined correction rule is: first, using a probability algorithm of Hough transform to find as many small straight lines as possible in the image; and then determining all the horizontal levels from the found small straight line. a straight line, and the straight lines in which the x coordinate values differ by less than the first preset difference (for example, 0.2 cm) are sequentially connected in the order of the corresponding y coordinate values, and are classified into several classes according to the size of the x coordinate value. Alternatively, the straight lines in which the determined y coordinate values differ by less than the second preset difference value (for example, 0.3 cm) are sequentially connected in the order of the magnitude of the corresponding x coordinate value, and are classified into several classes according to the size of the y coordinate value; All horizontal lines belonging to a class are regarded as a target class line, and the longest line closest to each target class line is found by least square method; finally, the slope of each long line is calculated, and the slope of each long line is calculated. The number of bits and the mean, the median and mean of the calculated slope are compared to determine the smaller one, and the image tilt is adjusted based on the smaller one determined. Of course, in other embodiments, other correction rules may also be employed.

a determining module 103, configured to determine, according to a predetermined mapping relationship between the ticket category and the to-be-identified field, a field to be identified corresponding to the identified ticket category;

After obtaining the ticket category of the ticket picture, the system may determine the to-be-identified field corresponding to the ticket category of the received ticket picture according to the mapping relationship between the predetermined ticket category and the to-be-identified field, and the ticket category corresponding to the identifier to be identified The number of fields may be one or more.

The second identification module 104 is configured to determine, according to a predetermined mapping relationship between the to-be-identified field and the first recognition model, a first identification model corresponding to each of the to-be-identified fields, and invoke a corresponding An identification model performs area recognition on the line character area of the obliquely corrected ticket picture to respectively identify the target line character area including the character information of each of the to-be-identified fields;

The system has a first mapping relationship table between the to-be-identified field and the first identification model; after the system determines each of the to-be-identified fields of the ticket image, the system can find each of the to-be-finished by searching the first mapping relationship table. Identifying a first recognition model corresponding to each of the fields; the system, for each field to be identified, calling the first recognition model corresponding to the to-be-identified field to perform area recognition on the line character region of the obliquely corrected ticket image, thereby respectively identifying the ticket The image contains the target line character area of the character information of each field to be recognized.

a third identification module 105, configured to determine, according to a predetermined mapping relationship between the to-be-identified field and the second recognition model, a second recognition model corresponding to each of the to-be-identified fields, and a target line character region for each of the to-be-identified fields And calling the corresponding second recognition model for character recognition to respectively identify the character information included in the target line character region of each of the to-be-identified fields, and identifying each of the identified to be recognized The character information of the field is associated with the ticket picture.

The system further has a second mapping relationship table between the to-be-identified field and the second identification model; the system identifies the target line character area of the character information of each to-be-identified field after the ticket picture is identified, a second mapping relationship table, first finding a second recognition model corresponding to each of the to-be-identified fields; and then, for each target character region of the to-be-identified field, calling a corresponding second recognition model for character recognition, thereby Each of the corresponding second recognition models identifies the character information included in the target line character region of each of the to-be-identified fields, and then associates the recognized character information of each of the to-be-identified fields with the ticket image to establish an association. Mapping relations.

In the technical solution of the embodiment, the ticket type in the received ticket picture is first identified by the pre-trained ticket picture recognition model, and the received ticket picture is tilt corrected by a predetermined correction rule; Determining, by the mapping relationship of the to-be-identified field, the to-be-identified field in the currently-received bill image; and determining, according to the mapping relationship between the to-be-identified field and the first recognition model, respectively, the first recognition model corresponding to each of the to-be-identified fields, The target line character region of each field to be identified is identified; finally, according to the mapping relationship between the field to be identified and the second recognition model, a second recognition model corresponding to each field to be identified is determined, and each of the second recognition models is respectively identified according to each second recognition model. The character information included in the target line character area of the to-be-identified field is associated with the current ticket image, so that the text information in the bill image uploaded by the user is automatically and accurately implemented. Identification.

In this embodiment, the training process of the ticket picture recognition model is as follows:

Step S1, preparing a preset number of ticket picture samples marked with corresponding picture categories for each preset ticket picture category;

For each preset ticket picture category, a preset number (for example, 1000 sheets) of coupon picture samples with corresponding picture categories are prepared; for example, there are two types of preset ticket picture categories, namely, outpatient tickets and hospitalization. The ticket prepares a preset number of ticket picture samples with the outpatient ticket and a preset number of ticket picture samples marked with the hospital ticket.

Step S2, dividing the ticket picture sample corresponding to each preset ticket picture category into a training subset of the first ratio and a verification subset of the second ratio, and mixing the ticket picture samples in each training subset to obtain a training set. And mixing the sample of the bill pictures in each verification subset to obtain a verification set;

For each of the preset ticket picture categories, the ticket picture samples are divided into a first proportion (for example, 80%) of the training subset and a second ratio (for example, 20%) of the verification subset, and then each of the obtained subsets The sample of the bill pictures in the training subset is mixed to obtain a training set (for example, the training set is formed by 80% of the bill picture sample of the outpatient bill and 80% of the bill picture sample of the hospital bill), and the respective verification subsets to be obtained The sample of the ticket pictures is mixed to obtain a verification set (eg, the verification set is formed by a mixture of 20% of the ticket picture sample of the outpatient ticket and 20% of the ticket picture sample of the hospitalized ticket).

Step S3, using the training set to train the ticket picture recognition model, and using the verification set to verify the accuracy of the ticket picture recognition model after the training set is completed;

After obtaining the training set and the verification set, the ticket picture recognition model is trained by using the obtained training set, and after the training of the ticket picture recognition model is completed by using the training set, the obtained verification set is used again. Verifying the accuracy of the ticket picture recognition model;

Step S4, if the accuracy rate is greater than or equal to the preset accuracy rate, the training ends;

The verification threshold of the accuracy rate (ie, the preset accuracy rate, for example, 98.5%) is preset in the system, and is used to check the training effect of the ticket picture recognition model; if the ticket image is used by the verification set If the accuracy of the recognition of the recognition model is greater than the preset accuracy, then the training of the ticket image recognition model reaches a preset standard, and the model training is ended.

In step S5, if the accuracy rate is less than the preset accuracy rate, the number of ticket picture samples corresponding to each preset ticket picture category is increased, and steps S2 and S3 are performed again.

If the accuracy of the verification of the bill image recognition model by the verification set is less than or equal to the preset accuracy rate, it indicates that the training of the bill image recognition model has not reached the preset standard, and may be the number of training sets. Not enough or the number of verification sets is not enough, so in this case, increase the number of ticket picture samples corresponding to each preset ticket picture category (for example, increase the fixed number each time or increase the random number each time), and then here Based on this, the above steps S2 and S3 are re-executed, and the loop is executed until the requirement of step S4 is reached, and the model training is ended.

In this embodiment, the ticket picture recognition model is preferably a deep convolutional neural network (for example, the deep convolutional neural network may be a SSD (Single Shot MultiBox Detector) algorithm selected in a CaffeNet environment. Model), the deep convolutional neural network model used in this application consists of one input layer, 13 convolutional layers, 5 pooling layers, 2 fully connected layers, and 1 sorting layer. The detailed structure of the deep convolutional neural network model is shown in the following table:

Layer Name	Batch Size	Kernel Size	Stride Size	Pad Size
Input	128	N/A	N/A	N/A
Conv1	128	3	1	1
Conv2	128	3	1	1
MaxPool1	128	2	2	0
Conv3	128	3	1	1
Conv4	128	3	1	1
MaxPool2	128	2	2	0
Conv5	128	3	1	1
Conv6	128	3	1	1
Conv7	128	3	1	1
MaxPool3	128	2	2	0
Conv8	128	3	1	1
Conv9	128	3	1	1
Conv10	128	3	1	1
MaxPool4	128	2	2	0
Conv11	128	3	1	1
Conv12	128	3	1	1
Conv13	128	3	1	1
MaxPool5	128	2	2	0
Fc1	4096	1	1	0
Fc2	2048	1	1	0
Softmax	3	N/A	N/A	N/A

Among them: Layer Name column indicates the name of each layer, Input indicates the input layer, Conv indicates the convolution layer of the model, Conv1 indicates the first convolution layer of the model, MaxPool indicates the maximum pooling layer of the model, and MaxPool1 indicates the model. The first maximum pooling layer, Fc represents the fully connected layer in the model, Fc1 represents the first fully connected layer in the model, Softmax represents the Softmax classifier; Batch Size represents the number of input images of the current layer; Kernel Size represents the current layer The scale of the convolution kernel (for example, the Kernel Size can be equal to 3, indicating that the scale of the convolution kernel is 3x3); the Stride Size indicates the moving step size of the convolution kernel, that is, moving to the next convolution position after completing one convolution Distance; Pad Size indicates the current network The size of the image fill in the layer.

In this embodiment, before the training process of the ticket picture recognition model, the ticket picture sample may be processed as follows:

According to the aspect ratio information and the position of the seal, the transposition of the bill picture is judged and the flip adjustment is made; when the aspect ratio is greater than 1, the height and width of the bill picture are reversed, and if the stamp position is on the left side of the bill picture, the bill is The image is rotated clockwise by ninety degrees. If the stamp position is on the right side of the bill image, the bill image is rotated counterclockwise by ninety degrees. When the aspect ratio is less than 1, the bill image height and width are not reversed. The position is on the lower side of the ticket picture, and the ticket image is rotated clockwise by one hundred and eighty degrees.

Find out the sample of the bill image with serious problems (for example, the key position information is missing or beyond the entire image range, and the stamp mark position is in the center of the bill, such as the bill sample sample that is obviously wrong), remove the bill image samples to ensure the bill image The accuracy of the sample ensures the training effect.

Further, in this embodiment, the first recognition model is a convolutional neural network model, and the training process for the first recognition model corresponding to a field to be identified is as follows:

Step C1: Obtain a preset number of bill picture samples for the to-be-identified field;

The preset number (for example, 100,000) of the ticket picture samples are randomly obtained, wherein the partial ticket picture sample contains the character information of the to-be-identified field, and the partial ticket picture sample does not include the character information of the to-be-identified field.

Step C2, the ticket picture sample containing the character information of the to-be-identified field is classified into the first training set, and the ticket picture sample not containing the character information of the to-be-identified field is classified into the second training set;

From the obtained ticket picture sample, the ticket picture sample containing the character information of the to-be-identified field is separated from the ticket picture sample not containing the character information of the to-be-identified field, and the ticket picture including the character information of the to-be-identified field is included The sample is classified into the first training set, and the ticket picture sample that does not contain the character information of the to-be-identified field is classified into the second training set.

Step C3: Extracting, from the first training set and the second training set, the first preset ratio of the ticket picture samples as the sample picture to be trained, and taking the remaining ticket picture samples in the first training set and the second training set as Verified sample image;

Extracting, from the first training set and the second training set, a first preset ratio (for example, 80%) of bill picture samples as sample pictures to be trained, and remaining the first training set and the second training set The ticket picture sample is used as a sample picture with verification, so that the sample picture of the ticket to be trained and the sample picture to be verified are included in the sample picture with and without the character information of the to-be-identified field, and the included and not included The ticket picture sample of the character information of the field is consistent in the proportion of the sample picture to be trained and the sample picture to be verified.

Step C4: performing model training using the extracted sample images to be trained to generate the first recognition model, and verifying the generated first recognition model by using each sample image to be verified;

Performing model training on the preset type model by using the extracted sample images to be trained, thereby obtaining the first recognition model, and then verifying the obtained first recognition model with each sample image to be verified, and obtaining The verification pass rate of the first recognition model.

Step C5, if the verification pass rate is greater than or equal to the preset threshold, the training is completed;

The verification threshold of the verification pass rate (ie, the preset threshold, for example, 98%) is preset in the system, and is used to check the training effect of the first recognition model; If the verification pass rate obtained by the first identification model verification is greater than the preset threshold, then the training of the first recognition model reaches the expected standard, and the model training is ended.

Step C6, if the verification pass rate is less than the preset threshold, increase the number of bill picture samples, and Repeat steps C2, C3, and C4.

If the verification pass rate obtained by verifying the first recognition model by each of the sample images to be verified is less than or equal to the preset threshold, it indicates that the training of the first recognition model has not reached the expected standard, which may be The number of sample pictures to be trained is insufficient or the number of sample pictures to be verified is insufficient, so in this case, the number of sample pictures of the ticket is increased (for example, each time a fixed number is added or a random number is added each time), and then On this basis, the above steps C2, C3 and C4 are re-executed, and the loop is executed until the requirement of step C5 is reached, and the model training is ended.

Further, in this embodiment, the second recognition model is a Long-Short Term Memory (LSTM), and the training process for the second recognition model corresponding to a field to be identified is as follows:

Step D1: Obtain a preset number of ticket picture samples for the to-be-identified field, where each ticket picture sample contains only one line of character information of the to-be-identified field, the font is black, the background is white, and each ticket picture sample is The name is named as the character information of the to-be-identified field contained therein;

Obtaining a preset number (for example, 100,000) of ticket picture samples; in the obtained ticket picture sample, each ticket picture sample includes and only contains one line of character information of the to-be-identified field, and the name of each ticket picture sample Named as the character information of the to-be-identified field contained therein; when the ticket picture sample is used for model training, the model can recognize the position of the character information according to the font color and the background color of the character information, thereby acquiring the Character information.

Step D2, dividing the ticket picture sample into a first data set and a second data set according to a ratio of X:Y, the number of picture samples in the first data set is greater than the number of picture samples in the second data set, and the first data set is used as training. Set, the second data set as a test set;

All the acquired ticket picture samples are divided into a first data set and a second data set according to a ratio of a preset ratio X:Y (X and Y are greater than 0), wherein the number of picture samples in the first data set is smaller than the second data set. The number of picture samples is large, that is, X is greater than Y (for example, X is 4, Y is 1); the first data set is used as a training set for training the model; and the second data set is used as a test set for training the test model. effect.

In step D3, the image samples in the first data set are sent to the time recurrent neural network model for model training, and the second data set is tested on the model every preset time to evaluate the effect of the current training model; The trained model performs character information recognition on the picture samples in the second data set, and compares with the names of the tested picture samples to calculate the recognition result and the error of the name of the picture sample.

The model is trained by the image samples in the first data set, and the second data set is tested on the model for each preset time (for example, every 1000 iterations) or at a preset frequency to detect the current The model effect of the training; specifically, during the test, the model obtained by the current training is used to identify the character information of the image sample in the second data set, and the recognized result is compared with the name of the tested picture sample, thereby calculating the recognized As a result of the error with the name of the picture sample, the preferred error calculation in this embodiment uses the edit distance as the calculation standard.

Step D4, if the error of the model identification of the image sample diverges, the training parameters are adjusted and retrained;

If the model discards the error of the image sample recognition, the model training does not meet the requirements. At this time, the model is trained according to the preset rules or randomly adjusted training parameters, so that the error of the recognition of the bill image by the model during training can be convergence.

In step D5, if the error of the model identification on the image sample converges, the model training is ended, and the generated model is used as the final second recognition model corresponding to the to-be-identified field.

When the model converges on the error of the image sample recognition, the trained model meets the requirements, and the model training ends. Practicing, and the generated model (ie, the model after the current training) is taken as the second recognition model corresponding to the final identified field.

Further, the present application further provides a computer readable storage medium storing a ticket information identification system, the ticket information identification system being executable by at least one processor to cause the at least one processing The ticket information identifying method in any of the above embodiments is performed.

The above description is only a preferred embodiment of the present invention, and is not intended to limit the scope of the invention, and the equivalent structural transformation, or direct/indirect use, of the present invention and the contents of the drawings are used in the inventive concept of the present invention. It is included in the scope of the patent protection of the present invention in other related technical fields.

Claims (20)

1. An electronic device, comprising: a memory, a processor, on the memory, a ticket information recognition system operable on the processor, wherein the ticket information recognition system is The following steps are implemented during execution:

   After receiving the picture of the bill to be processed, the pre-trained bill picture recognition model is used to identify the bill type in the received bill picture, and output the category identification result of the bill;

   Observing the received bill image with a predetermined correction rule;

   Determining, according to a predetermined mapping relationship between the ticket category and the to-be-identified field, a field to be identified corresponding to the identified ticket category;

   Determining, according to a predetermined mapping relationship between the to-be-identified field and the first recognition model, a first recognition model corresponding to each of the to-be-identified fields, and calling, for each of the to-be-identified fields, a first recognition model for tilt correction Performing area identification on the line character area of the ticket picture to respectively identify the target line character area including the character information of each of the to-be-identified fields;

   Determining, according to a predetermined mapping relationship between the to-be-identified field and the second recognition model, a second recognition model corresponding to each of the to-be-identified fields, and calling a corresponding second recognition model for each of the target line character regions of the to-be-identified field The character recognition is performed to identify the character information included in the target line character region of each of the to-be-identified fields, and the character information of each of the identified to-be-identified fields is associated with the ticket image.
2. The electronic device according to claim 1, wherein the training process of the ticket picture recognition model is as follows:

   S1. Prepare, for each preset ticket picture category, a preset number of ticket picture samples marked with corresponding picture categories;

   S2, dividing the ticket picture sample corresponding to each preset ticket picture category into a training subset of the first ratio and a verification subset of the second ratio, mixing the ticket picture samples in each training subset to obtain a training set, and Mixing the bill picture samples in each verification subset to obtain a verification set;

   S3. The ticket picture recognition model is trained by using the training set, and the accuracy of the ticket picture recognition model after the training set is completed is verified by using the verification set;

   S4. If the accuracy rate is greater than or equal to the preset accuracy rate, the training ends;

   S5. If the accuracy is less than the preset accuracy, increase the number of ticket picture samples corresponding to each preset ticket picture category, and perform steps S2 and S3 again.
3. The electronic device according to claim 1, wherein the first recognition model is a convolutional neural network model, and the training process for the first recognition model corresponding to a field to be identified is as follows:

   C1. Obtain a preset number of bill picture samples for the to-be-identified field;

   C2. The ticket picture sample containing the character information of the to-be-identified field is classified into the first training set, and the ticket picture sample not containing the character information of the to-be-identified field is classified into the second training set;

   C3. Extracting, from the first training set and the second training set, a first preset ratio of the ticket picture samples as the sample picture to be trained, and using the remaining ticket picture samples in the first training set and the second training set as the to-be-verified Sample picture

   C4, performing model training by using the extracted sample images to be trained to generate the first recognition model, and verifying the generated first recognition model by using each sample image to be verified;

   C5. If the verification pass rate is greater than or equal to a preset threshold, the training is completed;

   C6. If the verification pass rate is less than the preset threshold, increase the number of ticket picture samples, and repeat steps C2, C3, and C4.
4. The electronic device of claim 1 wherein said second recognition model is time The recursive neural network model, the training process for a second recognition model corresponding to a field to be identified is as follows:

   Obtaining a preset number of ticket picture samples for the to-be-identified field, each ticket picture sample includes only one line of character information of the to-be-identified field, the font is black, the background is white, and the name of each ticket picture sample is named The character information of the to-be-identified field contained therein;

   Dividing the bill picture sample into a first data set and a second data set according to a ratio of X:Y, the number of picture samples in the first data set is greater than the number of picture samples in the second data set, and the first data set is used as a training set, Two data sets as test sets;

   The image samples in the first data set are sent to the time recurrent neural network model for model training, and the second data set is tested on the model every preset time to evaluate the effect of the current training model; The model performs character information recognition on the picture samples in the second data set, and compares with the names of the tested picture samples to calculate the recognition result and the error of the name of the picture sample.

   If the model at the time of testing diverge the error in the recognition of the picture sample, adjust the training parameters and retrain;

   If the error of the model recognition on the image sample converges, the model training is ended, and the generated model is used as the final second recognition model corresponding to the to-be identified field.
5. The electronic device of claim 1 wherein said predetermined correction rule is:

   Use the probability algorithm of Hough transform to find as many small straight lines as possible in the image;

   All straight lines are determined from the found straight line, and the straight lines in which the x coordinate values differ by less than the first preset difference are sequentially connected in the order of the corresponding y coordinate values, according to the x coordinate The value size is divided into several categories, or the straight lines in which the y coordinate values of the determined straight lines differ by less than the second preset difference are sequentially connected in the order of the corresponding x coordinate values, and are classified into several classes according to the size of the y coordinate value;

   Use all horizontal lines belonging to a class as a target class line, and find the long line closest to each target class line by least squares method;

   Calculate the slope of each long line, calculate the median and mean of the slope of each long line, compare the median and mean of the calculated slope to determine the smaller one, and according to the smaller one determined Adjust the image tilt.
6. The electronic device according to claim 5, wherein the training process of the ticket picture recognition model is as follows:

   S1. Prepare, for each preset ticket picture category, a preset number of ticket picture samples marked with corresponding picture categories;

   S2, dividing the ticket picture sample corresponding to each preset ticket picture category into a training subset of the first ratio and a verification subset of the second ratio, mixing the ticket picture samples in each training subset to obtain a training set, and Mixing the bill picture samples in each verification subset to obtain a verification set;

   S3. The ticket picture recognition model is trained by using the training set, and the accuracy of the ticket picture recognition model after the training set is completed is verified by using the verification set;

   S4. If the accuracy rate is greater than or equal to the preset accuracy rate, the training ends;

   S5. If the accuracy is less than the preset accuracy, increase the number of ticket picture samples corresponding to each preset ticket picture category, and perform steps S2 and S3 again.
7. The electronic device according to claim 5, wherein the first recognition model is a convolutional neural network model, and the training process for the first recognition model corresponding to a field to be identified is as follows:

   C1. Obtain a preset number of bill picture samples for the to-be-identified field;

   C2. The ticket picture sample containing the character information of the to-be-identified field is classified into the first training set, and the ticket picture sample not containing the character information of the to-be-identified field is classified into the second training set;

   C3. Extracting, from the first training set and the second training set, a first preset ratio of the ticket picture samples as the sample picture to be trained, and using the remaining ticket picture samples in the first training set and the second training set as the to-be-verified Sample picture

   C4, performing model training by using the extracted sample images to be trained to generate the first recognition model, and verifying the generated first recognition model by using each sample image to be verified;

   C5. If the verification pass rate is greater than or equal to a preset threshold, the training is completed;

   C6. If the verification pass rate is less than the preset threshold, increase the number of ticket picture samples, and repeat steps C2, C3, and C4.
8. The electronic device according to claim 5, wherein the second recognition model is a time recurrent neural network model, and the training process for the second recognition model corresponding to a field to be identified is as follows:

   Obtaining a preset number of ticket picture samples for the to-be-identified field, each ticket picture sample includes only one line of character information of the to-be-identified field, the font is black, the background is white, and the name of each ticket picture sample is named The character information of the to-be-identified field contained therein;

   Dividing the bill picture sample into a first data set and a second data set according to a ratio of X:Y, the number of picture samples in the first data set is greater than the number of picture samples in the second data set, and the first data set is used as a training set, Two data sets as test sets;

   The image samples in the first data set are sent to the time recurrent neural network model for model training, and the second data set is tested on the model every preset time to evaluate the effect of the current training model; The model performs character information recognition on the picture samples in the second data set, and compares with the names of the tested picture samples to calculate the recognition result and the error of the name of the picture sample.

   If the model at the time of testing diverge the error in the recognition of the picture sample, adjust the training parameters and retrain;

   If the error of the model recognition on the image sample converges, the model training is ended, and the generated model is used as the final second recognition model corresponding to the to-be identified field.
9. A ticket information identification method, characterized in that the ticket information identification method comprises the steps of:

   After receiving the picture of the bill to be processed, the pre-trained bill picture recognition model is used to identify the bill type in the received bill picture, and output the category identification result of the bill;

   Observing the received bill image with a predetermined correction rule;

   Determining, according to a predetermined mapping relationship between the ticket category and the to-be-identified field, a field to be identified corresponding to the identified ticket category;

   Determining, according to a predetermined mapping relationship between the to-be-identified field and the first recognition model, a first recognition model corresponding to each of the to-be-identified fields, and calling, for each of the to-be-identified fields, a first recognition model for tilt correction Performing area identification on the line character area of the ticket picture to respectively identify the target line character area including the character information of each of the to-be-identified fields;

   Determining, according to a predetermined mapping relationship between the to-be-identified field and the second recognition model, a second recognition model corresponding to each of the to-be-identified fields, and calling a corresponding second recognition model for each of the target line character regions of the to-be-identified field The character recognition is performed to identify the character information included in the target line character region of each of the to-be-identified fields, and the character information of each of the identified to-be-identified fields is associated with the ticket image.
10. The ticket information identification method according to claim 9, wherein the training process of the ticket picture recognition model is as follows:

    S1. Prepare, for each preset ticket picture category, a preset number of ticket picture samples marked with corresponding picture categories;

    S2, dividing the sample of the bill picture corresponding to each preset bill picture category into the first proportion of the training sub- And a second proportional verification subset, mixing the ticket picture samples in each training subset to obtain a training set, and mixing the ticket picture samples in each verification subset to obtain a verification set;

    S3. The ticket picture recognition model is trained by using the training set, and the accuracy of the ticket picture recognition model after the training set is completed is verified by using the verification set;

    S4. If the accuracy rate is greater than or equal to the preset accuracy rate, the training ends;

    S5. If the accuracy is less than the preset accuracy, increase the number of ticket picture samples corresponding to each preset ticket picture category, and perform steps S2 and S3 again.
11. The ticket information identifying method according to claim 9, wherein the first recognition model is a convolutional neural network model, and the training process for the first recognition model corresponding to a field to be identified is as follows:

    C1. Obtain a preset number of bill picture samples for the to-be-identified field;

    C2. The ticket picture sample containing the character information of the to-be-identified field is classified into the first training set, and the ticket picture sample not containing the character information of the to-be-identified field is classified into the second training set;

    C3. Extracting, from the first training set and the second training set, a first preset ratio of the ticket picture samples as the sample picture to be trained, and using the remaining ticket picture samples in the first training set and the second training set as the to-be-verified Sample picture

    C4, performing model training by using the extracted sample images to be trained to generate the first recognition model, and verifying the generated first recognition model by using each sample image to be verified;

    C5. If the verification pass rate is greater than or equal to a preset threshold, the training is completed;

    C6. If the verification pass rate is less than the preset threshold, increase the number of ticket picture samples, and repeat steps C2, C3, and C4.
12. The ticket information identifying method according to claim 9, wherein the second recognition model is a time recurrent neural network model, and the training process for the second recognition model corresponding to a field to be identified is as follows:

    Obtaining a preset number of ticket picture samples for the to-be-identified field, each ticket picture sample includes only one line of character information of the to-be-identified field, the font is black, the background is white, and the name of each ticket picture sample is named The character information of the to-be-identified field contained therein;

    Dividing the bill picture sample into a first data set and a second data set according to a ratio of X:Y, the number of picture samples in the first data set is greater than the number of picture samples in the second data set, and the first data set is used as a training set, Two data sets as test sets;

    The image samples in the first data set are sent to the time recurrent neural network model for model training, and the second data set is tested on the model every preset time to evaluate the effect of the current training model; The model performs character information recognition on the picture samples in the second data set, and compares with the names of the tested picture samples to calculate the error between the recognition result and the name of the picture sample.

    If the model at the time of testing diverge the error in the recognition of the picture sample, adjust the training parameters and retrain;

    If the error of the model recognition on the image sample converges, the model training is ended, and the generated model is used as the final second recognition model corresponding to the to-be identified field.
13. The ticket information identifying method according to claim 9, wherein said predetermined correction rule is:

    Use the probability algorithm of Hough transform to find as many small straight lines as possible in the image;

    All straight lines are determined from the found straight line, and the straight lines in which the x coordinate values differ by less than the first preset difference are sequentially connected in the order of the corresponding y coordinate values, according to the x coordinate The value size is divided into several classes, or the determined y coordinate values in the straight line are less than the second preset difference. The straight line of the value is sequentially connected in the order of the size of the corresponding x coordinate value, and is divided into several classes according to the size of the y coordinate value;

    Use all horizontal lines belonging to a class as a target class line, and find the long line closest to each target class line by least squares method;

    Calculate the slope of each long line, calculate the median and mean of the slope of each long line, compare the median and mean of the calculated slope to determine the smaller one, and according to the smaller one determined Adjust the image tilt.
14. The ticket information identification method according to claim 13, wherein the training process of the ticket picture recognition model is as follows:

    S1. Prepare, for each preset ticket picture category, a preset number of ticket picture samples marked with corresponding picture categories;

    S2, dividing the ticket picture sample corresponding to each preset ticket picture category into a training subset of the first ratio and a verification subset of the second ratio, mixing the ticket picture samples in each training subset to obtain a training set, and Mixing the bill picture samples in each verification subset to obtain a verification set;

    S3. The ticket picture recognition model is trained by using the training set, and the accuracy of the ticket picture recognition model after the training set is completed is verified by using the verification set;

    S4. If the accuracy rate is greater than or equal to the preset accuracy rate, the training ends;

    S5. If the accuracy is less than the preset accuracy, increase the number of ticket picture samples corresponding to each preset ticket picture category, and perform steps S2 and S3 again.
15. The ticket information identifying method according to claim 13, wherein the first recognition model is a convolutional neural network model, and the training process for the first recognition model corresponding to a field to be identified is as follows:

    C1. Obtain a preset number of bill picture samples for the to-be-identified field;

    C2. The ticket picture sample containing the character information of the to-be-identified field is classified into the first training set, and the ticket picture sample not containing the character information of the to-be-identified field is classified into the second training set;

    C3. Extracting, from the first training set and the second training set, a first preset ratio of the ticket picture samples as the sample picture to be trained, and using the remaining ticket picture samples in the first training set and the second training set as the to-be-verified Sample picture

    C4, performing model training by using the extracted sample images to be trained to generate the first recognition model, and verifying the generated first recognition model by using each sample image to be verified;

    C5. If the verification pass rate is greater than or equal to a preset threshold, the training is completed;

    C6. If the verification pass rate is less than the preset threshold, increase the number of ticket picture samples, and repeat steps C2, C3, and C4.
16. The ticket information identifying method according to claim 13, wherein the second recognition model is a time recurrent neural network model, and the training process for the second recognition model corresponding to a field to be identified is as follows:

    Obtaining a preset number of ticket picture samples for the to-be-identified field, each ticket picture sample includes only one line of character information of the to-be-identified field, the font is black, the background is white, and the name of each ticket picture sample is named The character information of the to-be-identified field contained therein;

    Dividing the bill picture sample into a first data set and a second data set according to a ratio of X:Y, the number of picture samples in the first data set is greater than the number of picture samples in the second data set, and the first data set is used as a training set, Two data sets as test sets;

    The image samples in the first data set are sent to the time recurrent neural network model for model training, and the second data set is tested on the model every preset time to evaluate the effect of the current training model; The model performs character information recognition on the image samples in the second data set, and measures The name of the sample image to be tested is compared to calculate the error between the identified result and the name of the image sample.

    If the model at the time of testing diverge the error in the recognition of the picture sample, adjust the training parameters and retrain;

    If the error of the model recognition on the image sample converges, the model training is ended, and the generated model is used as the final second recognition model corresponding to the to-be identified field.
17. A computer readable storage medium, characterized in that the computer readable storage medium stores a ticket information identification system, the ticket information identification system being executable by at least one processor to cause the at least one processor to execute as follows step:

    After receiving the picture of the bill to be processed, the pre-trained bill picture recognition model is used to identify the bill type in the received bill picture, and output the category identification result of the bill;

    Observing the received bill image with a predetermined correction rule;

    Determining, according to a predetermined mapping relationship between the ticket category and the to-be-identified field, a field to be identified corresponding to the identified ticket category;

    Determining, according to a predetermined mapping relationship between the to-be-identified field and the first recognition model, a first recognition model corresponding to each of the to-be-identified fields, and calling, for each of the to-be-identified fields, a first recognition model for tilt correction Performing area identification on the line character area of the ticket picture to respectively identify the target line character area including the character information of each of the to-be-identified fields;

    Determining, according to a predetermined mapping relationship between the to-be-identified field and the second recognition model, a second recognition model corresponding to each of the to-be-identified fields, and calling a corresponding second recognition model for each of the target line character regions of the to-be-identified field The character recognition is performed to identify the character information included in the target line character region of each of the to-be-identified fields, and the character information of each of the identified to-be-identified fields is associated with the ticket image.
18. The computer readable storage medium of claim 17, wherein the training process of the ticket picture recognition model is as follows:

    S1. Prepare, for each preset ticket picture category, a preset number of ticket picture samples marked with corresponding picture categories;

    S2, dividing the ticket picture sample corresponding to each preset ticket picture category into a training subset of the first ratio and a verification subset of the second ratio, mixing the ticket picture samples in each training subset to obtain a training set, and Mixing the bill picture samples in each verification subset to obtain a verification set;

    S3. The ticket picture recognition model is trained by using the training set, and the accuracy of the ticket picture recognition model after the training set is completed is verified by using the verification set;

    S4. If the accuracy rate is greater than or equal to the preset accuracy rate, the training ends;

    S5. If the accuracy is less than the preset accuracy, increase the number of ticket picture samples corresponding to each preset ticket picture category, and perform steps S2 and S3 again.
19. The computer readable storage medium according to claim 17, wherein the first recognition model is a convolutional neural network model, and the training process for the first recognition model corresponding to a field to be identified is as follows:

    C1. Obtain a preset number of bill picture samples for the to-be-identified field;

    C2. The ticket picture sample containing the character information of the to-be-identified field is classified into the first training set, and the ticket picture sample not containing the character information of the to-be-identified field is classified into the second training set;

    C3. Extracting, from the first training set and the second training set, a first preset ratio of the ticket picture samples as the sample picture to be trained, and using the remaining ticket picture samples in the first training set and the second training set as the to-be-verified Sample picture

    C4. Perform model training by using the extracted sample images to be trained to generate the first identifier. Modeling, and verifying the generated first recognition model by using each sample image to be verified;

    C5. If the verification pass rate is greater than or equal to a preset threshold, the training is completed;

    C6. If the verification pass rate is less than the preset threshold, increase the number of ticket picture samples, and repeat steps C2, C3, and C4.
20. The computer readable storage medium according to claim 17, wherein the second recognition model is a time recurrent neural network model, and the training process for the second recognition model corresponding to a field to be identified is as follows:

    Obtaining a preset number of ticket picture samples for the to-be-identified field, each ticket picture sample includes only one line of character information of the to-be-identified field, the font is black, the background is white, and the name of each ticket picture sample is named The character information of the to-be-identified field contained therein;

    Dividing the bill picture sample into a first data set and a second data set according to a ratio of X:Y, the number of picture samples in the first data set is greater than the number of picture samples in the second data set, and the first data set is used as a training set, Two data sets as test sets;

    The image samples in the first data set are sent to the time recurrent neural network model for model training, and the second data set is tested on the model every preset time to evaluate the effect of the current training model; The model performs character information recognition on the picture samples in the second data set, and compares with the names of the tested picture samples to calculate the error between the recognition result and the name of the picture sample.

    If the model at the time of testing diverge the error in the recognition of the picture sample, adjust the training parameters and retrain;

    If the error of the model recognition on the image sample converges, the model training is ended, and the generated model is used as the final second recognition model corresponding to the to-be identified field.

Images