WO2021042747A1

WO2021042747A1 - Invoice picture recognition and verification method and system, device, and readable storage medium

Info

Publication number: WO2021042747A1
Application number: PCT/CN2020/087586
Authority: WO
Inventors: 夏良超
Original assignee: 深圳壹账通智能科技有限公司
Priority date: 2019-09-06
Filing date: 2020-04-28
Publication date: 2021-03-11
Also published as: CN110675546B; CN110675546A

Abstract

Embodiments of the present application provide an invoice picture recognition and verification method, comprising: receiving an invoice picture to be verified, and recognizing the invoice type of said invoice picture; determining whether the invoice type is a preset invoice type; if the invoice type is the preset invoice type, extracting a plurality of pieces of text data and picture data in said invoice picture; performing a first verification operation on said invoice picture according to the plurality of pieces of text data and picture data to generate a first verification result; if the first verification result is that said invoice picture is a compliant picture, extracting invoice basic data from the plurality of pieces of text data and picture data; executing a second verification operation according to the invoice basic data to generate a verification conclusion form; and sending the verification conclusion form to a client to display the verification conclusion form by means of the client. The embodiments of the present application further provide an invoice picture recognition and verification system, a device and a readable storage medium. According to the embodiments of the present application, invoice recognition and verification can be completed more efficiently and conveniently.

Description

Method, system, equipment and readable storage medium for recognizing and authenticating invoice pictures

This application requires the priority of a Chinese patent application filed with the Chinese Patent Office on September 6, 2019, with an application number of 201910844520.3 and an invention title of "Invoice Picture Recognition and Verification Method, System, Equipment, and Readable Storage Medium". The entire content is incorporated into this application by reference.

Technical field

The embodiments of the present application relate to the field of big data, and in particular to a method, system, computer device, and computer-readable storage medium for identifying and verifying invoice images.

Background technique

As an important part of my country's financial system, invoices are not only an important original document for accounting, but also the main effective means for tax authorities to conduct tax accounting, collection, and tax audits. Falsified invoices will cause trouble for the company or enterprise's reimbursement, accounting and other operations, and even financial losses. Therefore, in the reimbursement process of a company or enterprise, it is necessary for the relevant financial department to verify the authenticity of the invoice.

At present, in the reimbursement process of most companies or enterprises, the verification operation of the invoice by the financial department of the company or enterprise usually requires the financial staff to manually enter the invoice code, invoice number, invoice date, and issued amount (excluding tax) , Basic information such as the identification number of the issuing party, and then check the national tax website to determine the authenticity of the invoice. However, the inventor realized that using the above method to verify the invoices has the following problems: the first is that the query efficiency is low, and manual input is easy to make mistakes; the second is that when financial staff perform financial work such as reimbursement and entry, they need to start from In the batch of invoices, the invoices that meet the requirements are screened out, and the authenticity operation is performed, which is inefficient.

Therefore, how to efficiently and conveniently complete the identification and verification of invoices has become an urgent problem to be solved at present.

Summary of the invention

In view of this, the embodiments of this application provide a method, system, computer equipment, and computer-readable storage medium for identifying and verifying invoice pictures, which are used to solve the problem that financial personnel can select and verify invoices that meet the requirements from a large number of invoices. Really, the problem of inefficiency.

The embodiments of this application solve the above technical problems through the following technical solutions:

An invoice picture recognition and authenticity method, including:

Receiving the genuine invoice picture to be verified provided by the user through the client, and identifying the invoice type of the genuine invoice picture to be verified;

Determine whether the invoice type is a preset invoice type;

If the invoice type is a preset invoice type, extract multiple text data and image data in the original invoice picture to be verified;

Performing a first authenticity verification operation on the authentic invoice picture to be verified according to the plurality of text data and image data to generate a first authenticity verification result;

If the first verification result is that the true invoice picture to be verified is a compliant picture, extract basic invoice data from the multiple text data and picture data;

Perform a second verification operation according to the basic invoice data to generate a verification conclusion form;

The verification conclusion form is sent to the client, so that the verification conclusion form is displayed on the corresponding display interface of the client through the client.

In order to achieve the above objective, the embodiment of the present application also provides an invoice picture recognition and authenticity verification system, including:

The receiving module is used to receive the genuine invoice picture to be verified provided by the user through the client, and to identify the invoice type of the genuine invoice picture to be verified;

Judging module; used to determine whether the invoice type is a preset invoice type;

The first extraction module is configured to extract multiple text data and image data in the original invoice picture to be verified if the invoice type is a preset invoice type;

The first authenticity verification module is configured to perform a first authenticity verification operation on the to-be-verified authentic invoice picture according to the plurality of text data and image data to generate a first authenticity verification result;

The second extraction module is configured to extract basic invoice data from the plurality of text data and image data if the first verification result is that the original invoice picture to be verified is a compliant picture;

The second authenticity verification module is used to perform a second authenticity verification operation according to the basic data of the invoice to generate a verification conclusion form;

The display module is configured to send the verification conclusion form to the client, so as to display the verification conclusion form on the corresponding display interface of the client through the client.

In order to achieve the foregoing objective, an embodiment of the present application further provides a computer device. The computer device includes a memory, a processor, and a computer program stored on the memory and running on the processor, and the processor executes the The computer program realizes the steps of the method for recognizing and authenticating the invoice picture as described above.

In order to achieve the foregoing objective, the embodiments of the present application also provide a computer-readable storage medium having a computer program stored in the computer-readable storage medium, and the computer program may be executed by at least one processor, so that the at least A processor executes the steps of the invoice picture recognition and authenticity method described above.

The method, system, computer equipment, and computer-readable storage medium for identifying and authenticating invoice pictures provided in the embodiments of this application are used to identify and determine the invoice type of the authentic invoice picture to be verified before the authenticity verification operation is performed on the authentic invoice picture. According to the invoice type, the financial staff can more efficiently screen out the invoices that meet the requirements from the large batch of invoices when performing financial work such as reimbursement and entry; and after judging the invoice type of the true invoice picture to be verified, pass the invoice face The first verification operation of comparing the text data with the picture data and the second verification operation based on the basic invoice data extracted from the invoice picture, and the two verification operations of the invoice picture to be verified can enable the invoice to be verified The image verification result is more accurate.

The following describes the application in detail with reference to the accompanying drawings and specific embodiments, but it is not intended to limit the application.

Description of the drawings

Fig. 1 is a flowchart of the steps of the method for identifying and authenticating invoice pictures in the first embodiment of the application;

FIG. 2 is a schematic diagram of a specific flow of step S100 in FIG. 1;

FIG. 3 is a schematic diagram of a specific flow of step S100A in FIG. 2;

FIG. 4 is a schematic diagram of a specific flow of step S100B in FIG. 2;

FIG. 5 is a schematic diagram of a specific flow of step S106 in FIG. 1;

FIG. 6 is a schematic diagram of a specific flow of step S110 in FIG. 1;

FIG. 7 is a schematic diagram of the program modules of the second embodiment of the system for identifying and authenticating pictures of application invoices;

FIG. 8 is a schematic diagram of the hardware structure of the third embodiment of the computer equipment of this application.

detailed description

In order to make the purpose, technical solutions, and advantages of this application clearer, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present application, and are not used to limit the present application. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative work shall fall within the protection scope of this application.

The technical solutions between the various embodiments can be combined with each other, but they must be based on what can be achieved by a person of ordinary skill in the art. When the combination of technical solutions is contradictory or cannot be achieved, it should be considered that such a combination of technical solutions does not exist. It is not within the scope of protection required by this application.

Example one

Please refer to FIG. 1, which shows a flow chart of the method for recognizing and authenticating an invoice picture according to an embodiment of the present application. It can be understood that the flowchart in this method embodiment is not used to limit the order of execution of the steps. The following is an exemplary description with computer equipment as the main body of execution, and the details are as follows:

Step S100: Receive a genuine invoice picture to be verified provided by the user through the client, and identify the invoice type of the genuine invoice picture to be verified.

As shown in FIG. 2, in an exemplary embodiment, step S100 may further include:

In step S100A, a plurality of convolutional features are extracted from the authentic invoice image to be verified through a deep residual network (Deep Residual Network, DRN for short) model.

Specifically, the deep residual network model includes an input layer, a primary convolutional layer, multiple residual modules, a fully connected layer, and an output layer; the primary function of the primary convolutional layer is to convolve the input original data. Product; multiple residual modules are used to extract the convolutional features of the above-mentioned convolutional data; the fully connected layer has multiple nodes, and the main function is to classify the above-mentioned convolutional features.

Each residual module is divided into a main path and a shortcut, and re-superimposed and integrated at the end, that is, a residual module is composed of two layers of convolution and an identity map. A residual module can be expressed as y=F(x+{W _i })+x, where x represents the output obtained through the shortcut, W _i represents the weight of the residual module, and F(x+{W _i }) is the output The output from the main path, y is the sum of the output from the main path and the shortcut.

In this embodiment, referring to FIG. 3, step S100A may further include the following steps:

Step S100A1, input the original invoice picture to be verified into the input layer.

In step S100A2, the initial convolution layer performs a convolution operation on the true invoice picture to be verified in the input layer to obtain multiple convolution feature maps.

In step S100A3, multiple residual modules are used to extract multiple convolution features in the multiple convolution feature maps.

In step S100A4, multiple convolutional features are input to the fully connected layer for classification.

Step S100A5, output multiple convolution features through the output layer.

Step S100B, performing dimensionality reduction processing on the multiple convolution features to obtain multiple low-dimensional features.

Following the above example, the principal component analysis (PCA) method can be used to perform dimensionality reduction operations on the convolutional features extracted by the deep residual network model. Please refer to Figure 4 for details as follows:

In step S100B1, a 128-dimensional feature vector matrix is composed of multiple convolution features, and one convolution feature is expressed as a 128-dimensional feature vector;

Step S100B2, calculating the covariance matrix of the 128-dimensional eigenvector matrix;

Step S100B3: Decompose the eigenvalues of the covariance matrix, and select 64 eigenvalues corresponding to the eigenvalues to form a projection matrix according to the size of the eigenvalues;

Step S100B4, project the 128-dimensional feature vector matrix to the projection matrix to obtain a 64-dimensional feature vector matrix, the 64-dimensional feature vector matrix includes a plurality of low-dimensional features, and a low-dimensional feature is represented as a 64-dimensional feature vector .

Since the image information is concentrated in the feature vector with a large feature value, discarding the feature vector with a small feature value will not affect the image quality. Therefore, the eigenvalues of the covariance matrix are sorted from large to small, and the eigenvectors corresponding to the first 64 eigenvalues are selected to obtain multiple low-dimensional features after dimensionality reduction.

Step S100C, calculating the Euclidean distance between the multiple low-dimensional features and the feature of each picture in the pre-configured multi-view image database.

Specifically, the pre-configured multi-view image database contains invoice images taken at different angles for each invoice type.

The formula for calculating Euclidean distance is:

Where x and y represent two n-dimensional vectors: x (x ₁ , x ₂ ,..., x _n ), y (y ₁ , y ₂ ,..., y _n ).

In step S100D, a reference invoice picture in the multi-view image database is selected, and the Euclidean distance between the reference invoice picture and the invoice picture to be verified is greater than the other invoice pictures in the multi-view database and the authentic invoice to be verified Euclidean distance between.

Step S100E: Send a query request to the multi-view image database, where the query request is used to request the multi-view image database to query the reference invoice type of the reference invoice picture.

Step S100F, receiving the reference invoice type returned by the multi-view image database.

In step S100G, the reference invoice type is determined as the invoice type of the genuine invoice picture to be verified.

Step S102: Determine whether the invoice type is a preset invoice type.

Step S104, if the invoice type is a preset invoice type, extract multiple text data and image data in the invoice picture to be verified.

Specifically, the text data includes: text content, text attributes (font, font size, etc.), the position of the text content in the layout, and other data; the image data includes: image attributes (resolution, image color gamut, channel, etc.) , The position of the picture in the layout and other data.

Step S106: Perform a first authenticity verification operation on the authentic invoice image to be verified according to the multiple text data and image data to generate a first authenticity verification result.

As shown in FIG. 5, in an exemplary embodiment, step S106 may further include:

Step S106A, extracting the picture data of the invoice supervisor chapter picture data, the picture data of the special invoice chapter picture data of the seller and the two-dimensional code picture data from the plurality of picture data.

Specifically, the picture data of the invoice supervisor chapter picture data, the picture data of the special invoice chapter of the seller and the picture data of the two-dimensional code are extracted from the plurality of picture data in a traversal manner; wherein the two-dimensional code is an encrypted two-dimensional code.

Step S106B: parse the two-dimensional code image data to obtain a plurality of first field data, and match the plurality of first field data with a plurality of text field data in the plurality of text data to generate a text match Result data.

Specifically, if the matches are consistent, the generated text matching result data is represented as the text matching consistency; if the matches are inconsistent, the generated text matching result data is represented as the text matching inconsistency.

In an exemplary embodiment, the specific operation of parsing the image data of the two-dimensional code is as follows:

1.1. Invoke the zxing scanning program, scan the two-dimensional code image data through the zxing scanning program to obtain encrypted data and the public key data of the invoice issuer;

1.2. Decrypt and compare the encrypted data;

1.3. Judge whether the decryption is successful; if the decryption is successful, go to step 1.4; if the decryption is unsuccessful, then generate the decryption failure conclusion data and feed it back to the client.

1.4. Extract multiple first field data in the two-dimensional code image data according to the scanning result.

Step S106C: parse the picture data of the invoice production supervision chapter to obtain multiple second field data, and match the multiple second field data with multiple production supervision chapter fields pre-configured based on the national unified invoice production supervision chapter to generate production supervision chapter Chapter matching result data.

In an exemplary embodiment, specifically, if the matches are consistent, the generated production supervision chapter matching result data is represented as the production supervision chapter matching consistency; if the matches are inconsistent, the generated production supervision chapter matching result data is represented as the production supervision chapter matching is inconsistent.

The OCR recognition technology may be used to parse the picture data of the invoice supervisory chapter to obtain multiple second field data.

Step S106D: parse the picture data of the special invoice chapter of the seller to obtain a plurality of third field data, and match the plurality of third fields with the plurality of invoice chapter field data in each picture in the pre-configured invoice chapter database , To generate invoice-specific chapter matching result data.

Specifically, if the matches are consistent, the generated invoice-specific chapter matching result data is represented as the invoice-specific chapter matching is consistent; if the matches are inconsistent, the generated invoice-specific chapter matching result data is represented as the invoice-specific chapter matching inconsistency.

In an exemplary embodiment, OCR recognition technology may also be used to parse the picture data of the seller's invoice special chapter to obtain multiple third field data.

Step S106E: If the text matching result data, the production supervisor chapter matching result data, and the invoice special chapter matching result data all indicate that the matching is consistent, a first verification result is generated, and the first verification result is the true picture to be verified It is a compliance picture.

Step S106F, if any one of the text matching result data, the production supervision chapter matching result data, and the invoice special chapter matching result data indicates that the matching is inconsistent, a first verification result is generated, and the first verification result is all The real picture to be verified is a non-compliant picture.

Step S108: If the first authenticity verification result is that the authentic invoice picture to be verified is a compliant picture, extract basic invoice data from the multiple text data and image data.

Specifically, the basic invoice data includes multiple characteristic data such as invoice code, invoice number, total amount (excluding tax), invoice date, invoice verification code, and so on.

Step S110: Perform a second verification operation according to the basic invoice data to generate a verification conclusion form.

Before performing the second verification operation, it also includes obtaining the docking interface between the computer equipment and the electronic invoice verification system, as follows:

2.1. Send an acquisition request to the electronic invoice verification system, where the acquisition request is used to acquire the verification code image data of the electronic invoice verification system;

2.2. Analyze the verification code image data to extract the verification code information;

2.3. Obtain the docking interface through the verification code information, and send the basic invoice data to the electronic invoice verification system based on the docking interface.

Referring to FIG. 6, in step S110, it may further include:

Step S110A: Send the basic invoice data and the related data query instruction to the electronic invoice verification system. The related data query instruction is used to request the electronic invoice verification system to query multiple related relationships based on the image of the invoice to be verified. data;

Step S110B, receiving multiple associated data returned by the electronic invoice verification system;

Step S110C, extract multiple inspection parameters from the multiple associated data;

Specifically, the multiple query parameters include, but are not limited to, machine number, specification model, invoice number, invoice code, invoice date, check code, amount, unit, quantity, unit price, amount, tax rate, tax amount, and buyer's area information (Name, taxpayer identification number, account opening bank and account number, address, telephone), seller area information (name, taxpayer identification number, account opening bank and account number, address, telephone), remarks, password area, payee, review , Invoicing person, etc.

In step S110D, the final comparison result of the basic invoice data and a plurality of verification parameters is received to generate a verification conclusion form.

Specifically, if the final comparison result indicates that the basic invoice data is consistent with multiple inspection parameters, the verification conclusion data in the verification conclusion form generated for the authentic invoice picture to be verified is indicated as the authenticity to be verified. The invoice picture is a compliant invoice picture.

If the final comparison result indicates that the basic invoice data is inconsistent with multiple inspection parameters, the verification conclusion data in the verification conclusion form generated for the authentic invoice image to be verified is expressed as the authentic invoice image to be verified Picture of non-compliant invoice.

Step S112, sending the authenticity verification conclusion form to the client, so that the authenticity verification conclusion form is displayed on the corresponding display interface of the client through the client.

Example two

Please continue to refer to Figure 7, which shows a schematic diagram of the program modules of the invoice picture recognition and authenticity verification system of this application. In this embodiment, the invoice picture recognition and verification system 20 may include or be divided into one or more program modules, and the one or more program modules are stored in a storage medium and executed by one or more processors. , In order to complete this application and realize the above-mentioned method of identifying and authenticating invoice pictures The program module referred to in the embodiments of the present application refers to a series of computer program instruction segments capable of completing specific functions, and is more suitable for describing the execution process of the invoice picture recognition and verification system 20 in the storage medium than the program itself. The following description will specifically introduce the functions of each program module in this embodiment:

The receiving module 200 is configured to receive the genuine invoice picture to be verified provided by the user through the client, and to identify the invoice type of the genuine invoice picture to be verified.

Further, the receiving module 200 is also used for:

Extracting multiple convolutional features from the true invoice picture to be verified through a deep residual network model;

Performing dimensionality reduction processing on the multiple convolution features to obtain multiple low-dimensional features;

Calculating the Euclidean distance between the multiple low-dimensional features and the feature of each picture in the pre-configured multi-view image database;

Select the reference invoice picture in the multi-view image database, and the Euclidean distance between the reference invoice picture and the invoice picture to be verified is greater than the distance between the other invoice pictures in the multi-view database and the genuine invoice to be verified Euclidean distance

Sending a query request to the multi-view image database, where the query request is used to request the multi-view image database to query the reference invoice type of the reference invoice picture;

Receiving the reference invoice type returned by the multi-view image database;

The reference invoice type is determined as the invoice type of the genuine invoice picture to be verified.

Judging module 202; used to determine whether the invoice type is a preset invoice type.

The first extraction module 204 is configured to extract multiple text data and image data in the picture of the genuine invoice to be verified if the invoice type is a preset invoice type.

The first authenticity verification module 206 is configured to perform a first authenticity verification operation on the authentic invoice image to be verified according to the multiple text data and image data to generate a first authenticity verification result.

Further, the first authenticity verification module 206 is also used for:

Extracting the picture data of the invoice supervisor chapter picture data, the picture data of the seller's invoice special chapter picture data and the two-dimensional code picture data from the plurality of picture data;

Parsing the two-dimensional code image data to obtain a plurality of first field data, and matching the plurality of first field data with a plurality of text field data in the plurality of text data to generate text matching result data;

Parse the picture data of the invoice production supervision chapter to obtain multiple second field data, and match the multiple second field data with multiple production supervision chapter fields pre-configured based on the national unified invoice production supervision chapter to generate a production supervision chapter matching result data;

Analyze the picture data of the special invoice chapter of the seller to obtain a plurality of third field data, and match the plurality of third fields with the plurality of invoice special chapter field data in each picture of the pre-configured invoice special chapter database to generate Invoice special stamp matching result data;

If the text matching result data, the production supervision chapter matching result data, and the invoice special chapter matching result data all indicate that the matching is consistent, a first verification result is generated, and the first verification result is that the true picture to be verified is compliant image;

If any one of the text matching result data, the production supervision chapter matching result data, and the invoice special chapter matching result data indicates that the matching is inconsistent, a first verification result is generated, and the first verification result is the pending verification The real picture is a non-compliant picture.

The second extraction module 208 is configured to extract basic invoice data from the plurality of text data and image data if the first verification result is that the original invoice picture to be verified is a compliant picture.

The second authenticity verification module 210 is configured to perform a second authenticity verification operation according to the basic invoice data to generate a verification conclusion form.

Further, the second authenticity verification module 210 is also used for:

Sending basic invoice data and related data query instructions to the electronic invoice verification system, where the related data query instructions are used to request the electronic invoice verification system to query multiple related data based on the image of the invoice to be verified;

Receiving multiple associated data returned by the electronic invoice verification system;

Extracting multiple inspection parameters from the multiple associated data;

The final comparison result of the basic invoice data and the multiple inspection parameters is received to generate a verification conclusion form.

The display module 212 is configured to send the verification conclusion form to the client, so as to display the verification conclusion form on the corresponding display interface of the client through the client.

Optionally, the invoice picture recognition and authenticity verification system 20 further includes:

The obtaining module 214 is configured to send an obtaining request to the electronic invoice verification system, where the obtaining request is used to obtain the verification code image data of the electronic invoice verification system;

Analyzing the verification code image data to extract the verification code information;

The docking interface is obtained through the verification code information, and the docking interface is used to send basic data of the invoice to the electronic invoice verification system.

Example three

Refer to FIG. 8, which is a schematic diagram of the hardware architecture of the computer device according to the third embodiment of the present application. In this embodiment, the computer device 2 is a device that can automatically perform numerical calculation and/or information processing in accordance with pre-set or stored instructions. The computer device 2 may be a rack server, a blade server, a tower server, or a cabinet server (including an independent server or a server cluster composed of multiple servers). As shown in FIG. 8, the computer device 2 at least includes, but is not limited to, a memory 21, a processor 22, a network interface 23, and an invoice picture recognition and authenticity system 20 that can communicate with each other through a system bus. among them:

In this embodiment, the memory 21 includes at least one type of computer-readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory ( RAM), static random access memory (SRAM), read only memory (ROM), electrically erasable programmable read only memory (EEPROM), programmable read only memory (PROM), magnetic memory, magnetic disks, optical disks, etc. In some embodiments, the memory 21 may be an internal storage unit of the computer device 2, for example, a hard disk or a memory of the computer device 2. In other embodiments, the memory 21 may also be an external storage device of the computer device 2, such as a plug-in hard disk, a smart media card (SMC), and a secure digital (Secure Digital, SMC) equipped on the computer device 2. SD) card, flash card (Flash Card), etc. Of course, the memory 21 may also include both the internal storage unit of the computer device 2 and its external storage device. In this embodiment, the memory 21 is generally used to store the operating system and various application software installed in the computer device 2, such as the program code of the invoice picture recognition and verification system 20 in the third embodiment. In addition, the memory 21 can also be used to temporarily store various types of data that have been output or will be output.

The processor 22 may be a central processing unit (Central Processing Unit, CPU), a controller, a microcontroller, a microprocessor, or other data processing chips in some embodiments. The processor 22 is generally used to control the overall operation of the computer device 2. In this embodiment, the processor 22 is used to run the program code or processing data stored in the memory 21, for example, to run the invoice picture recognition and authenticity system 20 to implement the invoice picture recognition and authenticity method of the first embodiment.

The network interface 23 may include a wireless network interface or a wired network interface, and the network interface 23 is generally used to establish a communication connection between the computer device 2 and other electronic devices. For example, the network interface 23 is used to connect the computer device 2 with an external terminal through a network, and establish a data transmission channel and a communication connection between the computer device 2 and the external terminal. The network may be Intranet, Internet, Global System of Mobile Communication (GSM), Wideband Code Division Multiple Access (WCDMA), 4G network, 5G Network, Bluetooth (Bluetooth), Wi-Fi and other wireless or wired networks.

It should be pointed out that FIG. 8 only shows the computer device 2 with components 20-23, but it should be understood that it is not required to implement all the components shown, and more or fewer components may be implemented instead.

In this embodiment, the invoice picture recognition and verification system 20 stored in the memory 21 can also be divided into one or more program modules, and the one or more program modules are stored in the memory 21, and It is executed by one or more processors (the processor 22 in this embodiment) to complete the application.

For example, FIG. 7 shows a schematic diagram of the program modules of the second embodiment of the system for recognizing and verifying invoice pictures 20. In this embodiment, the system for recognizing and authenticating invoice pictures based on 20 can be divided into receiving modules 200, The judgment module 202, the first extraction module 204, the first authentication module 206, the second extraction module 208, the second authentication module 210, and the display module 212. Among them, the program module referred to in this application refers to a series of computer program instruction segments that can complete specific functions, and is more suitable than a program to describe the execution process of the invoice picture recognition and verification system 20 in the computer device 2. The specific functions of the program modules 200-212 have been described in detail in the second embodiment, and will not be repeated here.

Example four

This embodiment also provides a computer-readable storage medium, such as flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), only Readable memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disks, optical disks, servers, App application malls, etc., on which computer programs are stored, The corresponding function is realized when the program is executed by the processor. The computer-readable storage medium of this embodiment is used to store the invoice image recognition and authenticity system 20, and when executed by the processor, the invoice image recognition and authenticity method of the first embodiment is implemented. The computer-readable storage medium may be non-volatile or volatile.

The serial numbers of the foregoing embodiments of the present application are only for description, and do not represent the advantages and disadvantages of the embodiments.

Through the description of the above implementation manners, those skilled in the art can clearly understand that the above-mentioned embodiment method can be implemented by means of software plus the necessary general hardware platform, of course, it can also be implemented by hardware, but in many cases the former is better.的实施方式。

The above are only the preferred embodiments of the application, and do not limit the scope of the patent for this application. Any equivalent structure or equivalent process transformation made using the content of the description and drawings of the application, or directly or indirectly applied to other related technical fields , The same reason is included in the scope of patent protection of this application.

Claims

An invoice picture recognition and authenticity method, which includes:

Receiving the genuine invoice picture to be verified provided by the user through the client, and identifying the invoice type of the genuine invoice picture to be verified;

Determine whether the invoice type is a preset invoice type;

If the invoice type is a preset invoice type, extract multiple text data and image data in the original invoice picture to be verified;

Performing a first authenticity verification operation on the authentic invoice picture to be verified according to the plurality of text data and image data to generate a first authenticity verification result;

If the first verification result is that the true invoice picture to be verified is a compliant picture, extract basic invoice data from the multiple text data and picture data;

Perform a second verification operation according to the basic invoice data to generate a verification conclusion form;

The verification conclusion form is sent to the client, so that the verification conclusion form is displayed on the corresponding display interface of the client through the client.
The method for recognizing and authenticating an invoice picture according to claim 1, wherein the step of receiving the authentic invoice picture provided by the user through the client, and identifying the invoice type of the authentic invoice picture, further comprises:

Extracting multiple convolutional features from the true invoice picture to be verified through a deep residual network model;

Performing dimensionality reduction processing on the multiple convolution features to obtain multiple low-dimensional features;

Calculating the Euclidean distance between the multiple low-dimensional features and the feature of each picture in the pre-configured multi-view image database;

Select the reference invoice picture in the multi-view image database, and the Euclidean distance between the reference invoice picture and the invoice picture to be verified is greater than the distance between the other invoice pictures in the multi-view database and the genuine invoice to be verified Euclidean distance

Sending a query request to the multi-view image database, where the query request is used to request the multi-view image database to query the reference invoice type of the reference invoice picture;

Receiving the reference invoice type returned by the multi-view image database;

The reference invoice type is determined as the invoice type of the genuine invoice picture to be verified.
The method for recognizing and authenticating an invoice picture according to claim 2, wherein the step of extracting a plurality of convolutional features from the authentic invoice picture to be verified through a deep residual network model further comprises:

Input the picture of the true invoice to be verified into the input layer;

The initial convolution layer performs a convolution operation on the true invoice picture to be verified in the input layer to obtain multiple convolution feature maps;

Extracting multiple convolution features in the multiple convolution feature maps through multiple residual modules;

Multiple convolutional features are input to the fully connected layer for classification;

The multiple convolutional features after classification are output through the output layer.
The method for recognizing and authenticating an invoice picture according to claim 3, wherein the step of performing dimensionality reduction processing on the plurality of convolutional features to obtain a plurality of low-dimensional features further comprises:

Multiple convolution features are formed into a 128-dimensional feature vector matrix, and one convolution feature is represented as a 128-dimensional feature vector;

Calculate the covariance matrix of the 128-dimensional eigenvector matrix;

Decompose the eigenvalues of the covariance matrix, and select eigenvectors corresponding to 64 eigenvalues to form a projection matrix according to the size of the eigenvalues;

The 128-dimensional feature vector matrix is projected to the projection matrix to obtain a 64-dimensional feature vector matrix. The 64-dimensional feature vector matrix includes a plurality of low-dimensional features, and one low-dimensional feature is represented as a 64-dimensional feature vector.
The method for recognizing and authenticating an invoice picture according to claim 1, wherein a first authenticating operation is performed on the invoice picture to be authenticated according to the plurality of text data and picture data to generate a first authenticating result. Steps also include:

Extracting the picture data of the invoice supervisor chapter picture data, the picture data of the seller's invoice special chapter picture data and the two-dimensional code picture data from the plurality of picture data;

Parsing the two-dimensional code image data to obtain a plurality of first field data, and matching the plurality of first field data with a plurality of text field data in the plurality of text data to generate text matching result data;

Parse the picture data of the invoice production supervision chapter to obtain multiple second field data, and match the multiple second field data with multiple production supervision chapter fields pre-configured based on the national unified invoice production supervision chapter to generate a production supervision chapter matching result data;

Analyze the picture data of the special invoice chapter of the seller to obtain a plurality of third field data, and match the plurality of third fields with the plurality of invoice special chapter field data in each picture of the pre-configured invoice special chapter database to generate Invoice special stamp matching result data;

If the text matching result data, the production supervision chapter matching result data, and the invoice special chapter matching result data all indicate that the matching is consistent, the first verification result is that the true picture to be verified is a compliant picture;

If any one of the text matching result data, the production supervision chapter matching result data, and the invoice special chapter matching result data indicates that the matching is inconsistent, the first verification result is that the true picture to be verified is a non-compliant picture .
The method for recognizing and authenticating an invoice picture according to claim 1, wherein before the step of performing the second authenticating operation according to the basic data of the invoice, the method further comprises:

Sending an acquisition request to the electronic invoice verification system, where the acquisition request is used to acquire the verification code image data of the electronic invoice verification system;

Analyzing the verification code image data to extract the verification code information;

The docking interface is obtained through the verification code information, and the docking interface is used to send the basic data of the invoice to the electronic invoice verification system.
The method for identifying and authenticating an invoice picture according to claim 1, wherein the step of performing a second authenticity verification operation according to the basic data of the invoice to generate an authenticity verification conclusion form further comprises:

Sending basic invoice data and related data query instructions to the electronic invoice verification system, where the related data query instructions are used to request the electronic invoice verification system to query multiple related data based on the image of the invoice to be verified;

Receiving multiple associated data returned by the electronic invoice verification system;

Extracting multiple inspection parameters from the multiple associated data;

The final comparison result of the basic invoice data and the multiple inspection parameters is received to generate a verification conclusion form.
A computer device, the computer device includes a memory, a processor, and a computer program stored on the memory and running on the processor, wherein the processor implements the following steps when the processor executes the computer program:

Receiving the genuine invoice picture to be verified provided by the user through the client, and identifying the invoice type of the genuine invoice picture to be verified;

Determine whether the invoice type is a preset invoice type;

If the invoice type is a preset invoice type, extract multiple text data and image data in the original invoice picture to be verified;

Performing a first authenticity verification operation on the authentic invoice picture to be verified according to the plurality of text data and image data to generate a first authenticity verification result;

If the first verification result is that the true invoice picture to be verified is a compliant picture, extract basic invoice data from the multiple text data and picture data;

Perform a second verification operation according to the basic invoice data to generate a verification conclusion form;

The verification conclusion form is sent to the client, so that the verification conclusion form is displayed on the corresponding display interface of the client through the client.
8. The computer device according to claim 8, wherein, in the step of receiving a genuine invoice picture to be verified provided by the user through the client, and identifying the invoice type of the genuine invoice picture to be verified, the processor further executes the computer program achieve:

Extracting multiple convolutional features from the true invoice picture to be verified through a deep residual network model;

Performing dimensionality reduction processing on the multiple convolution features to obtain multiple low-dimensional features;

Calculating the Euclidean distance between the multiple low-dimensional features and the feature of each picture in the pre-configured multi-view image database;

Select the reference invoice picture in the multi-view image database, and the Euclidean distance between the reference invoice picture and the invoice picture to be verified is greater than the distance between the other invoice pictures in the multi-view database and the genuine invoice to be verified Euclidean distance

Sending a query request to the multi-view image database, where the query request is used to request the multi-view image database to query the reference invoice type of the reference invoice picture;

Receiving the reference invoice type returned by the multi-view image database;

The reference invoice type is determined as the invoice type of the genuine invoice picture to be verified.
9. The computer device according to claim 9, wherein, in the step of extracting multiple convolution features from the authentic invoice picture to be verified through a deep residual network model, the processor further executes the computer program to realize:

Input the picture of the true invoice to be verified into the input layer;

The initial convolution layer performs a convolution operation on the true invoice picture to be verified in the input layer to obtain multiple convolution feature maps;

Extracting multiple convolution features in the multiple convolution feature maps through multiple residual modules;

Multiple convolutional features are input to the fully connected layer for classification;

The multiple convolutional features after classification are output through the output layer.
The computer device according to claim 10, wherein, in the step of performing dimensionality reduction processing on the plurality of convolutional features to obtain a plurality of low-dimensional features, the processor further executes the computer program to realize:

Multiple convolution features are formed into a 128-dimensional feature vector matrix, and one convolution feature is represented as a 128-dimensional feature vector;

Calculate the covariance matrix of the 128-dimensional eigenvector matrix;

Decompose the eigenvalues of the covariance matrix, and select eigenvectors corresponding to 64 eigenvalues to form a projection matrix according to the size of the eigenvalues;

The 128-dimensional feature vector matrix is projected to the projection matrix to obtain a 64-dimensional feature vector matrix. The 64-dimensional feature vector matrix includes a plurality of low-dimensional features, and one low-dimensional feature is represented as a 64-dimensional feature vector.
8. The computer device according to claim 8, wherein the step of performing a first verification operation on the to-be-verified invoice picture according to the plurality of text data and image data to generate a first verification result, the processing The device also executes the computer program to realize:

Extracting the picture data of the invoice supervisor chapter picture data, the picture data of the seller's invoice special chapter picture data and the two-dimensional code picture data from the plurality of picture data;

Parsing the two-dimensional code image data to obtain a plurality of first field data, and matching the plurality of first field data with a plurality of text field data in the plurality of text data to generate text matching result data;

Parse the picture data of the invoice production supervision chapter to obtain multiple second field data, and match the multiple second field data with multiple production supervision chapter fields pre-configured based on the national unified invoice production supervision chapter to generate a production supervision chapter matching result data;

Analyze the picture data of the special invoice chapter of the seller to obtain a plurality of third field data, and match the plurality of third fields with the plurality of invoice special chapter field data in each picture of the pre-configured invoice special chapter database to generate Invoice special stamp matching result data;

If the text matching result data, the production supervision chapter matching result data, and the invoice special chapter matching result data all indicate that the matching is consistent, the first verification result is that the true picture to be verified is a compliant picture;

If any one of the text matching result data, the production supervision chapter matching result data, and the invoice special chapter matching result data indicates that the matching is inconsistent, the first verification result is that the true picture to be verified is a non-compliant picture .
8. The computer device according to claim 8, wherein, before the step of performing the second verification operation according to the basic data of the invoice, the processor further executes the computer program to realize:

Sending an acquisition request to the electronic invoice verification system, where the acquisition request is used to acquire the verification code image data of the electronic invoice verification system;

Analyzing the verification code image data to extract the verification code information;

The docking interface is obtained through the verification code information, and the docking interface is used to send the basic data of the invoice to the electronic invoice verification system.
8. The computer device according to claim 8, wherein, in the step of performing a second verification operation according to the basic data of the invoice to generate a verification conclusion form, the processor further executes the computer program to realize:

Sending basic invoice data and related data query instructions to the electronic invoice verification system, where the related data query instructions are used to request the electronic invoice verification system to query multiple related data based on the image of the invoice to be verified;

Receiving multiple associated data returned by the electronic invoice verification system;

Extracting multiple inspection parameters from the multiple associated data;

The final comparison result of the basic invoice data and the multiple inspection parameters is received to generate a verification conclusion form.
A computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, and the computer program can be executed by at least one processor, so that the at least one processor executes the following steps:

Receiving the genuine invoice picture to be verified provided by the user through the client, and identifying the invoice type of the genuine invoice picture to be verified;

Determine whether the invoice type is a preset invoice type;

If the invoice type is a preset invoice type, extract multiple text data and image data in the original invoice picture to be verified;

Performing a first authenticity verification operation on the authentic invoice picture to be verified according to the plurality of text data and image data to generate a first authenticity verification result;

If the first verification result is that the true invoice picture to be verified is a compliant picture, extract basic invoice data from the multiple text data and picture data;

Perform a second verification operation according to the basic invoice data to generate a verification conclusion form;

The verification conclusion form is sent to the client, so that the verification conclusion form is displayed on the corresponding display interface of the client through the client.
The computer-readable storage medium according to claim 15, wherein, in the step of receiving a genuine invoice picture to be verified provided by the user through the client, and identifying the invoice type of the genuine invoice picture to be verified, the computer program is also The implementation performed by at least one processor:

Extracting multiple convolutional features from the true invoice picture to be verified through a deep residual network model;

Performing dimensionality reduction processing on the multiple convolution features to obtain multiple low-dimensional features;

Calculating the Euclidean distance between the multiple low-dimensional features and the feature of each picture in the pre-configured multi-view image database;

Select the reference invoice picture in the multi-view image database, and the Euclidean distance between the reference invoice picture and the invoice picture to be verified is greater than the distance between the other invoice pictures in the multi-view database and the genuine invoice to be verified Euclidean distance

Sending a query request to the multi-view image database, where the query request is used to request the multi-view image database to query the reference invoice type of the reference invoice picture;

Receiving the reference invoice type returned by the multi-view image database;

The reference invoice type is determined as the invoice type of the genuine invoice picture to be verified.
The computer-readable storage medium according to claim 16, wherein the step of extracting a plurality of convolutional features from the authentic invoice image to be verified through a deep residual network model, the computer program is also processed by the at least one Implemented by the device:

Input the picture of the true invoice to be verified into the input layer;

The initial convolution layer performs a convolution operation on the true invoice picture to be verified in the input layer to obtain multiple convolution feature maps;

Extracting multiple convolution features in the multiple convolution feature maps through multiple residual modules;

Multiple convolutional features are input to the fully connected layer for classification;

The multiple convolutional features after classification are output through the output layer.
The computer-readable storage medium according to claim 17, wherein, in the step of performing dimensionality reduction processing on the plurality of convolution features to obtain a plurality of low-dimensional features, the computer program is further executed by the at least one processor Implementation implemented:

Multiple convolution features are formed into a 128-dimensional feature vector matrix, and one convolution feature is represented as a 128-dimensional feature vector;

Calculate the covariance matrix of the 128-dimensional eigenvector matrix;

Decompose the eigenvalues of the covariance matrix, and select eigenvectors corresponding to 64 eigenvalues to form a projection matrix according to the size of the eigenvalues;

The 128-dimensional feature vector matrix is projected to the projection matrix to obtain a 64-dimensional feature vector matrix. The 64-dimensional feature vector matrix includes a plurality of low-dimensional features, and one low-dimensional feature is represented as a 64-dimensional feature vector.
15. The computer-readable storage medium according to claim 15, wherein the step of performing a first verification operation on the to-be-verified invoice picture according to the plurality of text data and image data to generate a first verification result, The computer program is also executed by the at least one processor to realize:

Extracting the picture data of the invoice supervisor chapter picture data, the picture data of the seller's invoice special chapter picture data and the two-dimensional code picture data from the plurality of picture data;

Parsing the two-dimensional code image data to obtain a plurality of first field data, and matching the plurality of first field data with a plurality of text field data in the plurality of text data to generate text matching result data;

Parse the picture data of the invoice production supervision chapter to obtain multiple second field data, and match the multiple second field data with multiple production supervision chapter fields pre-configured based on the national unified invoice production supervision chapter to generate a production supervision chapter matching result data;

Analyze the picture data of the special invoice chapter of the seller to obtain a plurality of third field data, and match the plurality of third fields with the plurality of invoice special chapter field data in each picture of the pre-configured invoice special chapter database to generate Invoice special stamp matching result data;

If the text matching result data, the production supervision chapter matching result data, and the invoice special chapter matching result data all indicate that the matching is consistent, the first verification result is that the true picture to be verified is a compliant picture;

If any one of the text matching result data, the production supervision chapter matching result data, and the invoice special chapter matching result data indicates that the matching is inconsistent, the first verification result is that the true picture to be verified is a non-compliant picture .
The computer-readable storage medium according to claim 15, wherein, before the step of performing the second authenticity verification operation according to the basic invoice data, the computer program is further executed by the at least one processor to realize:

Sending an acquisition request to the electronic invoice verification system, where the acquisition request is used to acquire the verification code image data of the electronic invoice verification system;

Analyzing the verification code image data to extract the verification code information;

The docking interface is obtained through the verification code information, and the docking interface is used to send the basic data of the invoice to the electronic invoice verification system.