WO2024060759A1

WO2024060759A1 - Supply chain financial asset auditing method and apparatus, and device and medium

Info

Publication number: WO2024060759A1
Application number: PCT/CN2023/103531
Authority: WO
Inventors: 缪洲; 陈东来
Original assignee: 深圳前海环融联易信息科技服务有限公司
Priority date: 2022-09-21
Filing date: 2023-06-29
Publication date: 2024-03-28
Also published as: CN115601129A

Abstract

The present application relates to artificial intelligence technology. Provided are a supply chain financial asset auditing method and apparatus, and a device and a medium. The method comprises: performing image recognition on financial asset image data, so as to obtain position information and character information of each row of textboxes; performing multi-modal information extraction on the financial asset image data according to the recognized information, so as to obtain target information; using the target information to perform legitimacy verification on the financial asset image data; if the financial asset image data has passed the legitimacy verification, sending the target information to a configuration platform; according to feedback data of the configuration platform for the target information, querying for a target opinion from an audit opinion set; sending the target opinion to the configuration platform; and when the target opinion is selected, feeding back the target opinion. By means of the present application, automatic verification of supply chain financial assets can be realized by combining character recognition with multi-modal information extraction, thereby improving the verification efficiency; moreover, an audit opinion can be generated to assist in auditing the supply chain financial assets, thereby improving the auditing efficiency.

Description

Supply chain financial asset review methods, devices, equipment and media

This application claims priority to a Chinese patent application filed with the China Patent Office on September 21, 2022, with application number 202211156964.6 and the application name "Supply chain financial asset audit method, device, equipment and medium", the entire content of which is incorporated by reference. incorporated in this application.

Technical field

This application relates to the field of artificial intelligence technology, and in particular to a supply chain financial asset review method, device, equipment and medium.

Background technique

In the field of supply chain finance, especially for cross-border supply chain finance, applicants are aware that companies applying for loans need to upload invoices, customs declarations, contracts, purchase orders, bills of lading and other document images to prove the authenticity of the transaction.

Currently, lending institutions generally use manual review for the above loan information. Due to the large amount of data and the number of elements that need to be compared, the manual review method is very time-consuming and labor-intensive, the review efficiency is low, and it is error-prone.

Application content

The embodiments of this application provide a supply chain financial asset auditing method, device, computer equipment and storage medium, aiming to solve the problem of low efficiency and error-prone problems in supply chain financial asset data auditing.

In the first aspect, the embodiment of this application provides a supply chain financial asset audit method, which includes:

In response to the received review request for the financial asset image data, perform image recognition on the financial asset image data to obtain the position information and character information of each line of text boxes in the financial asset image data;

Perform multi-modal information extraction on the financial asset image data based on the position information and character information of each line of text box to obtain the target information;

Using the target information to verify the legality of the financial asset image data;

In response to the financial asset image data passing the legality verification, sending the target information to the configuration platform;

In response to the feedback data from the configuration platform for the target information, query the target opinions from the preconfigured review opinion set according to the feedback data;

Send the target opinion to the configuration platform;

In response to a selection instruction for the target opinion, the target opinion is fed back.

In the second aspect, embodiments of this application provide a supply chain financial asset audit device, which includes:

A recognition unit, configured to perform image recognition on the financial asset image data in response to the received review request for the financial asset image data, and obtain the position information and character information of each line of text boxes in the financial asset image data;

An extraction unit, configured to extract multimodal information from the financial asset image data according to position information and character information of each line of text boxes to obtain target information;

A verification unit configured to use the target information to verify the legality of the financial asset image data;

A sending unit, configured to send the target information to the configuration platform in response to the financial asset image data passing the legality check;

A query unit configured to respond to feedback data from the configuration platform for the target information, and query target opinions from a preconfigured set of review opinions based on the feedback data;

The sending unit is also used to send the target opinion to the configuration platform;

A feedback unit is used to feedback the target opinion in response to a selection instruction for the target opinion.

In a third aspect, an embodiment of the present application further provides a computer device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the following steps when executing the computer program:

In response to the financial asset image data passing the legality check, sending the target information to a configuration platform;

In response to the feedback data from the configuration platform for the target information, query the target opinions from the pre-configured review opinion set according to the feedback data;

Send the target opinion to the configuration platform;

In a fourth aspect, embodiments of the present application further provide a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when executed by a processor, the computer program causes the processor to perform the following operations: :

Send the target opinion to the configuration platform;

The embodiments of this application provide a method, device, equipment and medium for auditing supply chain financial assets, which can combine text recognition and multi-modal information extraction to realize automatic verification of supply chain financial assets, improve verification efficiency, and simultaneously Generate audit opinions to assist in the audit of supply chain financial assets and improve audit efficiency.

Description of drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to describe the embodiments. Obviously, the drawings in the following description are some embodiments of the present application, which are of great significance to this field. Ordinary technicians can also obtain other drawings based on these drawings without exerting creative efforts.

Figure 1 is a schematic diagram of the application scenario of the supply chain financial asset review method provided by the embodiment of this application;

Figure 2 is a schematic flow chart of the supply chain financial asset review method provided by the embodiment of this application;

Figure 3 is a schematic block diagram of a supply chain financial asset audit device provided by an embodiment of the present application;

Figure 4 is a schematic block diagram of a computer device provided by an embodiment of the present application.

Implementation

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of protection of this application.

It should be understood that, when used in this specification and the appended claims, the terms "comprises" and "comprises" indicate the presence of described features, integers, steps, operations, elements and/or components but do not exclude the presence of one or The presence or addition of multiple other features, integers, steps, operations, elements, components and/or collections thereof.

It should also be understood that the terminology used in the specification of the present application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used in this specification and the appended claims, the singular forms "a", "an" and "the" are intended to include the plural forms unless the context clearly dictates otherwise.

It will be further understood that the term "and/or" as used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items. .

Please refer to Figures 1 and 2. Figure 1 is a schematic diagram of the application scenario of the supply chain financial asset review method provided by the embodiment of the present application; Figure 2 is a flow diagram of the supply chain financial asset review method provided by the embodiment of the present application. The supply chain The financial asset audit method is applied to the server, and the method is executed through application software installed in the server.

As shown in Figure 2, the method includes steps S101 to S107.

S101. In response to the received audit request for financial asset image data, perform image recognition on the financial asset image data to obtain the position information and character information of each line of text boxes in the financial asset image data.

In this embodiment, the technical solution can be described using the server as the execution subject. The client used by the user (such as smart phones, tablets and other smart terminals) can interact with the server for data. For example, the server provides a supply chain financial asset review platform, and the user can log in to the supply chain financial asset review platform using the client. The user interactive interface of the supply chain financial asset review platform is displayed on the terminal interface of the user terminal, and there is at least one image upload interface in the user interactive interface. When the user selects an image as financial asset image data and uploads it to the server from the image upload interface, subsequent supply chain financial asset review can be performed in the server.

In this embodiment, the review request may be triggered by the loan applicant. Specifically, when a loan applicant uploads the financial asset image data, it is determined that the review request is triggered. Wherein, the financial asset image data may be in jpg, png or pdf format.

In this embodiment, OCR (Optical Character Recognition, optical character recognition) technology can be used to perform image recognition on the financial asset image data.

For example, the DBNet model can be used to perform text recognition on the financial asset image data to detect text information in the financial asset image data and generate text positioning information. Further, CRNN (Convolutional Recurrent Neural Network) is used to extract the single line text information of the located text, and the position information and character information of each line of text box in the financial asset image data are obtained.

S102. Extract multi-modal information from the financial asset image data according to the position information and character information of each line of text box to obtain target information.

For example, the SDMGR model can be used to read the position information and character information of the text box to obtain the field to which the text box belongs. That is, the text box is regarded as a node, and the label of each node and the distance between two adjacent nodes are obtained through the graph neural network. Relationship.

Specifically, multi-modal information extraction is performed on the financial asset image data based on the position information and character information of each line of text boxes to obtain target information, including:

Feature extraction is performed on the character information based on the position information of each line of text box to obtain the first feature corresponding to the graph node, where the first feature is an inter-modal feature composed of three modal features: text, visual and layout. generated by fusion;

The character information is abstracted to obtain an initial graph, where the initial graph is described by graph nodes, edges between nodes, and an adjacency matrix of the graph;

Partition the initial graph by clustering the graph nodes in the initial graph, and perform multiple iterative updates on the allocation matrix used for partitioning during the partitioning process;

Obtain the cluster representation and allocation matrix for each iterative update in the multiple iterative update process;

The first feature is updated by using the cluster representation and the allocation matrix during each iteration to obtain a second feature;

Perform node classification and link prediction based on the second feature to obtain the target information.

On the basis of text recognition, this embodiment further combines multi-modal information extraction technology to extract key information from the financial asset image data as the target information to assist in subsequent audits of supply chain financial assets.

S103. Use the target information to perform legality verification on the financial asset image data.

In this embodiment, the legality verification includes, but is not limited to: verification of the format of the financial asset image data, verification of the financial asset image data of a specified type (such as invoice type data), Verification of specified information, etc.

Specifically, using the target information to perform legality verification on the financial asset image data includes:

Determine whether the financial asset image data meets the format requirements according to the target information;

When the financial asset image data does not meet the format requirements, refuse to accept the review request for the financial asset image data;

Issue prompt information, where the prompt information is used to prompt that the financial asset image data has a format error and to prompt to re-upload the financial asset image data.

Among them, different types of financial asset image data can correspond to different format requirements to unify the formats of various types of documents.

By passing the format check and issuing effective prompts in a timely manner, you can respond quickly when there are format errors to improve review efficiency.

Further, when the financial asset image data meets the format requirements, detect the type of the financial asset image data;

When the financial asset image data is an invoice type, using the target information to verify the legality of the financial asset image data also includes, but is not limited to, one or a combination of the following verification methods:

(1) Detect whether the transaction amount of the financial asset image data is a positive number. When the transaction amount is not a positive number, an error in the transaction amount will be prompted.

At the same time, when the transaction amount is a positive number, it means that the transaction amount is correct and no prompt will be issued.

(2) Obtain the transaction party from the financial asset image data, perform authenticity verification on the transaction party, and when the transaction party fails the authenticity verification, prompt the transaction party that the information is incorrect.

The transaction parties may include buyers and sellers.

In this embodiment, the authenticity of the transaction party can be verified by calling a third-party platform (such as an enterprise query platform) or by querying local filings.

(3) Detect the number of loans corresponding to a transaction in the financial asset image data. When the number of loans is greater than or equal to one, it will prompt that the invoice number already exists.

This embodiment can avoid the situation of applying for multiple loans with one transaction by detecting the number of loans corresponding to one transaction.

(4) Detect whether the date format in the financial asset image data is a configuration format. When the date format is not the configuration format, a date format error is prompted.

The configuration format may be a universal date format, and the date format may be customized, which is not limited by this application.

(5) Obtain preconfigured business rules, and detect whether the financial asset image data complies with the business rules. When the financial asset image data does not comply with the business rules, a prompt is issued that the business requirements are not met.

Wherein, the business rules can be configured according to actual business requirement data.

This embodiment can achieve targeted verification of invoice-type financial asset image data to avoid information errors that affect subsequent asset audits.

Of course, this embodiment can also perform targeted verification on other types of financial asset image data, such as customs declarations, contracts, purchase orders, bills of lading, etc., which will not be described again here.

S104. In response to the financial asset image data passing the legality verification, send the target information to the configuration platform.

In this embodiment, when the financial asset image data fails the legality check, a prompt error message is issued, and a prompt may be given to re-upload the financial asset image data after modification.

In this embodiment, before sending the target information to the configuration platform, the method further includes:

Get pre-configured information matching rules;

Obtain configuration information from the target information according to the information matching rules;

Detect whether the configuration information satisfies the information matching rules and obtain a matching result;

Send the matching results to the configuration platform.

The configuration information may be predefined key information, such as the buyer, seller and amount on the invoice and the order. Further, according to the information matching rule, the buyer, seller and amount on the invoice and the order must match.

Wherein, the configuration platform may include a review platform for financial asset image data.

In this embodiment, before sending the target information to the configuration platform, the audit results of the configured key information are first sent to assist in faster auditing and improve audit efficiency.

Further, the target information is sent to the configuration platform so that the configuration platform can review the target information.

S105. In response to the feedback data of the configuration platform for the target information, query the target opinions from the preconfigured review opinion set according to the feedback data.

In this embodiment, the feedback data may be keywords uploaded by the loan reviewer for querying review opinions.

In this embodiment, querying the target opinion from a pre-configured review opinion set according to the feedback data includes:

determining the feedback data as keywords;

Use the keywords to perform a query in the review opinion collection to obtain the target opinion;

Wherein, the set of review opinions is a set obtained by clustering based on historical review opinions.

For example: the review opinions generated through keyword comparison can be selected by reviewers. For example, the reasons for rejecting the application can be:

1. Buyer names do not match and indicate in which documents there are mismatched key fields;

2. Seller name does not match;

3． Invoice numbers do not match;

4. Invoice dates do not match;

5. Product type does not match;

6. Transaction amount does not match;

7. Other rules set by the system do not match.

In this embodiment, by establishing a set of review opinions, the review opinions can be automatically matched based on the feedback keywords to assist the loan auditor in giving review opinions more conveniently and quickly, and improve the efficiency of supply chain financial asset review.

S106. Send the target opinion to the configuration platform.

In this embodiment, after sending the target opinion to the configuration platform, the platform can wait for the relevant reviewers to make a selection according to actual needs to determine whether to use the target opinion as the final review opinion.

In this embodiment, after sending the target opinion to the configuration platform, the method further includes:

In response to the received update request for the target opinion, obtain update data for the target opinion, optimize the target opinion based on the update data, and update the optimized target opinion to the Collection of review comments.

Through the above embodiment, the review opinion set can be updated according to the review opinions of the loan auditor, so that the opinions in the review opinion set are more accurate.

and/or in response to receiving a rejection instruction for the target opinion, receive the uploaded opinion, and provide feedback on the received opinion.

For example: when there is an error in text recognition or information extraction, and the auditor does not agree with the audit opinions generated by the system, he can reject the system's audit opinions.

Through the above embodiment, when the loan reviewer rejects the target opinion, the opinion uploaded by the loan reviewer can be used as the final review opinion, thereby avoiding review errors caused by the system opinion not meeting the requirements.

S107. In response to the selection instruction for the target opinion, feed back the target opinion.

In this embodiment, the selection instruction can be triggered by relevant staff, such as a loan reviewer.

In this embodiment, the target opinion may be fed back to the person who triggered the review request (such as a loan applicant). For example, the target opinion may be fed back to the terminal device such as a mobile phone or tablet of the loan applicant that triggered the review request. The entire review process can include all processes of supply chain finance such as lending and repayment. It automatically checks whether the files uploaded by users meet the format requirements and proposes error reasons, reducing the possibility of errors in user uploaded files. At the same time, it integrates document image recognition and asset review, For all supply chain finance processes such as asset management, lending, and repayment, applicants and reviewers can view and search previously uploaded or reviewed documents to facilitate file management. Reviewers can select final review opinions from the review opinions generated by the system, improving Audit efficiency while improving customer satisfaction.

In this embodiment, functional modules such as downloading, file search, loan status search, sorting, filtering, and batch export can also be provided for relevant users to facilitate file management by personnel with relevant permissions. For example: loan applicants can see all the files they have uploaded, download previously uploaded files, and search for previously uploaded files through various elements, which plays a role in file management. Loan reviewers can see and manage all documents they have reviewed or will review.

It can be seen from the above technical solutions that this embodiment can combine text recognition and multi-modal information extraction to realize automatic verification of supply chain financial assets, improve the verification efficiency, and can generate audit opinions to assist in supply chain finance. Review of assets to improve review efficiency.

The embodiment of the present application also provides a supply chain financial asset auditing device, which is used to execute any embodiment of the supply chain financial asset auditing method. Specifically, please refer to Figure 3, which is a schematic block diagram of the supply chain financial asset audit device 100 provided by an embodiment of the present application.

Among them, as shown in Figure 3, the supply chain financial asset audit device 100 includes an identification unit 101, an extraction unit 102, a verification unit 103, a sending unit 104, a query unit 105, and a feedback unit 106.

Wherein, the recognition unit 101 is configured to perform image recognition on the financial asset image data in response to the received review request for the financial asset image data, and obtain the position information of each line of text boxes in the financial asset image data. and character information.

The extraction unit 102 is used to extract multimodal information from the financial asset image data according to the position information and character information of each line of text box to obtain target information.

Specifically, the extraction unit 102 performs multi-modal information extraction on the financial asset image data based on the position information and character information of each row of text boxes to obtain target information, including:

Obtaining a cluster representation and an allocation matrix during each iterative update in the multiple iterative update processes;

Update the first feature using the cluster representation and allocation matrix in each iteration update to obtain the second feature;

The verification unit 103 is configured to use the target information to perform legality verification on the financial asset image data.

Specifically, the verification unit 103 uses the target information to perform legality verification on the financial asset image data, including:

When the financial asset image data is an invoice type, the verification unit 103 uses the target information to verify the legality of the financial asset image data, which also includes, but is not limited to, one or more of the following verification methods: Combination of ways:

The transaction parties may include buyers and sellers.

Among them, the configuration format can be a common date format, and the date format can be custom configured, which is not limited in this application.

(5) Obtain preconfigured business rules and detect whether the financial asset image data conforms to the business rules. When the financial asset image data does not conform to the business rules, a prompt is issued that the business requirements are not met.

This embodiment can implement targeted verification of invoice-type financial asset image data to avoid information errors that may affect subsequent asset review.

The sending unit 104 is configured to send the target information to the configuration platform in response to the financial asset image data passing the legality check.

In this embodiment, before sending the target information to the configuration platform, preconfigured information matching rules are obtained;

Send the matching results to the configuration platform.

The configuration information may be predefined key information, such as the buyer, seller and amount on the invoice and order. Further, according to the information matching rules, the buyer, seller and amount on the invoice and order must match.

Wherein, the configuration platform may include an audit platform for financial asset image data.

The query unit 105 is configured to respond to feedback data from the configuration platform for the target information, and query target opinions from a preconfigured set of review opinions based on the feedback data.

In this embodiment, the query unit 105 queries the target opinions from a preconfigured set of review opinions based on the feedback data, including:

Determine the feedback data as keywords;

Use the keyword to perform a query in the review opinion collection to obtain the target opinion;

The audit opinion set is a set obtained by clustering historical audit opinions.

2. The seller name does not match;

3． Invoice numbers do not match;

4. Invoice dates do not match;

5. Product type does not match;

6. Transaction amount does not match;

7. Other rules set by the system do not match.

In this embodiment, by establishing a set of review opinions, the review opinions can be automatically matched according to the feedback keywords, so as to assist the loan auditor in giving review opinions more conveniently and quickly, and improve the efficiency of supply chain financial asset review.

The sending unit 104 is also used to send the target opinion to the configuration platform.

In this embodiment, after sending the target opinion to the configuration platform, the relevant reviewers can wait for selection based on actual needs to determine whether to use the target opinion as the final review opinion.

In this embodiment, after sending the target opinion to the configuration platform, the sending unit 104 obtains update data for the target opinion in response to the received update request for the target opinion. According to the Update data to optimize the target opinion, and update the optimized target opinion to the review opinion set.

and/or in response to the receipt of a rejection instruction for the target opinion, receive the uploaded opinion, and provide feedback on the received opinion.

The feedback unit 106 is configured to feedback the target opinion in response to a selection instruction for the target opinion.

The above supply chain financial asset audit device can be implemented in the form of a computer program, and the computer program can run on the computer device as shown in Figure 4.

Please refer to Figure 4, which is a schematic block diagram of a computer device provided by an embodiment of the present application. The computer device 500 is a server or a server cluster. The server can be an independent server, or it can provide cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, and content delivery networks (Content Delivery Network, CDN), as well as cloud servers for basic cloud computing services such as big data and artificial intelligence platforms.

Referring to FIG. 4 , the computer device 500 includes a processor 502 , a memory, and a network interface 505 connected through a device bus 501 , where the memory may include a storage medium 503 and an internal memory 504 .

The storage medium 503 can store an operating system 5031 and a computer program 5032. When the computer program 5032 is executed, it can cause the processor 502 to execute the supply chain financial asset audit method.

The processor 502 is used to provide computing and control capabilities to support the operation of the entire computer device 500 .

The internal memory 504 provides an environment for the execution of the computer program 5032 in the storage medium 503. When the computer program 5032 is executed by the processor 502, it can cause the processor 502 to execute the supply chain financial asset audit method.

The network interface 505 is used for network communication, such as providing transmission of data information, etc. Those skilled in the art can understand that the structure shown in Figure 4 is only a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation on the computer device 500 to which the solution of the present application is applied. The specific computer device 500 may include more or fewer components than shown, some combinations of components, or a different arrangement of components.

The processor 502 is used to run the computer program 5032 stored in the memory to implement the supply chain financial asset audit method disclosed in the embodiment of this application.

Those skilled in the art can understand that the embodiment of the computer device shown in Figure 4 does not constitute a limitation on the specific configuration of the computer device. In other embodiments, the computer device may include more or fewer components than shown in the figure. Or combining certain parts, or different parts arrangements. For example, in some embodiments, the computer device may only include a memory and a processor. In such an embodiment, the structure and function of the memory and processor are consistent with the embodiment shown in FIG. 4 and will not be described again.

It should be understood that in this embodiment of the present application, the processor 502 may be a central processing unit (CPU), and the processor 502 may also be other general-purpose processors, digital signal processors (Digital Signal Processor, DSP), Application Specific Integrated Circuit (ASIC), off-the-shelf programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general processor may be a microprocessor or the processor may be any conventional processor.

In another embodiment of the present application, a computer-readable storage medium is provided. The computer-readable storage medium may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. The computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, the supply chain financial asset audit method disclosed in the embodiment of the present application is implemented.

It should be noted that the data involved in this case were all obtained legally.

Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the above-described equipment, devices and units can be referred to the corresponding processes in the foregoing method embodiments, and will not be described again here. Those of ordinary skill in the art can appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented with electronic hardware, computer software, or a combination of both. In order to clearly illustrate the relationship between hardware and software Interchangeability, in the above description, the composition and steps of each example have been generally described according to functions. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technology solution. Skilled artisans may implement the described functionality using different methods for each specific application, but such implementations should not be considered beyond the scope of this application.

In the several embodiments provided in this application, it should be understood that the disclosed equipment, devices and methods can be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of the units is only for logical functions. In actual implementation, there may be other division methods, and units with the same functions may also be assembled into one unit. Units, such as multiple units or components, may be combined or integrated into another device, or some features may be omitted, or not performed. In addition, the coupling or direct coupling or communication connection between each other shown or discussed may be an indirect coupling or communication connection through some interfaces, devices or units, or may be electrical, mechanical or other forms of connection.

The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or they may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiments of the present application.

In addition, each functional unit in various embodiments of the present application may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit. The above integrated units can be implemented in the form of hardware or software functional units.

If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it may be stored in a storage medium. Based on this understanding, the technical solution of the present application is essentially or contributes to the existing technology, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium , including several instructions to cause a computer device (which can be a personal computer, a backend server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of this application. The aforementioned storage media include: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), magnetic disk or optical disk and other media that can store program code.

The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

A supply chain financial asset review method, which includes:

In response to the received review request for the financial asset image data, perform image recognition on the financial asset image data to obtain the position information and character information of each line of text boxes in the financial asset image data;

Perform multi-modal information extraction on the financial asset image data based on the position information and character information of each line of text box to obtain the target information;

Using the target information to verify the legality of the financial asset image data;

In response to the financial asset image data passing the legality verification, sending the target information to the configuration platform;

In response to the feedback data from the configuration platform for the target information, query the target opinions from the pre-configured review opinion set according to the feedback data;

Send the target opinion to the configuration platform;

In response to a selection instruction for the target opinion, the target opinion is fed back.
The supply chain financial asset audit method according to claim 1, wherein the multimodal information extraction of the financial asset image data is performed according to the position information and character information of each line of text box to obtain the target information, including:

Feature extraction is performed on the character information based on the position information of each line of text box to obtain the first feature corresponding to the graph node, where the first feature is an inter-modal feature composed of three modal features: text, visual and layout. generated by fusion;

Abstracting the character information to obtain an initial graph, wherein the initial graph is described by graph nodes, edges between nodes, and an adjacency matrix of the graph;

Partition the initial graph by clustering the graph nodes in the initial graph, and perform multiple iterative updates on the allocation matrix used for division during the partitioning process;

Obtain the cluster representation and allocation matrix for each iterative update in the multiple iterative update process;

Update the first feature using the cluster representation and allocation matrix in each iteration update to obtain the second feature;

Perform node classification and link prediction based on the second feature to obtain the target information.
The supply chain financial asset audit method according to claim 1, wherein the use of the target information to perform legality verification on the financial asset image data includes:

Determine whether the financial asset image data meets the format requirements according to the target information;

When the financial asset image data does not meet the format requirements, refuse to accept the review request for the financial asset image data;

Issue prompt information, where the prompt information is used to prompt that the financial asset image data has a format error and to prompt to re-upload the financial asset image data.
The supply chain financial asset audit method according to claim 3, wherein the use of the target information to perform legality verification on the financial asset image data also includes:

When the financial asset image data meets the format requirements, detect the type of the financial asset image data;

When the financial asset image data is an invoice type, detect whether the transaction amount of the financial asset image data is a positive number, and when the transaction amount is not a positive number, prompt an error in the transaction amount; and/or

Obtain the transaction party from the financial asset image data, perform authenticity verification on the transaction party, and when the transaction party fails the authenticity verification, prompt the transaction party with incorrect information; and/or

Detect the number of loans corresponding to a transaction in the financial asset image data, and when the number of loans is greater than or equal to one, prompt that the invoice number already exists; and/or

Detect whether the date format in the financial asset image data is a configuration format, and when the date format is not the configuration format, prompt a date format error; and/or

Obtain preconfigured business rules, and detect whether the financial asset image data conforms to the business rules. When the financial asset image data does not conform to the business rules, a prompt is issued that the business requirements are not met.
The supply chain financial asset review method according to claim 1, wherein before sending the target information to the configuration platform, the method further includes:

Get pre-configured information matching rules;

Obtain configuration information from the target information according to the information matching rules;

Detect whether the configuration information satisfies the information matching rules and obtain a matching result;

Send the matching results to the configuration platform.
The supply chain financial asset review method according to claim 1, wherein querying target opinions from a preconfigured set of review opinions based on the feedback data includes:

Determine the feedback data as keywords;

Use the keywords to perform a query in the review opinion collection to obtain the target opinion;

Wherein, the set of review opinions is a set obtained by clustering based on historical review opinions.
The supply chain financial asset review method according to claim 1, wherein after sending the target opinion to the configuration platform, the method further includes:

In response to the received update request for the target opinion, obtain update data for the target opinion, optimize the target opinion based on the update data, and update the optimized target opinion to the Collection of audit opinions; and/or

In response to the received rejection instruction for the target opinion, the uploaded opinion is received, and the received opinion is fed back.
The supply chain financial asset review method according to claim 1, wherein the method further includes:

When the financial asset image data fails to pass the legality check, an error message is issued, and a prompt is given to re-upload the financial asset image data after modification.
A supply chain financial asset audit device, comprising:

A recognition unit, configured to perform image recognition on the financial asset image data in response to the received review request for the financial asset image data, and obtain the position information and character information of each line of text boxes in the financial asset image data;

An extraction unit, configured to extract multimodal information from the financial asset image data according to position information and character information of each line of text boxes to obtain target information;

A verification unit configured to use the target information to verify the legality of the financial asset image data;

A sending unit, configured to send the target information to the configuration platform in response to the financial asset image data passing the legality verification;

A query unit configured to respond to feedback data from the configuration platform for the target information, and query target opinions from a preconfigured set of review opinions based on the feedback data;

The sending unit is also used to send the target opinion to the configuration platform;

A feedback unit is configured to feed back the target opinion in response to the selection instruction for the target opinion.
A computer device includes a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor implements the following steps when executing the computer program:

In response to the received review request for the financial asset image data, perform image recognition on the financial asset image data to obtain the position information and character information of each line of text boxes in the financial asset image data;

Perform multi-modal information extraction on the financial asset image data based on the position information and character information of each line of text box to obtain the target information;

Using the target information to verify the legality of the financial asset image data;

In response to the financial asset image data passing the legality verification, sending the target information to the configuration platform;

In response to feedback data from the configuration platform for the target information, querying a target opinion from a pre-configured review opinion set according to the feedback data;

Send the target opinion to the configuration platform;

In response to a selection instruction for the target opinion, the target opinion is fed back.
The computer device according to claim 10, wherein the financial asset image data is subjected to multi-modal information extraction based on the position information and character information of each line of text box to obtain target information, including:

Feature extraction is performed on the character information based on the position information of each line of text box to obtain the first feature corresponding to the graph node, where the first feature is an inter-modal feature composed of three modal features: text, visual and layout. generated by fusion;

The character information is abstracted to obtain an initial graph, where the initial graph is described by graph nodes, edges between nodes, and an adjacency matrix of the graph;

Partition the initial graph by clustering the graph nodes in the initial graph, and perform multiple iterative updates on the allocation matrix used for partitioning during the partitioning process;

Obtain the cluster representation and allocation matrix for each iterative update in the multiple iterative update process;

The first feature is updated by using the cluster representation and the allocation matrix during each iteration to obtain a second feature;

Perform node classification and link prediction based on the second feature to obtain the target information.
The computer device according to claim 10, wherein said using the target information to perform legality verification on the financial asset image data includes:

Determine whether the financial asset image data meets the format requirements according to the target information;

When the financial asset image data does not meet the format requirements, refuse to accept the review request for the financial asset image data;

Issue prompt information, where the prompt information is used to prompt that the financial asset image data has a format error and to prompt to re-upload the financial asset image data.
The computer device according to claim 12, wherein said utilizing the target information to perform legality verification on the financial asset image data further includes:

When the financial asset image data meets the format requirements, detect the type of the financial asset image data;

When the financial asset image data is an invoice type, detect whether the transaction amount of the financial asset image data is a positive number, and when the transaction amount is not a positive number, prompt an error in the transaction amount; and/or

Obtain the transaction party from the financial asset image data, perform authenticity verification on the transaction party, and when the transaction party fails the authenticity verification, prompt the transaction party with incorrect information; and/or

Detect the number of loans corresponding to a transaction in the financial asset image data, and when the number of loans is greater than or equal to one, prompt that the invoice number already exists; and/or

Detect whether the date format in the financial asset image data is a configuration format, and when the date format is not the configuration format, prompt a date format error; and/or

Obtain preconfigured business rules, and detect whether the financial asset image data conforms to the business rules. When the financial asset image data does not conform to the business rules, a prompt is issued that the business requirements are not met.
The computer device according to claim 10, wherein before sending the target information to the configuration platform, the processor also implements the following steps when executing the computer program:

Get pre-configured information matching rules;

Obtain configuration information from the target information according to the information matching rules;

Detect whether the configuration information satisfies the information matching rules and obtain a matching result;

Send the matching results to the configuration platform.
The computer device of claim 10, wherein querying target opinions from a preconfigured set of review opinions based on the feedback data includes:

Determine the feedback data as keywords;

Use the keywords to perform a query in the review opinion collection to obtain the target opinion;

Wherein, the set of review opinions is a set obtained by clustering based on historical review opinions.
The computer device according to claim 10, wherein after sending the target opinion to the configuration platform, the processor also implements the following steps when executing the computer program:

In response to the received update request for the target opinion, obtain update data for the target opinion, optimize the target opinion according to the update data, and update the optimized target opinion to the Collection of audit opinions; and/or

In response to the received rejection instruction for the target opinion, the uploaded opinion is received, and the received opinion is fed back.
The computer device of claim 10, wherein the processor also implements the following steps when executing the computer program:

When the financial asset image data fails the legality check, a prompt error message is issued, and a prompt is given to re-upload the financial asset image data after modification.
A computer-readable storage medium, wherein the computer-readable storage medium stores a computer program that, when executed by a processor, causes the processor to perform the following operations:

In response to the received review request for the financial asset image data, perform image recognition on the financial asset image data to obtain the position information and character information of each line of text boxes in the financial asset image data;

Perform multi-modal information extraction on the financial asset image data based on the position information and character information of each line of text box to obtain the target information;

Using the target information to verify the legality of the financial asset image data;

In response to the financial asset image data passing the legality verification, sending the target information to the configuration platform;

In response to the feedback data from the configuration platform for the target information, query the target opinions from the preconfigured review opinion set according to the feedback data;

Send the target opinion to the configuration platform;

In response to a selection instruction for the target opinion, the target opinion is fed back.
The computer-readable storage medium as claimed in claim 18, wherein the multi-modal information extraction is performed on the financial asset image data based on the position information and character information of each line of text boxes to obtain target information, including:

Feature extraction is performed on the character information based on the position information of each line of text box to obtain the first feature corresponding to the graph node, where the first feature is an inter-modal feature composed of three modal features: text, visual and layout. generated by fusion;

Abstracting the character information to obtain an initial graph, wherein the initial graph is described by graph nodes, edges between nodes, and an adjacency matrix of the graph;

Partition the initial graph by clustering the graph nodes in the initial graph, and perform multiple iterative updates on the allocation matrix used for partitioning during the partitioning process;

Obtaining a cluster representation and an allocation matrix during each iterative update in the multiple iterative update processes;

Update the first feature using the cluster representation and allocation matrix in each iteration update to obtain the second feature;

Perform node classification and link prediction based on the second feature to obtain the target information.
The computer-readable storage medium as claimed in claim 18, wherein the use of the target information to perform legality verification on the financial asset image data includes:

determining whether the financial asset image data meets format requirements according to the target information;

When the financial asset image data does not meet the format requirements, refuse to accept the review request for the financial asset image data;

Issue prompt information, where the prompt information is used to prompt that the financial asset image data has a format error and to prompt to re-upload the financial asset image data.