CN116881736A - Information matching method, device, equipment and storage medium - Google Patents

Information matching method, device, equipment and storage medium Download PDF

Info

Publication number
CN116881736A
CN116881736A CN202311065633.6A CN202311065633A CN116881736A CN 116881736 A CN116881736 A CN 116881736A CN 202311065633 A CN202311065633 A CN 202311065633A CN 116881736 A CN116881736 A CN 116881736A
Authority
CN
China
Prior art keywords
matching
data
transaction
information
matching information
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202311065633.6A
Other languages
Chinese (zh)
Inventor
黄剑佳
李伟
蒋忆成
程鹏
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Industrial and Commercial Bank of China Ltd ICBC
Original Assignee
Industrial and Commercial Bank of China Ltd ICBC
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Industrial and Commercial Bank of China Ltd ICBC filed Critical Industrial and Commercial Bank of China Ltd ICBC
Priority to CN202311065633.6A priority Critical patent/CN116881736A/en
Publication of CN116881736A publication Critical patent/CN116881736A/en
Pending legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/22Matching criteria, e.g. proximity measures
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q40/00Finance; Insurance; Tax strategies; Processing of corporate or income taxes
    • G06Q40/02Banking, e.g. interest calculation or account maintenance
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q40/00Finance; Insurance; Tax strategies; Processing of corporate or income taxes
    • G06Q40/04Trading; Exchange, e.g. stocks, commodities, derivatives or currency exchange

Landscapes

  • Engineering & Computer Science (AREA)
  • Business, Economics & Management (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Finance (AREA)
  • Accounting & Taxation (AREA)
  • General Physics & Mathematics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • General Engineering & Computer Science (AREA)
  • Evolutionary Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • Development Economics (AREA)
  • Economics (AREA)
  • Marketing (AREA)
  • Strategic Management (AREA)
  • Technology Law (AREA)
  • General Business, Economics & Management (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The present disclosure provides an information matching method, apparatus, device, storage medium, and program product, which can be applied to the fields of big data and financial technology. The information matching method comprises the following steps: determining first matching information and second matching information in response to receiving a request for performing a matching operation; determining difference data and intersection data between the first matching information and the second matching information, wherein the intersection data comprises at least one transaction record in the first matching information and the second matching information at the same time, and the difference data comprises at least one transaction record in one of the first matching information or the second matching information; based on the first matching information and the second matching information, content comparison is carried out on each transaction record in the intersection data, and a comparison result is obtained; and determining matching result data according to the comparison result and the difference set data.

Description

Information matching method, device, equipment and storage medium
Technical Field
The present disclosure relates to the field of big data and financial technology, and more particularly to information matching methods, apparatuses, devices, media and program products.
Background
With the continuous development of computer technology, transaction systems are generally disposed in a distributed environment, and a plurality of service nodes in the distributed environment perform different transaction functions. After executing the transaction, the transaction information generated by the plurality of service nodes needs to be matched so as to ensure the consistency of the transaction information.
In the prior art, information matching is mainly realized through an http interface, for example, servers of a plurality of transaction systems or service nodes related to the same transaction information are connected, one transaction system or one service node is taken as a host, the transaction information from other transaction systems or service nodes is acquired through a host interface, and then the acquired transaction information is matched with the transaction information stored in the host, so that information matching is completed.
In the process of implementing the disclosed concept, the inventor finds that at least the following problems exist in the related art: the information matching processing mode is single, and only one transaction related to the internet can be matched, so that the problems of high difficulty and low efficiency in processing large-batch information matching requirements are caused.
Disclosure of Invention
In view of the foregoing, the present disclosure provides an information matching method, apparatus, device, medium, and program product.
According to a first aspect of the present disclosure, there is provided an information matching method, including: in response to receiving a request for performing a matching operation, determining first matching information and second matching information, wherein the first matching information comprises M transaction records of at least one data source, the second matching information comprises N transaction records of at least one data source, M is more than or equal to 2, and N is more than or equal to 2; determining difference data and intersection data between the first matching information and the second matching information, wherein the intersection data comprises at least one transaction record in the first matching information and the second matching information at the same time, and the difference data comprises at least one transaction record in one of the first matching information or the second matching information; based on the first matching information and the second matching information, content comparison is carried out on each transaction record in the intersection data, and a comparison result is obtained; and determining matching result data according to the comparison result and the difference set data.
According to an embodiment of the present disclosure, the first matching information includes a first matching table, the second matching information includes a second matching table, the first matching table and the second matching table each include a transaction number, the determining difference set data and intersection set data between the first matching information and the second matching information includes: generating a first database code statement according to the table name of the first matching table and the table name of the second matching table; and comparing the transaction numbers of the M transaction records with the transaction numbers of the N transaction records by executing the first database code sentence to obtain the intersection data and the difference data.
According to an embodiment of the present disclosure, the obtaining the intersection data and the difference data by executing the first database code sentence and comparing the transaction numbers of the M transaction records and the transaction numbers of the N transaction records includes: taking the M transaction records as a reference, determining Q transaction records which are not matched with the transaction numbers of the M transaction records from the N transaction records, wherein N is more than or equal to Q is more than or equal to 1; based on the N transaction records, determining P transaction records which are not matched with the transaction numbers of the N transaction records from the M transaction records, wherein M is more than or equal to P is more than or equal to 1; forming the P transaction records and the Q transaction records into the difference set data; and determining (M-P) transaction records matching the transaction numbers of the N transaction records from the M transaction records, and determining the (M-P) transaction records as intersection data; or (N-Q) transaction records matched with the transaction numbers of the M transaction records are determined from the N transaction records, and the (N-Q) transaction records are determined as intersection data.
According to an embodiment of the disclosure, the first matching information includes a first field value of each of the M transaction records, the second matching information includes a second field value of each of the N transaction records, and the first field value and the second field value are used to characterize a transaction characteristic of the transaction record; the content comparison is performed on each transaction record in the intersection data based on the first matching information and the second matching information, so as to obtain a comparison result, including: generating a second database code statement according to the first matching information and the second matching information; and comparing the first field value with the second field value by executing the second database code statement to obtain a comparison result.
According to an embodiment of the present disclosure, the first field value or the second field value includes a plurality of character strings, each character string being used for characterizing one transaction data of the transaction record; the plurality of character strings are spliced to obtain the first field value or the second field value.
According to an embodiment of the present disclosure, the above-mentioned matching result data includes matching detail data and matching statistical data; and determining matching result data according to the comparison result and the difference set data, wherein the matching result data comprises: forming matching difference data by at least one transaction record, of which the comparison result represents that the first field value and the second field value are not matched, and the difference data; at least one transaction record, of which the comparison result represents that the first field value and the second field value are matched, is formed into matching consistent data; and generating the matching detail data and the matching statistical data according to the matching difference data and the matching consistency data.
According to an embodiment of the present disclosure, the above method further includes: exporting the matching result data into a first matching result file in a binary data format; converting the matching result file into a second matching result file in a target data format by using a big data engine; and transmitting the second matching result file to a file processing server in the distributed system.
A second aspect of the present disclosure provides an information matching apparatus, including: the first determining module is used for determining first matching information and second matching information in response to receiving a request for executing matching operation, wherein the first matching information comprises M transaction records of at least one data source, the second matching information comprises N transaction records of at least one data source, M is more than or equal to 2, and N is more than or equal to 2; a second determining module configured to determine difference data and intersection data between the first matching information and the second matching information, where the intersection data includes at least one transaction record in both the first matching information and the second matching information, and the difference data includes at least one transaction record in either the first matching information or the second matching information; the first comparison module is used for comparing the content of each transaction record in the intersection data based on the first matching information and the second matching information to obtain a comparison result; and the second comparison module is used for determining matching result data according to the comparison result and the difference set data.
A third aspect of the present disclosure provides an electronic device, comprising: one or more processors; and a memory for storing one or more programs, wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to perform the information matching method.
A fourth aspect of the present disclosure also provides a computer-readable storage medium having stored thereon executable instructions that, when executed by a processor, cause the processor to perform the above-described information matching method.
The fifth aspect of the present disclosure also provides a computer program product comprising a computer program which, when executed by a processor, implements the above-described information matching method.
According to an embodiment of the present disclosure, when a large-batch information matching requirement is processed, first matching information and second matching information are determined in response to receiving a request for performing a matching operation; determining difference data and intersection data between the first matching information and the second matching information; based on the first matching information and the second matching information, content comparison is carried out on each transaction record in the intersection data, and a comparison result is obtained; and determining matching result data according to the comparison result and the difference set data. The embodiment of the disclosure can realize the processing of large-batch information matching by the technical means, and at least partially solves the technical problems of large difficulty and low efficiency in processing large-batch information matching in the prior art.
In an embodiment of the disclosure, data from a plurality of data sources is stored in a data lake first, and in response to receiving a request for performing a matching operation, intersection data and difference data comprising a plurality of transaction records are obtained directly based on first matching information and second matching information in the data lake; and then, batch matching of a plurality of data sources can be directly realized based on difference set and intersection data, online matching is not relied, single-stroke information matching is not needed to be realized by relying on an interface between two data sources, the matching efficiency is improved, and batch reconciliation with simple operation is realized.
Drawings
The foregoing and other objects, features and advantages of the disclosure will be more apparent from the following description of embodiments of the disclosure with reference to the accompanying drawings, in which:
fig. 1 schematically illustrates an application scenario of an information matching method according to an embodiment of the present disclosure;
FIG. 2 schematically illustrates a flow chart of an information matching method according to an embodiment of the disclosure;
FIG. 3 schematically illustrates a flow chart of a difference set data determination method according to an embodiment of the disclosure;
FIG. 4 schematically illustrates a flow chart of an intersection data determination method according to an embodiment of the disclosure;
FIG. 5 schematically illustrates a flow chart of a job scheduling method according to an embodiment of the present disclosure;
fig. 6 schematically shows a block diagram of a structure of an information matching apparatus according to an embodiment of the present disclosure; and
fig. 7 schematically illustrates a block diagram of an electronic device adapted for an information matching method according to an embodiment of the present disclosure.
Detailed Description
Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. It should be understood that the description is only exemplary and is not intended to limit the scope of the present disclosure. In the following detailed description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the embodiments of the present disclosure. It may be evident, however, that one or more embodiments may be practiced without these specific details. In addition, in the following description, descriptions of well-known structures and techniques are omitted so as not to unnecessarily obscure the concepts of the present disclosure.
The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. The terms "comprises," "comprising," and/or the like, as used herein, specify the presence of stated features, steps, operations, and/or components, but do not preclude the presence or addition of one or more other features, steps, operations, or components.
All terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art unless otherwise defined. It should be noted that the terms used herein should be construed to have meanings consistent with the context of the present specification and should not be construed in an idealized or overly formal manner.
Where expressions like at least one of "A, B and C, etc. are used, the expressions should generally be interpreted in accordance with the meaning as commonly understood by those skilled in the art (e.g.," a system having at least one of A, B and C "shall include, but not be limited to, a system having a alone, B alone, C alone, a and B together, a and C together, B and C together, and/or A, B, C together, etc.).
In the technical solution of the present disclosure, the related user information (including, but not limited to, user personal information, user image information, user equipment information, such as location information, etc.) and data (including, but not limited to, data for analysis, stored data, displayed data, etc.) are information and data authorized by the user or sufficiently authorized by each party, and the related data is collected, stored, used, processed, transmitted, provided, disclosed, applied, etc. and processed, all in compliance with the related laws and regulations and standards of the related country and region, necessary security measures are taken, no prejudice to the public order, and corresponding operation entries are provided for the user to select authorization or rejection.
In the prior art, the process of on-line information matching includes: for the transaction information A, a server B and a server C related to the transaction information A are connected, one of the server A, the server B and the server C is used as a host, information related to the transaction information A is acquired from the other server through an interface between the two servers, and the acquired information is matched with the transaction information locally stored in the host so as to determine whether the information related to the transaction information A in the host is consistent with the information related to the transaction information A in the other servers.
However, the information matching can only be performed on-line for the server related to the transaction information by on-line, and the matching can not be performed for other transaction information, so that the information matching efficiency is low. In addition, because the corresponding information is acquired through the online http interface in each information matching process, the operation of acquiring information from a plurality of servers in a batch scene cannot be realized very difficultly.
Therefore, the prior art has the problems of single processing mode, large difficulty in processing large-batch information matching requirements and low efficiency.
The embodiment of the disclosure provides an information matching method, which comprises the steps of responding to a received request for executing matching operation, determining first matching information and second matching information, wherein the first matching information comprises M transaction records of at least one data source, the second matching information comprises N transaction records of at least one data source, M is more than or equal to 2, and N is more than or equal to 2; determining difference data and intersection data between the first matching information and the second matching information, wherein the intersection data comprises at least one transaction record in the first matching information and the second matching information at the same time, and the difference data comprises at least one transaction record in one of the first matching information or the second matching information; based on the first matching information and the second matching information, content comparison is carried out on each transaction record in the intersection data, and a comparison result is obtained; and determining matching result data according to the comparison result and the difference set data.
Fig. 1 schematically illustrates an application scenario of information matching according to an embodiment of the present disclosure.
As shown in fig. 1, the application scenario 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 is a medium used to provide a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, among others.
The user may interact with the server 105 through the network 104 using at least one of the first terminal device 101, the second terminal device 102, the third terminal device 103, to receive or send messages, etc. Various communication client applications, such as a shopping class application, a web browser application, a search class application, an instant messaging tool, a mailbox client, social platform software, etc. (by way of example only) may be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103.
The first terminal device 101, the second terminal device 102, the third terminal device 103 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smartphones, tablets, laptop and desktop computers, and the like.
The server 105 may be a server providing various services, such as a background management server (by way of example only) providing support for websites browsed by the user using the first terminal device 101, the second terminal device 102, and the third terminal device 103. The background management server may analyze and process the received data such as the user request, and feed back the processing result (e.g., the web page, information, or data obtained or generated according to the user request) to the terminal device.
It should be noted that, the information matching method provided by the embodiments of the present disclosure may be generally performed by the server 105. Accordingly, the information matching apparatus provided by the embodiments of the present disclosure may be generally provided in the server 105. The information matching method provided by the embodiments of the present disclosure may also be performed by a server or a server cluster that is different from the server 105 and is capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and/or the server 105. Accordingly, the information matching apparatus provided by the embodiments of the present disclosure may also be provided in a server or a server cluster that is different from the server 105 and is capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and/or the server 105.
For example, the user may perform a transaction operation at a mobile banking client in the first terminal device 101, the second terminal device 102, and the third terminal device 103, and the first terminal device 101, the second terminal device 102, and the third terminal device 103 send transaction information to the server 105, where the server 105 is equipped with a data lake. The server 105 determines first matching information and second matching information in response to receiving a request for performing a matching operation; determining difference data and intersection data between the first matching information and the second matching information; based on the first matching information and the second matching information, content comparison is carried out on each transaction record in the intersection data, and a comparison result is obtained; and determining matching result data according to the comparison result and the difference set data.
It should be understood that the number of terminal devices, networks and servers in fig. 1 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.
The information matching method of the disclosed embodiment will be described in detail below with reference to fig. 2 to 5 based on the scenario described in fig. 1.
Fig. 2 schematically illustrates a flow chart of an information matching method according to an embodiment of the present disclosure.
As shown in fig. 2, the method 200 includes operations S210 to S240.
In operation S210, in response to receiving a request for performing a matching operation, first matching information and second matching information are determined.
According to embodiments of the present disclosure, at a server idle time, e.g., every morning, the server may load information from different data sources through job scheduling tasks and post-process the information into a data lake, wherein the information from the different data sources includes information needed to perform an information matching operation.
According to embodiments of the present disclosure, a data source includes a source of information required to provide information matching, such as a computer providing matching information in a distributed environment.
According to an embodiment of the present disclosure, the first matching information comprises at least two transaction records of at least one data source; the second matching information includes at least two transaction records of at least one data source. The first matching information and the second matching information are respectively from different data sources, e.g. different servers in the same distributed environment.
According to the embodiment of the disclosure, after a client initiates a matching operation, a server determines information needed to perform the matching operation from a data lake in response to receiving a request for performing the matching operation, and determines first matching information and second matching information according to a data source of the information.
For example, the data source of the first matching information may be a server storing transaction records for a plurality of payee accounts of one payee; the data source of the second matching information may be a server storing transaction records for payment accounts corresponding to the plurality of payers. And the server loads transaction records of a plurality of collection accounts of one collection party and transaction records of payment accounts corresponding to the plurality of payment parties through a job scheduling task at idle time, and inputs the transaction records into the data lake. When the client initiates the matching operation, the server determines matching information required by the matching operation from the data lake, determines the matching information of the transaction records of a plurality of collection accounts of which the data source is a collection party as first matching information, and determines the matching information of the transaction records of the payment accounts of which the data source is a plurality of payment parties as second matching information.
Difference data and intersection data between the first matching information and the second matching information are determined in operation S220.
According to an embodiment of the present disclosure, the intersection data includes at least one transaction record in both the first matching information and the second matching information; the difference set data includes at least one transaction record in one of the first matching information or the second matching information.
According to an embodiment of the present disclosure, the intersection data and the difference data may be a data table having a preset format. Each transaction record in the intersection data and the difference data at least comprises a transaction number and transaction record content, wherein the transaction number of the same transaction record is the same, namely the transaction number of the same transaction record in the first matching information and the second matching information is the same.
According to an embodiment of the present disclosure, the transaction record content in the first matching information is referred to as a first transaction record content, and the transaction record content in the second matching information is referred to as a second transaction record content. Wherein the intersection data and the difference data include only the first transaction record content and the second transaction record content. In the intersection data, neither the first transaction record content nor the second transaction record content is empty; the first transaction record content and the second transaction record content in the difference set data have one empty.
According to the embodiment of the disclosure, the transaction records with different transaction numbers in the first matching information and the second matching information can be formed into difference set data; and forming intersection data by transaction records with different transaction numbers in the first matching information and the second matching information.
For example, the first matching information may be table a and the second matching information may be table B. Table A contains transaction records with transaction number 1 and transaction records with transaction number 2; table B contains transaction records with a transaction number of 1 and transaction records with a transaction number of 3. Since tables a and B each include data of transaction number 1, intersection data can be generated from transaction records of transaction number 1 in table a, transaction records of transaction number 1 in table B. Since the transaction record of transaction number 2 is included only in table a and the transaction record of transaction number 3 is included only in table B, difference data can be generated from the transaction record of transaction number 2 and the transaction record of transaction number 3. In the difference set data, the first transaction record content with the transaction number of 2 is the same as the record content in the table A, and the second transaction record content is empty; the first transaction record with transaction number 3 is empty and the second transaction record is the same as in table B.
In operation S230, content comparison is performed on each transaction record in the intersection data based on the first matching information and the second matching information, so as to obtain a comparison result.
According to an embodiment of the present disclosure, the comparison result is used to indicate whether the first transaction record content and the second transaction record content included in each transaction record in the intersection data are the same.
According to the embodiment of the disclosure, a value may be assigned to the comparison result corresponding to each transaction record, for example, the comparison result of the transaction record with the same first transaction record content and the second transaction record content may be assigned to 1, and the comparison result of the transaction record with different first transaction record content and second transaction record content may be assigned to 0.
According to the embodiment of the disclosure, the comparison result table with the preset format can be generated based on the transaction number and the corresponding comparison result, and the comparison result table at least comprises the transaction number and the comparison result, and the comparison result and the transaction record are connected through the transaction number.
In operation S240, matching result data is determined according to the comparison result and the difference set data.
According to embodiments of the present disclosure, the matching result data may include data in which information matches are consistent and data in which information matches are inconsistent.
Because the intersection data contains transaction records with the same transaction record content and transaction records with different transaction record content, a plurality of transaction records with different transaction record content in the intersection data and a plurality of transaction records in the difference data can be used together as data with inconsistent information matching, and a plurality of transaction records with the same transaction record content form data with consistent information matching.
For example, the transaction record with the comparison result of 0 in the intersection data and the transaction record composition information in the difference data may be matched and inconsistent, and the transaction record with the comparison result of 1 in the intersection data may be matched and consistent.
According to an embodiment of the present disclosure, when a large-batch information matching requirement is processed, first matching information and second matching information are determined in response to receiving a request for performing a matching operation; determining difference data and intersection data between the first matching information and the second matching information; based on the first matching information and the second matching information, content comparison is carried out on each transaction record in the intersection data, and a comparison result is obtained; and determining matching result data according to the comparison result and the difference set data, so that batch information matching from a plurality of data sources can be realized. The embodiment of the disclosure can realize the processing of large-batch information matching by the technical means, and at least partially solves the technical problems of large difficulty and low efficiency in processing large-batch information matching in the prior art.
In an embodiment of the disclosure, data from a plurality of data sources is stored in a data lake first, and in response to receiving a request for performing a matching operation, intersection data and difference data comprising a plurality of transaction records are obtained directly based on first matching information and second matching information in the data lake; and then, based on difference set and intersection set data, batch matching of a plurality of data sources can be directly realized, online matching is not needed, single-stroke information matching is not needed to be realized by relying on an interface between two data sources, the matching efficiency is improved, and batch reconciliation with simple operation is realized.
According to an embodiment of the present disclosure, the first matching information includes a first matching table, the second matching information includes a second matching table, and the first matching table and the second matching table each include a transaction number.
According to an embodiment of the present disclosure, determining difference set data and intersection set data between first matching information and second matching information includes: generating a first database code statement according to the table name of the first matching table and the table name of the second matching table; and comparing the transaction numbers of the M transaction records with the transaction numbers of the N transaction records by executing the first database code statement to obtain intersection data and difference data.
According to the embodiment of the disclosure, the first matching information and the second matching information may be recorded in a data table of a preset format, the data table corresponding to the first matching information is a first matching table, the data table corresponding to the second matching information is a second matching table, the first matching table and the second matching table each at least comprise a transaction number, and the first matching table and the second matching table each have a corresponding table name.
According to the embodiment of the disclosure, the first database code statement may be used to perform a comparison operation on the first matching information and the second matching information, and may be executed through a large data platform Spark, intersection data and difference data. The Spark of the big data platform has the characteristics of high running speed, strong usability, support of complex inquiry, support of real-time stream processing, good fault tolerance and the like. By loading information in the data lake and using the large data platform Spark to execute the information matching method, batch information matching in a distributed environment can be realized rapidly, and the problem of information matching through online is solved.
According to an embodiment of the present disclosure, the first database code statement may be an SQL (Structured Query Language ) statement, generated from table names of the first and second matching tables.
According to another embodiment of the present disclosure, the first database code statement may also be an HQL (Hibernate Query Language ) statement, generated from object class names in the first and second matching information, such as a transaction number.
According to embodiments of the present disclosure, a first database code statement may be generated from a logical or database code statement authoring template of data matching to be performed.
According to an embodiment of the present disclosure, after the server generates the first database code statement, the first database code statement is executed through Spark to determine difference data and intersection data.
For example, the first database code sentence matches the transaction number in the first matching information with the transaction number in the second matching information one by one, and forms intersection data from a plurality of transaction records having the same transaction number, and forms difference data from a plurality of intersection records having different transaction numbers.
For example, there are the following for table 1 characterizing the first matching information and table 2 characterizing the second matching information: table 1 contains transaction records with transaction number 1 and transaction records with transaction number 2; table 2 contains transaction records with transaction number 1 and transaction record with transaction number 3.
TABLE 1
Transaction numbering Transaction record content
1 HT00002|||2|||0
2 HT00004|||1|||5000000
TABLE 2
Transaction numbering Transaction record content
1 HT0002|||1|||1000
3 HT00006|||1|||6000000
By executing the first database code statement, intersection data may be generated from two data records having transaction numbers 1, and difference data may be generated from two data records having transaction numbers 2 and 3.
TABLE 3 Table 3
Transaction numbering TABLE 1 transaction record content Table 2 transaction record content
2 HT00004|||1|||5000000
3 HT00006|||1|||6000000
TABLE 4 Table 4
Transaction numbering TABLE 1 transaction record content Table 2 transaction record content
1 HT00002|||2|||0 HT0002|||1|||1000
According to embodiments of the present disclosure, both the difference set data and the intersection set data may be stored in the form of a data table. For example, the intersection data table and the difference data table are shown in tables 3 and 4.
According to the embodiment of the disclosure, the intersection data and the difference data in the first matching information and the second matching information are determined through the first database code statement, so that the information matching operation can be realized in the database without matching through online.
According to an embodiment of the present disclosure, by executing a first database code statement, comparing the transaction numbers of the M transaction records with the transaction numbers of the N transaction records, obtaining intersection data and difference data includes: taking M transaction records as references, determining Q transaction records which are not matched with the transaction numbers of the M transaction records from the N transaction records, wherein N is more than or equal to Q is more than or equal to 1; determining P transaction records which are not matched with the transaction numbers of the N transaction records from the M transaction records by taking the N transaction records as a reference, wherein M is more than or equal to P is more than or equal to 1; forming difference set data by the P transaction records and the Q transaction records; and determining (M-P) transaction records matching the transaction numbers of the N transaction records from the M transaction records, and determining the (M-P) transaction records as intersection data; or (N-Q) transaction records matching the transaction numbers of the M transaction records are determined from the N transaction records, and the (N-Q) transaction records are determined as intersection data.
According to the embodiment of the disclosure, since the transaction records may belong to only the first matching information or the second matching information, the matching of the transaction numbers only at one side may result in missing the transaction records in the difference set data, and thus, missing the transaction records in the difference set data may be avoided by the matching of the transaction numbers at both sides.
For example, with the first matching information as a reference, determining whether the transaction number of the transaction record in the second matching information is the same as the transaction number of the transaction record in the first matching information, generating intersection data if the transaction number is the same, and generating difference data if the transaction number is different; and determining whether the transaction number of the transaction record in the first matching information is the same as the transaction number of the transaction record in the second matching information by taking the second matching information as a reference, generating intersection data if the transaction number is the same, and generating difference data if the transaction number is different.
According to an embodiment of the present disclosure, the intersection data and the difference data may be a data table in a preset format, and at least include a transaction number, a first transaction record content and a second transaction record content, where neither the first transaction record content nor the second transaction record content of the intersection data is null, and one of the first transaction record content and the second transaction record content of the difference data is null, but cannot be null at the same time.
For example, difference set data generated based on the first matching information, the first transaction record content is empty; and generating difference set data based on the second matching information, wherein the second transaction record content is empty.
According to an embodiment of the present disclosure, the first database code statement includes a difference set determination statement and an intersection determination statement, wherein the difference set determination statement includes a first difference set determination statement, a second difference set determination statement, and a difference set combination statement, and the intersection determination statement includes a first intersection determination statement and a second intersection determination statement.
Specifically, the first difference set determination statement may be a database code statement generated based on the second matching information and the table name of the first matching information table; the second difference set determination statement may be a database code statement generated based on table names of the first matching information and the second matching information table; the difference set combination statement may be a database code statement that generates difference set data based on the transaction record determined by the first difference set determination statement and the transaction record determined by the second difference set determination statement; the first intersection determination statement may be a database code statement generated based on table names of the first matching information and the second matching information table; the second intersection statement may be a database code statement generated based on the second matching information and the intersection data.
According to the embodiment of the disclosure, a first difference set determination statement may be executed based on a transaction number of a transaction record in the first matching information, and a transaction record that does not match the transaction number of the transaction record of the first matching information may be determined from the transaction record of the second matching information; executing a second difference set determination statement by taking the transaction number of the transaction record in the second matching information as a reference, and determining a transaction record which is not matched with the transaction number of the transaction record of the second matching information from the transaction record of the first matching information; and finally executing a difference set combination statement, and forming the transaction record determined by the first difference set determination statement and the transaction record determined by the second difference set determination statement into difference set data.
According to the embodiment of the disclosure, the first matching information may include a transaction record that is not matched with the transaction number of the transaction record of the second matching information, and the second matching information may also include a transaction record that is not matched with the transaction number of the transaction record of the first matching information, so that the first matching information and the second matching information are respectively used as reference to perform two comparisons, thereby avoiding missing of the difference set data.
According to an embodiment of the disclosure, a first intersection determination statement may be executed, where a transaction record having the same transaction number as a transaction record in the second matching information is determined from the transaction records of the first matching information, intersection data is generated, the generated intersection data transaction number and the first transaction record content are not null, and the second transaction record content is null. And then executing a second intersection determination statement, determining a transaction record with the same transaction number as the transaction record in the intersection data from the transaction records of the second matching information, and generating second transaction record content in the intersection data according to the transaction characteristics of the transaction record in the second matching information.
According to another embodiment of the present disclosure, a second intersection determination statement may be executed, a transaction record having the same transaction number as the transaction record in the first matching information is determined from the transaction records in the second matching information, intersection data is generated, the generated intersection data transaction number and the second transaction record content are not empty, the first transaction record content is empty, next, the first intersection determination statement is executed, the transaction record having the same transaction number as the transaction record in the intersection data is determined from the transaction record in the first matching information, and the first transaction record content in the intersection data is generated.
Fig. 3 schematically illustrates a flow chart of a method of determining difference set data according to an embodiment of the disclosure.
As shown in fig. 3, the method 300 for determining difference set data of this embodiment includes operations S321 to S327 as one specific embodiment of operation S220.
According to an embodiment of the present disclosure, operation S220 is to determine difference data and intersection data between the first matching information and the second matching information.
In operation S321, a first difference set determination statement is generated and executed.
In operation S322, it is determined whether a transaction record in the second matching information has a transaction record having a different transaction number from the transaction number in the first matching information.
According to an embodiment of the present disclosure, if there is a transaction record in the second matching information that is different from the transaction number in the first matching information, operation S323 is performed; if there is no transaction record in the second matching information that is different from the transaction number in the first matching information, operation S324 is performed.
In operation S323, a transaction record is stored.
In operation S324, a second difference set determination statement is generated and executed.
In operation S325, it is determined whether a transaction record in the first matching information has a transaction record having a different transaction number from a transaction record in the second matching information.
According to an embodiment of the present disclosure, if the transaction record in the first matching information has a transaction record having a different transaction number from the transaction record in the second matching information, operation S326 is performed; if the transaction record in the first matching information does not have a transaction record having a different transaction number from the transaction record in the second matching information, operation S327 is performed.
In operation S326, a transaction record is stored.
According to an embodiment of the present disclosure, operation S327 is performed based on the transaction records obtained in operation S323 and operation S326.
In operation S327, a difference set combination statement is executed, generating difference set data.
Fig. 4 schematically illustrates a flowchart of a method of determining intersection data according to an embodiment of the disclosure.
As shown in fig. 4, the method 400 for determining intersection data of this embodiment includes operations S421 to S427 as a specific embodiment of operation S220.
According to an embodiment of the present disclosure, operation S220 is to determine difference data and intersection data between the first matching information and the second matching information.
In operation S421, a first intersection determination sentence is generated and executed.
In operation S422, it is determined whether the transaction record in the first matching information has the same transaction record as the transaction number of the transaction record in the second matching information.
According to an embodiment of the present disclosure, if the transaction record in the first matching information has the same transaction record as the transaction number of the transaction record in the second matching information, operation S423 is performed; if the transaction record in the first matching information does not have the same transaction record as the transaction number of the transaction record in the second matching information, operation S424 is performed.
In operation S423, intersection data is generated, and the transaction number and the first transaction record content are written.
In operation S424, a second intersection determination sentence is generated and executed.
In operation S425, it is determined whether the transaction record in the second matching information has the same transaction record as the transaction number of the transaction record in the intersection data.
According to an embodiment of the present disclosure, if the transaction record in the second matching information has the same transaction record as the transaction number of the transaction record in the intersection data, operation S426 is performed; if the transaction record in the second matching information does not have the same transaction record as the transaction number of the transaction record in the intersection data, operation S427 is performed.
In operation S426, the second transaction record content is written in the intersection data.
According to an embodiment of the present disclosure, after writing the second transaction record content in the intersection data, operation S427 is performed.
In operation S427, intersection data is generated.
For example, based on the transaction records in table 1, a first difference set determination statement is executed to determine, from the transaction records in table 2, a transaction record that does not match the transaction number of the transaction record in table 1, and the stored transaction record is shown in table 5:
TABLE 5
Transaction numbering TABLE 1 transaction record content Table 2 transaction record content
3 HT00006|||1|||6000000
Based on the transaction records in table 2, a second difference set determination statement is executed to determine a transaction record from the transaction records in table 1 that does not match the transaction number of the transaction record in table 2, the stored transaction record being as shown in table 6:
TABLE 6
Transaction numbering TABLE 1 transaction record content Table 2 transaction record content
2 HT00004|||1|||5000000
Executing a difference set combination statement, and combining the transaction record determined by the first difference set determination statement and the transaction record determined by the second difference set determination statement into difference set data, wherein the difference set data is shown in a table 7;
TABLE 7
Transaction numbering TABLE 1 transaction record content Table 2 transaction record content
2 HT00004|||1|||5000000
3 HT00006|||1|||6000000
Executing a first intersection determination statement, determining a transaction record with the same transaction number as the transaction record in table 2 from the transaction records in table 1, and generating intersection data, wherein the generated intersection data transaction number and the table 1 transaction record content are not null, and the table 2 transaction record content is null, and the generated intersection data is shown in table 8;
TABLE 8
Transaction numbering TABLE 1 transaction record content Table 2 transaction record content
1 HT00002|||2|||0
Next, executing a second intersection determination statement, determining a transaction record with the same transaction number as the intersection data from the transaction records in table 2, and generating table 2 transaction record content in the intersection data according to the transaction record in table 2, wherein the generated intersection data is shown in table 9;
TABLE 9
Transaction numbering TABLE 1 transaction record content Table 2 transaction record content
1 HT00002|||2|||0 HT0002|||1|||1000
According to the embodiment of the disclosure, the first matching information and the second matching information are respectively used as the reference, the bilateral comparison is performed, and the difference set data and the intersection set data are determined, so that the difference set data and the intersection set data of the first matching information and the second matching information are not missed, and the accuracy of information matching is ensured.
According to an embodiment of the present disclosure, the first matching information includes respective first field values of the M transaction records, and the second matching information includes respective second field values of the N transaction records, the first field values and the second field values being used to characterize transaction characteristics of the transaction records.
According to an embodiment of the present disclosure, based on first matching information and second matching information, content comparison is performed on each transaction record in intersection data, to obtain a comparison result, including: generating a second database code statement according to the first matching information and the second matching information; and comparing the first field value with the second field value by executing the second database code statement to obtain a comparison result.
According to an embodiment of the present disclosure, the first field value and the second field value comprise a plurality of strings, each string being used to characterize one transaction data of the transaction record; and the plurality of character strings are spliced to obtain a first field value or a second field value.
According to embodiments of the present disclosure, the transaction characteristics may characterize a plurality of characteristics of the transaction record content, such as transaction type, user identity, transaction status, transaction time, transaction amount, and the like.
For example, the first field value of table 1 is: HT00002 2 0, HT00002 may characterize transaction type, 2 a payee, 0 a transaction failure, etc.
According to embodiments of the present disclosure, multiple transaction characteristics may also be characterized in a transaction record by setting multiple fields.
According to the embodiment of the disclosure, the comparison result is generated by comparing the first field value and the second field value in the intersection data, so that the comparison of the transaction characteristics of different data sources with the same transaction number is realized, the requirement of realizing large-batch information matching by using a database is further met, and online matching is not needed.
According to an embodiment of the present disclosure, the matching result data includes matching detail data and matching statistics data.
According to an embodiment of the present disclosure, determining matching result data from the comparison result and the difference set data includes: at least one transaction record, of which the comparison result represents that the first field value and the second field value are not matched, and difference data form matching difference data; at least one transaction record of which the comparison result represents that the first field value is matched with the second field value is formed into matching consistent data; and generating matching detail data and matching statistical data according to the matching difference data and the matching consistent data.
According to an embodiment of the present disclosure, the matching detail data includes detailed information matching results, the matching statistics include statistics of the information matching results, and the matching detail data and the matching statistics are generated from matching difference data and matching consistency data.
According to an embodiment of the present disclosure, the matching difference data may include a transaction record of a transaction number corresponding to a result of 0 in the comparison result and a transaction record in the difference set data, and the matching agreement data may include a transaction record of a transaction number corresponding to a result of 1 in the comparison result.
According to an embodiment of the present disclosure, the matching difference data and the matching consistent data may be a data table in a preset format, and at least include a serial number, a transaction number, a matching date, a matching result, a first transaction record content, a second transaction record content, a difference value, failure information, and the like.
According to embodiments of the present disclosure, the serial number may be related to the transaction number or may be related to the transaction characteristic, characterizing the location of the transaction record in the matching discrepancy data or matching reconciliation data. The matching date may characterize the date on which the matching operation was performed. The discrepancy value may characterize a difference between the first transaction record and the second transaction record, and if a null value exists in the first transaction record and the second transaction record, the discrepancy value is null. The failure information may characterize the cause of the failure of the information match.
According to the embodiment of the disclosure, the matching result represents the result of information matching of the transaction record, for example, the information matching result is consistent, and the matching result is 1; the transaction record is a transaction record unique to the first matching information, and the matching result is 2; the transaction record is a transaction record unique to the second matching information, and the matching result is 3; transaction records with the same transaction number but different transaction characteristics in the first matching information and the second matching information are matched with 4; the information matching fails due to network interruption and the like, and the matching result is 5.
According to an embodiment of the present disclosure, composing at least one transaction record, of which the comparison result characterizes that the first field value and the second field value do not match, with difference set data into matching difference data, includes: in the comparison result, determining a corresponding first transaction number with a result of 0; determining a first transaction record corresponding to the first transaction number in the intersection data; writing the first transaction record into the matching difference data after processing; and writing the matched difference data after processing all transaction records in the difference set data.
According to an embodiment of the present disclosure, generating matching detail data and matching statistics from matching difference data and matching agreement data includes: in the comparison result, determining a corresponding second transaction number with the result of 1; determining a second transaction record corresponding to the second transaction number in the intersection data; and writing the matched consistent data after processing the second transaction record.
Table 10
TransactionNumbering device TABLE 1 transaction record content Table 2 transaction record content
1 HT00002|||2|||0 HT0002|||1|||1000
For example, according to the comparison result table obtained as described above, the corresponding transaction number 1 with the determination result of 0 is determined, and the transaction record with the transaction number 1 is determined in the intersection data as shown in table 10.
Since the contents of the transaction record in table 1 are different from those of the transaction record in table 2 in the intersection data table, the transaction record with the transaction number 1 can be written into the matching difference data table as shown in table 11:
TABLE 11
According to the embodiment of the disclosure, based on the comparison result, the intersection data and the difference data, the matching difference data and the matching consistent data are generated, and then the matching detail data and the matching statistical data are further generated according to the matching difference data and the matching consistent data, so that the matching result data is obtained, and the efficiency is improved.
According to an embodiment of the present disclosure, the information matching method further includes: exporting the matching result data into a first matching result file in a binary data format; converting the matching result file into a second matching result file in a target data format by using a big data engine; and transmitting the second matching result file to a file processing server in the distributed system.
According to an embodiment of the present disclosure, the first matching result file may be a binary format file; the second matching result file may be a TXT text format file and a CHecK format file.
According to an embodiment of the present disclosure, exporting matching result data into a first matching result file in binary data format includes: the server executes the job, exports the current day content in the matching detail data, and generates a file in a binary format.
According to an embodiment of the present disclosure, converting a matching result file into a second matching result file in a target data format using a big data engine includes: and the big data engine transfers the file in the binary format to generate a file in the TXT text format and a file in the CHeck format.
According to an embodiment of the present disclosure, the second matching result file is transmitted to a file processing server in the distributed system, including: the server automatically performs the ETRANCFG task and passes the generated TXT text format file and the check format file to the downstream file server.
According to the embodiment of the disclosure, the conversion of the file format is completed by executing the job, and the file generated by information matching is processed and transmitted by automatically executing the task to transmit the file to the downstream file server, so that the efficiency is improved.
Fig. 5 schematically illustrates a flow chart of a job calling method according to an embodiment of the present disclosure.
As shown in fig. 5, the job call 500 of this embodiment includes operations S521 to S522.
In operation S501, a job schedule loads a matching file.
In operation S502, a matching file is processed and entered into a data lake.
In operation S503, spark performs an information matching job. For example, the information matching operation may be performed with the matching file in the data lake as the first matching information or the second matching information.
In operation S504, it is determined whether the matching result data is successfully generated.
According to an embodiment of the present disclosure, if the matching result data is not successfully generated, operation S505 is performed; if the match result data is successfully generated, operation S506 is performed.
In operation S505, data generation fails.
In operation S506, a job is exported, and a first matching result file is generated.
In operation S507, it is determined whether the first matching result file is successfully generated.
According to an embodiment of the present disclosure, if the first matching result file is not successfully generated, operation S508 is performed; if the first matching result file is successfully generated, operation S509 is performed.
For example, it may be determined from the first result string whether the first matching result file was successfully generated. The first result string is/datafs/bdsp/export/F-GCMS/# YYYYMDD#, which indicates that the first matching result file is successfully generated, and otherwise, indicates that the first matching result file is not successfully generated.
In operation S508, the first matching result file generation fails.
In operation S509, the job number is transferred, and a second matching result file is generated.
In operation S510, it is determined whether the second matching result file is successfully generated.
According to an embodiment of the present disclosure, if the second matching result file is not successfully generated, operation S511 is performed; if the second matching result file is successfully generated, operation S512 is performed. For example, it may be determined from the second result string whether the second matching result file was successfully generated. If the second result string is/datafs/bdsp/export/F-GCMS/scfin_lageecon, the second matching result file is successfully generated, otherwise, the second matching result file is not successfully generated.
In operation S511, the second matching result file generation fails.
In operation S512, the second matching result file is transmitted to the distributed system file processing server.
In operation S513, it is determined whether the file processing server successfully received the second matching result file.
According to an embodiment of the present disclosure, if the file processing server successfully receives the second matching result file, operation S515 is performed; if the file processing server does not successfully receive the second matching result file, operation S514 is performed.
In operation S514, the information matching fails.
In operation S515, the information matching is successful.
According to the embodiment of the disclosure, the matching files are processed and input into the data lake through scheduling and loading, the operations such as information matching, matching result file transfer and the like are executed, a service group is formed, and different requirements of different stages of the information matching method are met.
Fig. 6 schematically shows a block diagram of a structure of an information matching apparatus according to an embodiment of the present disclosure.
As shown in fig. 6, the information matching apparatus 600 of this embodiment includes a first determination module 610, a second determination module 620, a first comparison module 630, and a second comparison module 640.
The first determining module 610 is configured to determine, in response to receiving a request to perform a matching operation, first matching information and second matching information, where the first matching information includes M transaction records of at least one data source, and the second matching information includes N transaction records of at least one data source, M is greater than or equal to 2, and N is greater than or equal to 2. In an embodiment, the first determining module 610 may be configured to perform the operation S210 described above, which is not described herein.
A second determining module 620 is configured to determine difference data and intersection data between the first matching information and the second matching information, where the intersection data includes at least one transaction record in both the first matching information and the second matching information, and the difference data includes at least one transaction record in either the first matching information or the second matching information. In an embodiment, the second determining module 620 may be configured to perform the operation S220 described above, which is not described herein.
The first comparison module 630 is configured to compare contents of each transaction record in the intersection data based on the first matching information and the second matching information, so as to obtain a comparison result. In an embodiment, the first comparing module 630 may be used to perform the operation S230 described above, which is not described herein.
And a second comparison module 640 for determining matching result data based on the comparison result and the difference set data. In an embodiment, the second comparison module 640 may be used to perform the operation S240 described above, which is not described herein.
According to an embodiment of the present disclosure, the second determination module 620 includes a first generation sub-module and a second generation sub-module.
The first generation sub-module is used for generating a first database code statement according to the table name of the first matching table and the table name of the second matching table.
The second generation sub-module is used for comparing the transaction numbers of the M transaction records with the transaction numbers of the N transaction records by executing the first database code statement to obtain intersection data and difference data.
According to an embodiment of the present disclosure, the second generation sub-module includes a first determination unit, a second determination unit, a first generation unit, and a third determination unit.
The first determining unit is used for determining Q transaction records which are not matched with the transaction numbers of the M transaction records from the N transaction records by taking the M transaction records as a reference, wherein N is more than or equal to Q is more than or equal to 1.
The second determining unit is used for determining P transaction records which are not matched with the transaction numbers of the N transaction records from the M transaction records by taking the N transaction records as a reference, wherein M is more than or equal to P is more than or equal to 1.
The first generation unit is used for forming P transaction records and Q transaction records into difference set data.
The third determining unit is used for determining (M-P) transaction records matched with the transaction numbers of the N transaction records from the M transaction records, and determining the (M-P) transaction records as intersection data; or (N-Q) transaction records matching the transaction numbers of the M transaction records are determined from the N transaction records, and the (N-Q) transaction records are determined as intersection data.
According to an embodiment of the present disclosure, the first comparison module 630 includes a third generation sub-module and a fourth generation sub-module.
The third generation sub-module is used for generating a second database code statement according to the first matching information and the second matching information.
And the fourth generation sub-module is used for comparing the first field value with the second field value by executing the second database code statement to obtain a comparison result.
According to an embodiment of the present disclosure, the second comparison module 640 includes a fifth generation sub-module, a sixth generation sub-module, and a seventh generation sub-module.
The fifth generation sub-module is used for forming matching difference data by at least one transaction record, of which the comparison result represents that the first field value and the second field value are not matched, and the difference data.
The sixth generation sub-module is used for forming matching consistent data from at least one transaction record of which the comparison result represents that the first field value and the second field value are matched.
And the seventh generation submodule is used for generating matching detail data and matching statistical data according to the matching difference data and the matching consistent data.
According to an embodiment of the present disclosure, the information matching apparatus 600 further includes a first generating module, a second generating module, and a transmitting module.
The first generation module is used for exporting the matching result data into a first matching result file in a binary data format.
The second generation module is used for converting the matching result file into a second matching result file in the target data format by using the big data engine.
The transmission module is used for transmitting the second matching result file to a file processing server in the distributed system.
Any of the first determination module 610, the second determination module 620, the first comparison module 630, and the second comparison module 640 may be combined in one module to be implemented, or any of the modules may be split into a plurality of modules according to an embodiment of the present disclosure. Alternatively, at least some of the functionality of one or more of the modules may be combined with at least some of the functionality of other modules and implemented in one module.
According to embodiments of the present disclosure, at least one of the first determination module 610, the second determination module 620, the first comparison module 630, and the second comparison module 640 may be implemented at least in part as hardware circuitry, such as a Field Programmable Gate Array (FPGA), a Programmable Logic Array (PLA), a system on a chip, a system on a substrate, a system on a package, an Application Specific Integrated Circuit (ASIC), or may be implemented in hardware or firmware in any other reasonable manner of integrating or packaging the circuitry, or in any one of or a suitable combination of three of software, hardware, and firmware. Alternatively, at least one of the first determination module 610, the second determination module 620, the first comparison module 630, and the second comparison module 640 may be at least partially implemented as a computer program module, which when executed, may perform the corresponding functions.
Fig. 7 schematically illustrates a block diagram of an electronic device adapted for an information matching method according to an embodiment of the present disclosure.
As shown in fig. 7, an electronic device 700 according to an embodiment of the present disclosure includes a processor 701 that can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM) 702 or a program loaded from a storage section 708 into a Random Access Memory (RAM) 703. The processor 701 may include, for example, a general purpose microprocessor (e.g., a CPU), an instruction set processor and/or an associated chipset and/or a special purpose microprocessor (e.g., an Application Specific Integrated Circuit (ASIC)), or the like. The processor 701 may also include on-board memory for caching purposes. The processor 701 may comprise a single processing unit or a plurality of processing units for performing different actions of the method flows according to embodiments of the disclosure.
In the RAM 703, various programs and data necessary for the operation of the electronic apparatus 700 are stored. The processor 701, the ROM 702, and the RAM 703 are connected to each other through a bus 704. The processor 701 performs various operations of the method flow according to the embodiments of the present disclosure by executing programs in the ROM 702 and/or the RAM 703. Note that the program may be stored in one or more memories other than the ROM 702 and the RAM 703. The processor 701 may also perform various operations of the method flow according to embodiments of the present disclosure by executing programs stored in the one or more memories.
According to an embodiment of the present disclosure, the electronic device 700 may further include an input/output (I/O) interface 705, the input/output (I/O) interface 705 also being connected to the bus 704. The electronic device 700 may also include one or more of the following components connected to the input/output I/O interface 705: an input section 706 including a keyboard, a mouse, and the like; an output portion 707 including a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, a speaker, and the like; a storage section 708 including a hard disk or the like; and a communication section 709 including a network interface card such as a LAN card, a modem, or the like. The communication section 709 performs communication processing via a network such as the internet. The drive 710 is also connected to the I/O interface 705 as needed. A removable medium 711 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 710 as necessary, so that a computer program read therefrom is mounted into the storage section 708 as necessary.
The present disclosure also provides a computer-readable storage medium that may be embodied in the apparatus/device/system described in the above embodiments; or may exist alone without being assembled into the apparatus/device/system. The computer-readable storage medium carries one or more programs which, when executed, implement methods in accordance with embodiments of the present disclosure.
According to embodiments of the present disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium, which may include, for example, but is not limited to: a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this disclosure, a computer-readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. For example, according to embodiments of the present disclosure, the computer-readable storage medium may include ROM 702 and/or RAM 703 and/or one or more memories other than ROM 702 and RAM 703 described above.
Embodiments of the present disclosure also include a computer program product comprising a computer program containing program code for performing the methods shown in the flowcharts. The program code, when executed in a computer system, causes the computer system to perform the methods provided by embodiments of the present disclosure.
The above-described functions defined in the system/apparatus of the embodiments of the present disclosure are performed when the computer program is executed by the processor 701. The systems, apparatus, modules, units, etc. described above may be implemented by computer program modules according to embodiments of the disclosure.
In one embodiment, the computer program may be based on a tangible storage medium such as an optical storage device, a magnetic storage device, or the like. In another embodiment, the computer program may also be transmitted, distributed over a network medium in the form of signals, downloaded and installed via the communication section 709, and/or installed from the removable medium 711. The computer program may include program code that may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the foregoing.
In such an embodiment, the computer program may be downloaded and installed from a network via the communication portion 709, and/or installed from the removable medium 711. The above-described functions defined in the system of the embodiments of the present disclosure are performed when the computer program is executed by the processor 701. The systems, devices, apparatus, modules, units, etc. described above may be implemented by computer program modules according to embodiments of the disclosure.
According to embodiments of the present disclosure, program code for performing computer programs provided by embodiments of the present disclosure may be written in any combination of one or more programming languages, and in particular, such computer programs may be implemented in high-level procedural and/or object-oriented programming languages, and/or assembly/machine languages. Programming languages include, but are not limited to, such as Java, c++, python, "C" or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, partly on a remote computing device, or entirely on the remote computing device or server. In the case of remote computing devices, the remote computing device may be connected to the user computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computing device (e.g., connected via the Internet using an Internet service provider).
The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
Those skilled in the art will appreciate that the features recited in the various embodiments of the disclosure and/or in the claims may be provided in a variety of combinations and/or combinations, even if such combinations or combinations are not explicitly recited in the disclosure. In particular, the features recited in the various embodiments of the present disclosure and/or the claims may be variously combined and/or combined without departing from the spirit and teachings of the present disclosure. All such combinations and/or combinations fall within the scope of the present disclosure.
While the foregoing is directed to embodiments of the present disclosure, other and further details of the invention may be had by the present application, it is to be understood that the foregoing description is merely exemplary of the present disclosure and that no limitations are intended to the scope of the disclosure, except insofar as modifications, equivalents, improvements or modifications may be made without departing from the spirit and principles of the present disclosure.

Claims (11)

1. An information matching method, comprising:
determining first matching information and second matching information in response to receiving a request for executing a matching operation, wherein the first matching information comprises M transaction records of at least one data source, the second matching information comprises N transaction records of at least one data source, M is more than or equal to 2, and N is more than or equal to 2;
Determining difference data and intersection data between the first matching information and the second matching information, wherein the intersection data comprises at least one transaction record in the first matching information and the second matching information at the same time, and the difference data comprises at least one transaction record in one of the first matching information or the second matching information;
based on the first matching information and the second matching information, content comparison is carried out on each transaction record in the intersection data, and a comparison result is obtained; and
and determining matching result data according to the comparison result and the difference set data.
2. The method of claim 1, wherein the first matching information comprises a first matching table and the second matching information comprises a second matching table, the first matching table and the second matching table each comprising a transaction number, the determining difference data and intersection data between the first matching information and the second matching information comprising:
generating a first database code statement according to the table name of the first matching table and the table name of the second matching table; and
and comparing the transaction numbers of the M transaction records with the transaction numbers of the N transaction records by executing the first database code statement to obtain the intersection data and the difference data.
3. The method of claim 2, wherein the comparing the transaction numbers of the M transaction records and the transaction numbers of the N transaction records by executing the first database code statement to obtain the intersection data and the difference data comprises:
taking the M transaction records as a reference, determining Q transaction records which are not matched with the transaction numbers of the M transaction records from the N transaction records, wherein N is more than or equal to Q is more than or equal to 1;
determining P transaction records which are not matched with the transaction numbers of the N transaction records from the M transaction records by taking the N transaction records as a reference, wherein M is more than or equal to P is more than or equal to 1;
forming the P transaction records and the Q transaction records into the difference set data; and
determining (M-P) transaction records from the M transaction records that match the transaction numbers of the N transaction records, and determining the (M-P) transaction records as intersection data; or (N-Q) transaction records matched with the transaction numbers of the M transaction records are determined from the N transaction records, and the (N-Q) transaction records are determined as intersection data.
4. The method of claim 1, wherein the first matching information comprises a first field value for each of the M transaction records, the second matching information comprises a second field value for each of the N transaction records, the first field value and the second field value being used to characterize a transaction characteristic of the transaction record; based on the first matching information and the second matching information, content comparison is performed on each transaction record in the intersection data to obtain a comparison result, and the method comprises the following steps:
Generating a second database code statement according to the first matching information and the second matching information;
and comparing the first field value with the second field value by executing the second database code statement to obtain a comparison result.
5. The method of claim 4, wherein the first or second field value comprises a plurality of strings, each string being used to characterize one transaction data of the transaction record; and the plurality of character strings are spliced to obtain the first field value or the second field value.
6. The method of claim 4, wherein the match result data comprises match detail data and match statistics;
and determining matching result data according to the comparison result and the difference set data, wherein the matching result data comprises:
forming matching difference data by at least one transaction record, of which the comparison result represents that the first field value and the second field value are not matched, and the difference data;
at least one transaction record, of which the comparison result represents that the first field value and the second field value are matched, is formed into matching consistent data;
and generating the matching detail data and the matching statistical data according to the matching difference data and the matching consistent data.
7. The method of claim 1 or 6, further comprising:
exporting the matching result data into a first matching result file in a binary data format;
converting the matching result file into a second matching result file in a target data format by using a big data engine; and
and transmitting the second matching result file to a file processing server in the distributed system.
8. An information matching apparatus comprising:
the first determining module is used for determining first matching information and second matching information in response to receiving a request for executing matching operation, wherein the first matching information comprises M transaction records of at least one data source, the second matching information comprises N transaction records of at least one data source, M is more than or equal to 2, and N is more than or equal to 2;
a second determining module configured to determine difference data and intersection data between the first matching information and the second matching information, wherein the intersection data includes at least one transaction record in both the first matching information and the second matching information, and the difference data includes at least one transaction record in either the first matching information or the second matching information;
The first comparison module is used for comparing the content of each transaction record in the intersection data based on the first matching information and the second matching information to obtain a comparison result; and
and the second comparison module is used for determining matching result data according to the comparison result and the difference set data.
9. An electronic device, comprising:
one or more processors;
storage means for storing one or more programs,
wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to perform the method of any of claims 1-7.
10. A computer readable storage medium having stored thereon executable instructions which, when executed by a processor, cause the processor to perform the method according to any of claims 1-7.
11. A computer program product comprising a computer program which, when executed by a processor, implements the method according to any one of claims 1 to 7.
CN202311065633.6A 2023-08-23 2023-08-23 Information matching method, device, equipment and storage medium Pending CN116881736A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202311065633.6A CN116881736A (en) 2023-08-23 2023-08-23 Information matching method, device, equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202311065633.6A CN116881736A (en) 2023-08-23 2023-08-23 Information matching method, device, equipment and storage medium

Publications (1)

Publication Number Publication Date
CN116881736A true CN116881736A (en) 2023-10-13

Family

ID=88258896

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202311065633.6A Pending CN116881736A (en) 2023-08-23 2023-08-23 Information matching method, device, equipment and storage medium

Country Status (1)

Country Link
CN (1) CN116881736A (en)

Similar Documents

Publication Publication Date Title
CN111857888A (en) Transaction processing method and device
CN111950249A (en) Message generation method and device, electronic equipment and storage medium
CN115357761A (en) Link tracking method and device, electronic equipment and storage medium
CN111881329A (en) Account balance management method and system
CN111104556A (en) Service processing method and device
CN113190558A (en) Data processing method and system
CN110865797A (en) Method and device for processing dynamic attributes of services
CN116881736A (en) Information matching method, device, equipment and storage medium
US20220129332A1 (en) Handling of Metadata for Microservices Processing
CN114780361A (en) Log generation method, device, computer system and readable storage medium
CN114238585A (en) Query method and device based on 5G message, computer equipment and storage medium
CN113392093A (en) Service processing method and device, electronic equipment and computer readable medium
CN112559646A (en) Report downloading method and device
CN113704222A (en) Method and device for processing service request
CN111127077A (en) Recommendation method and device based on stream computing
CN114969059B (en) Method and device for generating order information, electronic equipment and storage medium
CN115484149B (en) Network switching method, network switching device, electronic equipment and storage medium
CN115952522A (en) System interaction method and device
CN113627998A (en) Order data processing method and device, electronic equipment and computer readable medium
CN117493443A (en) Data processing method and device, electronic equipment and readable storage medium
CN116012158A (en) Service processing method, device, equipment and storage medium
CN116436975A (en) Resource calling method, device, equipment and medium applied to server cluster
CN116340280A (en) Parallel dual-system heterogeneous database comparison method, device, equipment and storage medium
CN116310423A (en) Image recognition method, device, electronic equipment and readable storage medium
CN115375262A (en) Data generation method, apparatus, device, computer readable medium and program product

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination