CN115731065A - Data processing method of account checking system, electronic equipment and storage medium - Google Patents
Data processing method of account checking system, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN115731065A CN115731065A CN202211542865.1A CN202211542865A CN115731065A CN 115731065 A CN115731065 A CN 115731065A CN 202211542865 A CN202211542865 A CN 202211542865A CN 115731065 A CN115731065 A CN 115731065A
- Authority
- CN
- China
- Prior art keywords
- data
- report
- financial
- reconciliation
- different types
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Abstract
The application discloses a data processing method of a reconciliation system, electronic equipment and a storage medium, wherein the data processing method of the reconciliation system comprises the following steps: receiving different types of financial data, wherein the types of the financial data are defined based on the reconciliation system; processing data of different types of financial data by using a plurality of calculation engines to obtain a plurality of partition data tables; and forming a reconciliation report corresponding to the reconciliation system based on the partition data table. Through the mode, the data processing method of the account checking system can be used for rapidly and accurately processing different financial data.
Description
Technical Field
The present application relates to the field of data processing technologies, and in particular, to a data processing method of an account checking system, an electronic device, and a storage medium.
Background
In an account checking system or an account checking center, data processing is usually corresponding operations based on Oracle database storage, and because the operations based on the Oracle database are operated in a single machine, the data cannot be synchronously processed, the time consumption is large, and a large amount of time is consumed for processing when the amount of the data is large.
Disclosure of Invention
The application provides a data processing method of a reconciliation system, electronic equipment and a storage medium, which can rapidly and accurately process and process different types of data based on a plurality of computing engines of a big data platform so as to solve the problem that the reconciliation system generates a reconciliation report.
In order to solve the technical problem, the present application adopts a technical solution that: the data processing method of the account checking system is applied to a big data platform, the big data platform comprises a plurality of computing engines, and the method comprises the following steps: receiving different types of financial data, wherein the types of the financial data are defined based on the reconciliation system; processing data of different types of financial data by using a plurality of calculation engines to obtain a plurality of partition data tables; and forming a reconciliation report corresponding to the reconciliation system based on the partition data table.
Wherein receiving different types of financial data comprises: and receiving different types of financial data, and converting the data format of the financial data into the data format of the big data platform.
The financial data comprises at least one of batch distribution return disc data, point exchange data, settlement-disregarding exchange data, point expiration data, daily change point data and point balance summary data.
Wherein, utilize a plurality of calculation engine to carry out data processing to the financial data of different grade type, obtain a plurality of subregion data sheet, include: and processing transaction distribution data, activity distribution data and exchange data of different types of financial data by using a plurality of calculation engines to obtain a plurality of partition data tables.
And when the accumulated point balance summarized data is processed, the corresponding relation between the bank serial number and the account serial number of the reconciliation system is increased.
Wherein, form the statement of account checking that accounts checking system corresponds based on the subregion data table, include: and forming a data summary report and a credit card balance report corresponding to the reconciliation system based on the partition data table.
The data summary report comprises at least one of an active daily report, an active monthly report, an accounting daily report and an accounting monthly report.
The credit card balance report comprises an overdue difference report and an exchange difference report.
Wherein, utilize a plurality of calculation engine to carry out data processing to the financial data of different grade type, after obtaining a plurality of subregion data sheet, include: and storing a plurality of partition data tables by utilizing a snapshot technology.
Wherein, utilize a plurality of calculation engine to carry out data processing to the financial data of different grade type, after obtaining a plurality of subregion data sheet, include: and responding to a data acquisition instruction of the account checking system, and sending a partition data table corresponding to the data acquisition instruction to the account checking system.
In order to solve the above technical problem, another technical solution adopted by the present application is: there is provided an electronic device comprising a memory for storing a computer program and a processor for executing the computer program to implement a data processing method of a reconciliation system.
In order to solve the above technical problem, another technical solution adopted by the present application is: there is provided a computer-readable storage medium having stored therein a computer program for implementing a data processing method of a reconciliation system when executed by a processor.
The beneficial effect of this application is: different from the prior art, the data processing method of the reconciliation system provided by the application carries out distributed processing on different financial data through the plurality of computing engines of the big data platform so as to obtain the plurality of partitioned data tables, can realize processing on the financial data, can form a reconciliation report corresponding to the reconciliation system based on the partitioned data tables, and can solve the problem of timeliness of the report generated by the reconciliation system based on the big data platform.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts. Wherein:
fig. 1 is a schematic flowchart of a first embodiment of a reconciliation system data processing method provided in the present application;
fig. 2 is a schematic flowchart of a second embodiment of a reconciliation system data processing method provided by the present application;
FIG. 3 is a schematic diagram of one embodiment of point reconciliation system data processing provided herein;
fig. 4 is a schematic flowchart of a data processing method of a reconciliation system according to a third embodiment of the present application;
fig. 5 is a schematic flowchart of a fourth embodiment of a reconciliation system data processing method provided by the present application;
FIG. 6 is a schematic structural diagram of an embodiment of an electronic device provided in the present application;
FIG. 7 is a block diagram of an embodiment of a computer-readable storage medium provided in the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments in the present application without making any creative effort belong to the protection scope of the present application.
Referring to fig. 1, fig. 1 is a schematic flowchart of a first embodiment of a reconciliation system data processing method provided by the present application, where the method is applied to an Oracle database, and the method includes:
step 11: receiving different types of financial data; wherein the type of the financial data is defined based on the reconciliation system.
In some embodiments, the financial data received by the Oracle database is peripheral data comprising at least one of batch distribution return disc data, points redemption data, settlement discounting redemption data, points expiration data, daily change points data, and points balance summary data.
The batch distribution of the disk data refers to data collected by a bank system; the point exchange data refers to data related to points collected by a third-party platform (such as an operator client and a bank client), and the point data can be used for exchange, namely the point data meeting the exchange condition, such as data displayed on a point column on a display interface of the bank client; disregarding the exchange data refers to data collected by the third party platform and related to the points, wherein the point data cannot be used for exchange, namely the point data which does not meet the exchange condition; the point overdue data refers to point data corresponding to points which are not used after the issued points are expired; daily change point data refers to point data that all third party platforms change in point data every day; the point balance summary data refers to point data corresponding to points left after the points are issued.
In some embodiments, different types of data are loaded into the Oracle database through Sqlldr. Wherein, sqlldr is a data loading tool of an Oracle database, and is generally used for migrating files/data of an operating system to the Oracle database; an Oracle database is a relational database.
In addition, the fund reconciliation needs to be carried out in a reconciliation center or a reconciliation system, wherein the reconciliation refers to reconciliation of the accounting flow stored by the third-party platform and the clearing flow and clearing file returned by the bank system so as to check the consistency of the accounting data of the third-party platform and the clearing data of the bank and ensure that the daily predicted occurrence amount of each accounts of the paid banks of the gold and silver banks of the third-party platform is consistent with the actual occurrence amount. Wherein, all financial services provided by the third-party platform are established on the bank fund system.
Step 12: and processing the financial data of different types by using the script file to obtain a difference data table and a summary data table.
In some embodiments, based on an Oracle database, the script file is used for processing transaction distribution data, activity distribution data and exchange data of different types of financial data to obtain a plurality of partition data tables. The script file is a shell script, and the shell script enables the text file to be executed by writing a command into the text file.
The difference data table is a data table formed by data with difference between data collected by a third-party platform and data fed back by a bank; the summary data table may be a data table obtained by counting all financial data.
In some embodiments, the obtained financial data may be stored while different types of financial data obtained may be processed using script files to obtain difference data tables and summary data tables.
Step 13: and forming a reconciliation report corresponding to the reconciliation system based on the difference data table and the summary data table.
In some embodiments, a final settlement data table may be obtained based on the difference data table and the summary data table, the settlement data table conforming to a reconciliation statement of the reconciliation system.
It is worth noting that the time spent for obtaining the reconciliation report form by the above method is long, and because the data is processed on the Oracle database, all logics of the Oracle database are operated on a single machine, and the efficiency cannot be improved by a distributed computing mode, and the efficiency of obtaining the reconciliation report form by the above method is slow and the time cost is high.
In addition, only the difference data table and the summary data table are obtained by processing the data by using the script file, the dirty data and/or the abnormal data cannot be well processed by the script file, and at this time, the manual assistance is needed, so that the labor and the time are consumed.
In order to solve the above problem, the present application provides another reconciliation system data processing method, as shown in fig. 2, fig. 2 is a schematic flow chart of a second embodiment of the reconciliation system data processing method provided in the present application, and the method is applied to a big data platform, where the big data platform includes a plurality of computing engines, and includes:
step 21: receiving different types of financial data; wherein the type of the financial data is defined based on the reconciliation system.
The financial data comprises at least one of batch distribution return disc data, point exchange data, settlement-disregarding exchange data, point expiration data, daily change point data and point balance summary data.
The reconciliation system can be a credit reconciliation system or a reconciliation system corresponding to the rest data.
In some embodiments, different types of financial data are loaded to the big data platform by way of file2 hive. The file2hive mode refers to converting a file format into a hive format.
In addition, the big data platform has the characteristics of large storage memory and high throughput.
In some embodiments, the data format of the financial data and the data format of the big data are not necessarily consistent, and at this time, the data formats of the received different types of financial data may be converted into the data format of the big data platform, so that the obtained financial data is processed based on a plurality of computing engines of the big data platform.
Step 22: and processing the financial data of different types by using a plurality of calculation engines to obtain a plurality of partition data tables.
The computing engine is a spark, the spark is a memory-based fast, universal and extensible big data analysis computing engine, and the spark is used for processing data, so that the time for processing the data can be saved. In a large data platform, data can be processed in a hive2hive mode.
In some embodiments, based on the big data platform, transaction distribution data processing, activity distribution data processing and/or exchange data processing are carried out on different types of financial data by using a plurality of computing engines, and a plurality of partition data tables are obtained.
In addition, in order to reduce the subsequent data checking process, the corresponding relation between the bank serial number and the account serial number of the reconciliation system can be increased when the point balance summarized data is processed.
Step 23: and forming a reconciliation report corresponding to the reconciliation system based on the partition data table.
In some embodiments, a data summary report and a credit card balance report corresponding to the reconciliation system may be formed based on the partitioned data table. The data summarizing report comprises at least one of an active daily report, an active monthly report, an accounting daily report and an accounting monthly report; the credit card balance report comprises an expired difference report and an exchange difference report.
In some embodiments, within a big data platform, the data may be stored using a HDFS (Hadoop Distributed File System) data File table structure. As shown in fig. 3, different financial data are loaded into an HDFS data file table structure in a big data platform in a file2hive manner, and then based on the HDFS data file table structure, the loaded financial data are processed by using a computing engine and the hive2hive manner, such as transaction distribution data processing, activity distribution data processing, exchange data processing, and/or third-party platform balance data processing, where based on the distribution data processing, the activity distribution data processing, and the exchange data, a data summary report and a credit card balance report may be obtained. In addition, based on the data summary report, data tables such as an active daily report, an active monthly report, an accounting daily report and an accounting monthly report can be generated; based on the credit card balance report, an overdue difference report and an exchange difference report can be obtained.
The processed data can directly generate a report on a report platform and can be written back to an original database in a hive2db mode.
The third party platform can acquire processed target data from the big data platform in a db2hive mode, for example, the database of the point reconciliation system acquires data such as batch distribution data, activity distribution data, negative point data and transaction activity distribution data from the big data platform in a db2hive mode.
Different from the prior art, the reconciliation system data processing method provided by the application can perform distributed processing on different types of financial data based on a plurality of computing engines of a big data platform and generate a reconciliation report corresponding to the reconciliation system, can solve the aging problem of the report by utilizing the characteristics of large storage and high throughput of the big data platform, and also supports data sharing, and other platforms can acquire corresponding processed data from the big data platform based on business requirements.
Referring to fig. 4, fig. 4 is a schematic flowchart of a third embodiment of the reconciliation system data processing method provided in the present application, which is applied to a big data platform, where the big data platform includes a plurality of computing engines, and the method includes:
step 41: receiving different types of financial data; wherein the type of the financial data is defined based on the reconciliation system.
Step 42: and processing the financial data of different types by using a plurality of calculation engines to obtain a plurality of partition data tables.
Step 43: and storing a plurality of partition data tables by utilizing a snapshot technology.
Therein, the snapshot technique refers to a fully available copy with respect to a given data set, the copy comprising an image of the corresponding data at a certain point in time (the point in time at which the copy begins). The data table stored using the snapshot technique may be a copy or replica of the original data table.
In addition, the snapshot technology supports data backtracking, dirty data and/or abnormal data can be more conveniently checked, and report data rerun is realized.
Step 44: and forming a reconciliation report corresponding to the reconciliation system based on the partition data table.
Different from the prior art, the data processing method of the account checking system provided by the application can perform distributed processing on different financial data through the plurality of computing engines of the big data platform to obtain the plurality of partitioned data tables, and the account checking report corresponding to the account checking system can be formed based on the partitioned data tables, namely the account checking logic is realized through the plurality of computing engines of the big data platform, so that the problem of timeliness of the report generated by the account checking system can be solved, and the processed data is stored by utilizing a snapshot technology, so that data backtracking, data re-running and other operations are facilitated.
Referring to fig. 5, fig. 5 is a schematic flowchart of a fourth embodiment of the reconciliation system data processing method provided in the present application, which is applied to a big data platform, where the big data platform includes a plurality of computing engines, and the method includes:
step 51: receiving different types of financial data; wherein the type of the financial data is defined based on the reconciliation system.
Step 52: and processing the financial data of different types by using a plurality of calculation engines to obtain a plurality of partition data tables.
Step 53: and responding to a data acquisition instruction of the account checking system, and sending a partition data table corresponding to the data acquisition instruction to the account checking system.
In some embodiments, the partition data table may be sent to the database of the reconciliation system in the hive2db manner.
Step 54: and forming a reconciliation report corresponding to the reconciliation system based on the partition data table.
Different from the prior art, the data processing method of the reconciliation system provided by the application can perform distributed processing on different financial data through a plurality of computing engines of the big data platform to obtain a plurality of partition data tables, and can form a reconciliation report corresponding to the reconciliation system based on the partition data tables, namely, the reconciliation logic is realized based on the big data platform, and the operations of data running condition checking, data re-running and the like can be performed by combining with the current scheduling platform.
Referring to fig. 6, fig. 6 is a schematic structural diagram of an embodiment of an electronic device provided in the present application, where the electronic device 60 includes a memory 601 and a processor 602, the memory 601 is used for storing a computer program, and the processor 602 is used for executing the computer program to implement the data processing method of the reconciliation system of any embodiment described above, which is not described again here.
Referring to fig. 7, fig. 7 is a schematic structural diagram of an embodiment of a computer-readable storage medium 70 provided in the present application, in which a computer program 701 is stored, and a computer
When executed by the processor, the program 701 is configured to implement the 5 data processing method of the reconciliation system according to any of the embodiments described above, which is not described herein again.
In summary, the data processing method of the reconciliation system provided by the application can rapidly and accurately process different financial data based on a plurality of computing engines of a big data platform, namely, the reconciliation logic can be realized based on the computing engines of the big data platform, and the reconciliation logic can be combined with the current account
Some scheduling platforms perform operations such as data operation condition checking, data re-running, data run-back and the like, and in addition, 0, data obtained by processing through a big data platform supports data sharing and data backtracking. Compared with the application to an Oracle database, the method is more simplified in the overall process.
The processor referred to in this application may be referred to as a Central Processing Unit (CPU), may be an integrated circuit chip, or may be a general-purpose processor or a digital signal processor
A Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) 5 or other programmable logic device, discrete gate or transistor logic, discrete hardware components.
The storage medium used in the present application includes various media that can store program codes, such as a usb disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), or an optical disk.
The above description is only an embodiment of the present application, and not intended to limit the scope of the present application to the above 0, and all modifications of equivalent structures and equivalent processes, which are made by using the contents of the specification and the drawings of the present application, or which are directly or indirectly applied to other related technical fields, are also included in the scope of the present application.
Claims (10)
1. A data processing method of a reconciliation system is applied to a big data platform, wherein the big data platform comprises a plurality of computing engines, and the method comprises the following steps:
receiving different types of financial data; the type of the financial data is defined based on the reconciliation system;
processing the financial data of different types by using a plurality of calculation engines to obtain a plurality of partition data tables;
and forming a reconciliation report corresponding to the reconciliation system based on the partition data table.
2. The method of claim 1, wherein receiving different types of financial data comprises:
and receiving different types of financial data, and converting the data format of the financial data into the data format of the big data platform.
3. The method of claim 1, wherein the financial data comprises at least one of batch issue return data, points redemption data, outstanding settlement redemption data, points expiration data, daily change points data, and points balance summary data;
the processing of the financial data of different types by using the plurality of calculation engines to obtain a plurality of partitioned data tables includes:
and processing transaction distribution data, activity distribution data and exchange data of different types of financial data by using the plurality of calculation engines to obtain a plurality of partition data tables.
4. The method of claim 3, wherein the correspondence between bank serial numbers and reconciliation system account serial numbers is increased when processing point balance summary data.
5. The method of claim 3, wherein the forming a reconciliation report corresponding to the reconciliation system based on the partition data table comprises:
and forming a data summary report and a credit card balance report corresponding to the reconciliation system based on the partition data table.
6. The method of claim 5, wherein the data summary report comprises at least one of an active daily report, an active monthly report, an audit daily report, and an audit monthly report;
the credit card balance report comprises an overdue difference report and an exchange difference report.
7. The method of claim 1, wherein said data processing of said financial data of different types using said plurality of computing engines to obtain a plurality of partitioned data tables comprises:
and storing a plurality of partition data tables by utilizing a snapshot technology.
8. The method of claim 1, wherein said data processing of said financial data of different types using said plurality of computing engines to obtain a plurality of partitioned data tables comprises:
and responding to a data acquisition instruction of the account checking system, and sending a partition data table corresponding to the data acquisition instruction to the account checking system.
9. An electronic device, characterized in that the electronic device comprises a memory for storing a computer program and a processor for executing the computer program to implement the method according to any of claims 1-8.
10. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, is adapted to carry out the method of any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211542865.1A CN115731065A (en) | 2022-12-02 | 2022-12-02 | Data processing method of account checking system, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211542865.1A CN115731065A (en) | 2022-12-02 | 2022-12-02 | Data processing method of account checking system, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115731065A true CN115731065A (en) | 2023-03-03 |
Family
ID=85299955
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211542865.1A Pending CN115731065A (en) | 2022-12-02 | 2022-12-02 | Data processing method of account checking system, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115731065A (en) |
-
2022
- 2022-12-02 CN CN202211542865.1A patent/CN115731065A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112232818A (en) | Data reconciliation method and device, computer equipment and storage medium | |
| CN115760258A (en) | Intelligent bid document generation method, system, computer device and storage medium | |
| CN111125106B (en) | Batch running task execution method, device, server and storage medium | |
| CN113312259B (en) | Interface testing method and device | |
| CN113256355B (en) | Method, device, medium, equipment and system for determining integral rights and interests in real time | |
| CN113626527A (en) | Financial data processing method and system | |
| CN115147202B (en) | Banking account processing system, method and device | |
| CN114579654B (en) | Unified operation management method and platform system for multi-payment settlement system of bank | |
| CN115731065A (en) | Data processing method of account checking system, electronic equipment and storage medium | |
| CN116091127A (en) | Integral issuing method and device, electronic equipment and storage medium | |
| CN114969071A (en) | Data updating method and device | |
| CN113902415A (en) | Financial data checking method and device, computer equipment and storage medium | |
| CN116188190B (en) | Multi-batch semi-real-time reconciliation method and system for high-concurrency payment system | |
| CN106528718B (en) | Method and apparatus for processing data from third party | |
| CN116382924B (en) | Recommendation method and device for resource allocation, electronic equipment and storage medium | |
| CN111736808B (en) | Subscription processing method and system of transaction report | |
| CN113971552B (en) | Batch data processing method, device, equipment and storage medium | |
| CN113971007B (en) | Information processing method, device, electronic equipment and medium | |
| EP4357923A1 (en) | Method and system for generating key performance indicator prediction model for multi-cloud applications | |
| CN111967960B (en) | Batch adjustment method and device for account data | |
| CN116450659A (en) | Transaction information accounting method, device, equipment and medium based on distributed system | |
| CN114490299A (en) | Batch operation performance analysis method, device and equipment | |
| CN117689243A (en) | Data processing method and device | |
| CN114647671A (en) | Data query method, data query device, electronic equipment, storage medium and program product | |
| CN114240610A (en) | Automatic fund collection method and device, computer equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |