CN116467352A - Transaction inquiry system - Google Patents
Transaction inquiry system Download PDFInfo
- Publication number
- CN116467352A CN116467352A CN202310473486.XA CN202310473486A CN116467352A CN 116467352 A CN116467352 A CN 116467352A CN 202310473486 A CN202310473486 A CN 202310473486A CN 116467352 A CN116467352 A CN 116467352A
- Authority
- CN
- China
- Prior art keywords
- data
- transaction
- module
- real
- historical
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012545 processing Methods 0.000 claims abstract description 77
- PUXBEKLSMBVFNW-UHFFFAOYSA-N triphenylene-2,3,6,7,10,11-hexamine hexahydrochloride Chemical compound Cl.Cl.Cl.Cl.Cl.Cl.NC1=CC=2C3=CC(=C(C=C3C3=CC(=C(C=C3C2C=C1N)N)N)N)N PUXBEKLSMBVFNW-UHFFFAOYSA-N 0.000 claims abstract description 25
- 230000007246 mechanism Effects 0.000 claims abstract description 7
- 238000004458 analytical method Methods 0.000 claims description 4
- 238000004140 cleaning Methods 0.000 claims description 3
- 238000012546 transfer Methods 0.000 claims description 3
- 238000013461 design Methods 0.000 abstract description 5
- 230000008878 coupling Effects 0.000 abstract description 4
- 238000010168 coupling process Methods 0.000 abstract description 4
- 238000005859 coupling reaction Methods 0.000 abstract description 4
- 230000002093 peripheral effect Effects 0.000 description 11
- 238000000034 method Methods 0.000 description 10
- 230000006870 function Effects 0.000 description 7
- 230000008569 process Effects 0.000 description 7
- 238000010586 diagram Methods 0.000 description 6
- 238000013500 data storage Methods 0.000 description 4
- 230000003993 interaction Effects 0.000 description 3
- 238000007619 statistical method Methods 0.000 description 3
- 238000009825 accumulation Methods 0.000 description 2
- 238000013475 authorization Methods 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 238000004806 packaging method and process Methods 0.000 description 2
- 230000000737 periodic effect Effects 0.000 description 2
- 230000009471 action Effects 0.000 description 1
- 230000006978 adaptation Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000007405 data analysis Methods 0.000 description 1
- 238000013499 data model Methods 0.000 description 1
- PCHJSUWPFVWCPO-UHFFFAOYSA-N gold Chemical compound [Au] PCHJSUWPFVWCPO-UHFFFAOYSA-N 0.000 description 1
- 239000010931 gold Substances 0.000 description 1
- 229910052737 gold Inorganic materials 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000008520 organization Effects 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 238000000926 separation method Methods 0.000 description 1
- 230000001502 supplementing effect Effects 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/219—Managing data history or versioning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2282—Tablespace storage structures; Management thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/242—Query formulation
- G06F16/2433—Query languages
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
- G06F16/252—Integrating or interfacing systems involving database management systems between a Database Management System and a front-end application
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q40/00—Finance; Insurance; Tax strategies; Processing of corporate or income taxes
- G06Q40/04—Trading; Exchange, e.g. stocks, commodities, derivatives or currency exchange
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- Business, Economics & Management (AREA)
- Accounting & Taxation (AREA)
- Computational Linguistics (AREA)
- Finance (AREA)
- Software Systems (AREA)
- Development Economics (AREA)
- Economics (AREA)
- Marketing (AREA)
- Strategic Management (AREA)
- Technology Law (AREA)
- General Business, Economics & Management (AREA)
- Mathematical Physics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application provides a transaction inquiry system which can be used in the financial field. The transaction inquiry system comprises: a source data layer, a base common layer, an application data layer, and CMQ; the source data layer comprises a first source data processing module based on a distributed processing engine (Flink) architecture, the basic public layer comprises a first basic public module based on the Flink architecture, and the application data layer comprises a first application data module based on the Flink architecture and a HATP database. By the CMQ and the receiving mechanism of the real-time transaction data of the Flink, timeliness and accuracy of receiving the real-time transaction data can be improved. Through the multi-layer design, the data in the transaction system can be uniformly managed in a layering way, so that the data of different business scenes are not interfered with each other, and the data processing coupling degree is reduced. The HATP database is used for storing real-time transaction data processed for a plurality of times, so that the impact of high concurrency query on a transaction query system can be reduced, and the query efficiency is effectively improved.
Description
Technical Field
The present application relates to the financial arts, and more particularly, to a transaction inquiry system.
Background
At present, the transaction information system of each bank has a larger information receiving range, relates to hundreds of peripheral products and channels, and covers various aspects such as transaction detail data, code table data, account information, user information and the like. Each peripheral product and channel need report data with different channels for the online receipt of data is untimely, and the processing of different logics is needed to different data products of reporting generally, probably leads to trade opponent information to splice in time, provides incomplete service for the user when the user carries out trade inquiry, and user's use experience is relatively poor.
Disclosure of Invention
The application provides a transaction inquiry system which is used for solving the problem that the current transaction system cannot provide a completed transaction inquiry service for a user and improving the use experience of the user.
The application provides a transaction inquiry system, which comprises: a source data layer, a base common layer, an application data layer, and a distributed message queue CMQ; the source data layer comprises a first source data processing module based on a distributed processing engine (Flink) architecture, the basic public layer comprises a first basic public module based on the Flink architecture, the application data layer comprises a first application data module based on the Flink architecture, and the hybrid/analysis processing HATP database;
the CMQ is used for acquiring real-time transaction data of different transaction products, and/or different transaction systems, and/or different transaction channels;
the first source data module is configured to read real-time transaction data from the CMQ module and transfer the real-time transaction data to the first base common module;
the first basic public module is used for processing the real-time transaction data, generating a transaction data summary table with multiple data dimensions, and sending the transaction data summary table with the multiple data dimensions to the first application data module;
The first application data module is used for carrying out secondary processing on the transaction data summary tables with the multiple data dimensions, storing the transaction data summary tables in the HATP database, and providing real-time transaction data query service through the transaction data summary tables in the HATP database.
Optionally, the first source data module is specifically configured to:
acquiring real-time transaction data from the CMQ;
performing data cleaning on the real-time transaction data to obtain a temporary transaction interface table and a temporary transaction information table, wherein the temporary transaction interface table comprises asset data in the real-time transaction data, and the temporary transaction information table comprises scene data in the real-time transaction data;
and sending the temporary transaction interface table and the temporary transaction information table to the first basic public module.
Optionally, the transaction data summary table of multiple data dimensions includes: a transaction detail information table, transaction summary information and transaction data of a plurality of query dimensions; the transaction summary information comprises a transaction summary list and a transaction summary index list, and the transaction data of the plurality of query dimensions at least comprises mechanism data, product data, commodity data, currency data, merchant data and function data;
The first basic public module is specifically configured to:
and processing data of the temporary transaction interface table and the temporary transaction information table to obtain transaction data of the transaction detail information table, the transaction summary information and the plurality of query dimensions.
Optionally, the first application data module is specifically configured to:
and carrying out secondary processing on the transaction data summary table with the multiple data dimensions according to the generation time of each data in the transaction data summary table with the multiple data dimensions to obtain transaction data summary tables with at least two time spans with each data dimension, and storing the transaction data summary tables in the HATP database.
Optionally, the application data layer further includes: a pushing module;
the first basic public module is further used for sending the real-time transaction data to the pushing module;
the pushing module is used for pushing the real-time transaction data to the message downlink gateway; and the message downlink gateway sends the real-time transaction data to a user terminal corresponding to the real-time transaction data.
Optionally, the first application data module is further configured to perform batch secondary processing on real-time transaction data in the pushing period according to a preset pushing period, so as to obtain a transaction summarizing result in the pushing period;
The pushing module is further configured to batch push the transaction summary result in the pushing period to the message downlink gateway and/or the external sharing platform.
Optionally, the transaction inquiry system further includes: the data lake, the source data layer further comprises a second source data processing module based on a Spark architecture of a computing engine, the basic public layer comprises a second basic public module based on the Spark architecture, and the application data layer comprises a second application data module based on the Spark architecture;
the data lake is used for storing historical transaction data of the transaction inquiry system;
the second source data module is used for acquiring historical transaction data from the data lakes in batches and transmitting the historical transaction data to the second basic public module;
the second basic public module is used for processing the historical transaction data read in batches to generate a historical transaction data summary table of various data dimensions of each user;
the second application data module is used for performing secondary processing on historical transaction data summary tables of multiple data dimensions of each user, storing the processed historical transaction data summary tables in the HATP database, and providing historical transaction data query service for each user transaction data summary table in the HATP database.
Optionally, the second source data module is specifically configured to:
acquiring historical transaction data from the data lakes in batches in a subscription mode;
generating a historical transaction text, currency information of a historical transaction and historical transaction information except the historical transaction text and the currency information of the historical transaction according to the historical transaction data;
and sending the historical transaction text, the currency information of the historical transaction and the historical transaction information to the second basic public module.
Optionally, the second application data module is specifically configured to provide at least one of a subscription service, a billing related service, a financial calendar service, and a non-financial service.
Optionally, the second application data module is further configured to send a historical transaction data summary table of multiple data dimensions of each user after secondary processing to a data lake for storage.
According to the transaction inquiry system, the timeliness and the accuracy of receiving the real-time transaction data can be improved through the CMQ and the receiving mechanism of the real-time transaction data of the Flink. Through the multi-layer design, the data in the transaction system can be uniformly managed in a layering way, so that the data of different business scenes are not interfered with each other, and the data processing coupling degree is reduced. The HATP database is used for storing real-time transaction data processed for a plurality of times, so that the impact of high concurrency query on a transaction query system can be reduced, the query efficiency is effectively improved, and the satisfaction degree of users is further improved.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the application and together with the description, serve to explain the principles of the application.
Fig. 1 is a schematic diagram of a transaction inquiry system according to an embodiment of the present application;
FIG. 2 is a second schematic diagram of a transaction inquiry system according to an embodiment of the present disclosure;
fig. 3 is a schematic diagram of a third architecture of the transaction inquiry system according to the embodiment of the present application.
Specific embodiments thereof have been shown by way of example in the drawings and will herein be described in more detail. These drawings and the written description are not intended to limit the scope of the inventive concepts in any way, but to illustrate the concepts of the present application to those skilled in the art by reference to specific embodiments.
Detailed Description
Reference will now be made in detail to exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, the same numbers in different drawings refer to the same or similar elements, unless otherwise indicated. The implementations described in the following exemplary examples are not representative of all implementations consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with some aspects of the present application as detailed in the accompanying claims.
In the embodiments of the present application, the words "first", "second", etc. are used to distinguish identical items or similar items having substantially the same function and action, and the order of them is not limited. It will be appreciated by those of skill in the art that the words "first," "second," and the like do not limit the amount and order of execution, and that the words "first," "second," and the like do not necessarily differ.
In the embodiments of the present application, words such as "exemplary" or "such as" are used to mean examples, illustrations, or descriptions. Any embodiment or design described herein as "exemplary" or "for example" should not be construed as preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "such as" is intended to present related concepts in a concrete fashion.
It should be noted that, the user information (including but not limited to user equipment information, user personal information, etc.) and the data (including but not limited to data for analysis, stored data, presented data, etc.) referred to in the present application are information and data authorized by the user or fully authorized by each party, and the collection, use and processing of the related data need to comply with the related laws and regulations and standards, and provide corresponding operation entries for the user to select authorization or rejection.
It should be noted that the transaction inquiry system of the present application may be used in the financial field, and may also be used in any field other than the financial field, and the application field of the transaction inquiry system is not limited in this application.
At present, the transaction information system of each bank has a larger information receiving range, relates to hundreds of peripheral products and channels, and covers various aspects such as transaction detail data, code table data, account information, user information and the like. Each peripheral product and channel need report data with different channels for the online receipt of data is untimely, and the processing of different logic is needed to be carried out to the data product that different general reports, probably leads to trade opponent information to splice in time, and the service that provides for the user when the user carries out trade inquiry through trade information system is incomplete, and user's use experience is relatively poor.
In view of this, the application provides a transaction inquiry system, through receiving multiple real-time data of peripheral products and reporting of channels based on distributed message queues (Cloud Message Queue, CMQ) +distributed processing engine Flink, and performing data processing in modes of splicing, supplementing, splitting and the like on the data, when a user performs transaction inquiry, completed transaction information can be provided for the user, so that the use experience of the user is improved.
The following describes the technical solutions of the present application and how the technical solutions of the present application solve the above technical problems in detail with specific embodiments. The following embodiments may be implemented independently or combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.
Fig. 1 is a schematic diagram of an architecture of a transaction query system according to an embodiment of the present application, as shown in fig. 1, where the transaction query system includes a source data layer, a base public layer, an application data layer, and a distributed message queue CMQ.
Wherein the source data layer comprises a first source data processing module based on a distributed processing engine (Flink) architecture, the underlying common layer comprises a first underlying common module based on the Flink architecture, the application data layer comprises a first application data module based on the Flink architecture, and a hybrid/analytics processing (Hybrid Transaction Analytical Processing, HATP) database.
Illustratively, the CMQ is configured to obtain real-time transaction data of different transaction products, and/or different transaction systems, and/or different transaction channels.
In the embodiment of the application, CMQ is a distributed message queue service, which can provide a reliable asynchronous communication mechanism based on messages, and can store the received and transmitted messages between different applications (or different components of the same application) in a distributed deployment in a reliable and effective CMQ queue, so as to prevent the messages from being lost. CMQ supports multiple processes to read and write simultaneously, and the receiving and transmitting are not interfered with each other, and all applications or components are not required to be in an operation state all the time. That is, CMQ is a message middleware for passing messages between different applications.
In this embodiment of the present application, the transaction product may be a product of a transaction service provided to a user, for example, an accumulation fund, various financial products, etc., the channel may be a channel for the user to conduct a transaction, for example, an online banking, an automatic teller machine, an offline business hall, etc., and the transaction system may be, for example, a deposit system, a loan system.
The transaction inquiry system may obtain real-time transaction data sent by the connected transaction products, transaction systems, transaction channels, etc. through CMQ and store the real-time transaction data in the CMQ queue. The real-time transaction data may be raw data of unprocessed transactions, and each transaction product, each transaction system and each transaction channel that send the real-time transaction data to CMQ may be preset. For example, the real-time transaction data may be stored in a CMQ queue in chronological order, or may be stored in accordance with the source of the real-time data.
Alternatively, in one possible implementation manner, the transaction query system may also use message middleware such as kafka, MQ, and the like to obtain real-time transaction data, where the type of the message middleware is not limited in the embodiments of the present application.
Alternatively, in one possible implementation, CMQ may also be message middleware external to the transaction querying system.
Illustratively, the first source data module is configured to read real-time transaction data from the CMQ module and communicate the real-time transaction data to the first underlying public module.
In this embodiment of the present application, the first source data module may interact with CMQ through the link to obtain real-time transaction data stored in the CMQ queue.
Wherein, a flank is a (batch unified) framework and distributed processing engine for state computation of unbounded and bounded data streams. Under the Flink architecture, all data is composed of streams, and offline data is a bounded data stream; real-time data is an unbounded stream, i.e. an unbounded stream.
In this embodiment of the present application, when the first source data module obtains the real-time transaction data stored in the CMQ queue through the link, the data model and the data format of the real-time transaction data are not changed, that is, the original state of the real-time transaction data is maintained.
For example, the first source data module may interact with CMQ through the Flink at preset time intervals to obtain the data stored in CMQ.
Optionally, in a possible implementation manner, the first data source module may further clean the real-time transaction data when acquiring the real-time transaction data stored in CMQ. For example, error data and data which do not meet the specification in the real-time transaction data are removed.
In this embodiment of the present application, when the first source data module obtains real-time transaction data, the real-time transaction data may be stored and sent to the first basic public module, so that the first basic public module processes the real-time data.
Optionally, in a possible implementation manner, the first source data module may store and acquire real-time transaction data, and when receiving a data acquisition request based on the link sent by the first basic public module, package the real-time data required by the first basic public module, and send the real-time data to the first basic public module. The query request sent by the first underlying public module may be executed by a Flink structured query statement (Flink Structured Query Language, flink SQL). For example, the first source data module may store real-time transaction data that may be written to one or more data tables for storage when the real-time transaction data is obtained. Wherein, the data in the data table can be classified according to the transaction currency information and the transaction scene information.
The first basic public module is used for processing the real-time transaction data, generating a transaction data summary table with multiple data dimensions, and sending the transaction data summary table with the multiple data dimensions to the first application data module.
In this embodiment of the present application, the transaction data summary table with multiple data dimensions may be a data table obtained by summarizing real-time transaction data with different processing methods. For example, a transaction detail information table of each user in a dimension of the user, an institution transaction information table in a dimension of the institution, a transaction information detail wide table in a plurality of dimensions, and a various index wide table of transaction summary data in a plurality of dimensions.
For example, after the first basic public module obtains the real-time transaction data, the real-time transaction data may be processed according to a preset data processing policy, or a preset data processing model, to obtain a transaction data summary table with multiple data dimensions.
In this embodiment of the present application, when the first basic public module generates the transaction data summary table with multiple data dimensions, the transaction data summary table with multiple data dimensions may be stored, and the transaction data summary table with multiple data dimensions may be sent to the first application data module. Or when the first basic public module receives the data acquisition request based on the Flink SQL sent by the first application data module, packaging a transaction data summary table required by the first application data module, and sending the transaction data summary table to the first application data module.
The first application data module is configured to perform secondary processing on the transaction data summary table with multiple data dimensions, store the processed transaction data summary table in the HATP database, and provide a real-time transaction data query service through the transaction data summary table in the HATP database.
In the embodiment of the application, the HATP database is a database compatible with online transaction processing and online data analysis, and can support high-concurrency data writing and query.
The secondary processing treatment is carried out on the transaction data summary table with the multiple data dimensions, so that the transaction data summary table with the multiple data dimensions can be classified and processed according to the time dimension generated by the data, and each transaction data summary table classified according to the time dimension is obtained. For example, the user's transaction statement information table is divided according to the time dimension, and a transaction statement information hottable (transaction data within 1 year), a transaction statement information hottable (transaction data within 1 year to 10 years), and a transaction statement information hottable (transaction data over 10 years) are obtained. Wherein the transaction data summary table of the classified multiple data dimensions is stored in the HATP database.
In this embodiment of the present application, the first application data module may provide a user-oriented transaction query interface for a user-oriented module of the transaction query system, and the user may input a query instruction through the transaction query interface.
When the first application data module receives the query instruction, the data can be queried in the corresponding transaction data summary table according to the query condition corresponding to the query instruction, and the query result is displayed to the user. For example, if the user needs to query the transaction details for 1 month, the first application data module may screen the transaction detail information hotlist for the data required by the user.
Optionally, in some embodiments, the first application data module may further support multilingual query requirements, and may provide multilingual query services through preset language identification, code tables and translation.
According to the transaction inquiry system provided by the embodiment of the application, the timeliness and accuracy of receiving the real-time transaction data can be improved through the CMQ and the receiving mechanism of the real-time transaction data of the Flink. Through the multi-layer design, the data in the transaction system can be uniformly managed in a layering way, so that the data of different business scenes are not interfered with each other, and the data processing coupling degree is reduced. The HATP database is used for storing real-time transaction data processed for a plurality of times, so that the impact of high concurrency query on a transaction query system can be reduced, the query efficiency is effectively improved, and the satisfaction degree of users is further improved.
Fig. 2 is a schematic diagram of a second structure of a transaction query system according to an embodiment of the present application, where, on the basis of the embodiment shown in fig. 1, the transaction query system further includes processing historical transaction data, as shown in fig. 2, and the transaction query system further includes: data lake.
The source data layer further comprises a second source data processing module based on a Spark architecture of the computing engine, the basic public layer comprises a second basic public module based on the Spark architecture, and the application data layer comprises a second application data module based on the Spark architecture.
In the embodiment of the application, spark is a fast and general calculation engine designed for large-scale data processing, supports functions of SQL query, text processing, machine learning and the like,
illustratively, the data lake is configured to store historical transaction data of the transaction querying system.
In the embodiments of the present application, a data lake is a repository or system that stores data in a raw format, which stores data as it is, without prior structuring of the data. A data lake may store structured data (e.g., tables in a relational database), semi-structured data (e.g., CSV, log, XML, JSON), unstructured data (e.g., email, document, PDF) and binary data (e.g., graphics, audio, video).
When the data stored in the data lake is acquired, the data lake can be acquired through interaction with a data lake tool Hive. Hive can map data files in a data lake into one or more database tables and provide SQL query functionality.
The historical transaction data can comprise historical transaction data obtained from the abutting existing database by the transaction inquiry system, and can also comprise summarized data which are issued by the first application data module and are subjected to secondary processing on the real-time transaction data. That is, all data of the transaction inquiry system is stored in the data lake.
Illustratively, the second source data module is configured to obtain historical transaction data from the data lake batch and transfer the historical transaction data to the second underlying public module.
In the embodiment of the application, the second source data module acquires the historical transaction data in batches from the data lake through interaction with the data lake. For example, historical transaction data query requests are sent to Hive in batches through Spark SQL query sentences, the Hive obtains storage positions of historical transaction data corresponding to the query requests in a data lake according to the query requests and a database table, obtains the historical transaction data from the data lake according to the storage positions of the historical transaction data, and then sends the historical transaction data to a second source data module.
Because all data of the transaction inquiry system are stored in the data lake, when the second source data module acquires the historical transaction data in batches, all transaction data of each user can be acquired simultaneously, so that the second basic public module forms different types of full data of each user according to all transaction data of each user.
Optionally, in one possible implementation manner, the second source data module may obtain, from the database table, a storage location of the historical transaction data corresponding to the query request in the data lake through interaction with Hive, and the second source data module may read the historical transaction data from the data lake according to the storage location of the historical transaction data.
In this embodiment of the present application, after the second source data module obtains the historical transaction data in batches from the data lake, the historical transaction data may be stored, and the historical transaction data may be sent to the second basic public module. So that the second basic public module processes the historical transaction data.
Optionally, in a possible implementation manner, after the second source data module obtains the historical transaction data from the data lake in batches, the historical transaction data may be stored. And when receiving the Spark SQL-based query request sent by the second basic public module, sending the corresponding historical transaction data to the second basic public module. The second source data module may store the historical transaction data obtained in batch as it is. Alternatively, the second source data module may store historical transaction data after preliminary classification. For example, the preliminary classification is performed according to different information types in the historical transaction information. The information types may include: transaction text, transaction currency information, and the like.
The second basic public module is used for processing the historical transaction data read in batches to generate a historical transaction data summary table with multiple data dimensions for each user.
In the embodiment of the application, the historical transaction data summary table of multiple data dimensions of each user may be a data table for summarizing historical transaction data of different dimensions of the user in different manners. Such as a user panorama information data table, a user asset data table, a user billing data table, a user transaction detail information table, etc.
For example, after the second basic public module obtains the historical transaction data, the historical transaction data may be processed according to a preset data processing policy, or a preset data processing model, to obtain a transaction data summary table of multiple data dimensions of each user.
In this embodiment of the present application, when the second basic public module generates the transaction data summary table of multiple data dimensions of each user, the transaction data summary table of multiple data dimensions of each user may be stored, and the transaction data summary table of multiple data dimensions of each user is sent to the second application data module. Or when the second basic public module receives the data acquisition request based on Spark SQL sent by the second application data module, packaging transaction data summary tables of multiple data dimensions of each user needed by the second application data module, and sending the transaction data summary tables to the second application data module.
The second application data module is configured to perform secondary processing on a historical transaction data summary table of multiple data dimensions of each user, store the processed historical transaction data summary table in the HATP database, and provide a historical transaction data query service through each user transaction data summary table in the HATP database.
The transaction data summary table of multiple data dimensions of each user can be subjected to secondary processing, so that the transaction data summary table of multiple data dimensions of each user can be subjected to classification processing for the type of query service required by the user, and different types of classified transaction data summary tables can be obtained. For example, the transaction details information table of each user may be classified according to the transaction details information table of each user such as a subscription service, a billing service, and a financial calendar service of the user, where the classified transaction data summary tables with multiple data dimensions are stored in the HATP database.
In this embodiment of the present application, the second application data module may also provide a user-oriented transaction query interface for a user-oriented module of the transaction query system, where the user may input a query instruction through the transaction query interface.
When the second application data module receives the query instruction, the second application data module can query data in the corresponding classified historical transaction data summary table according to the query condition corresponding to the query instruction, and display a query result to the user. For example, if the user needs subscription data, the second application data module may query the classification table corresponding to the subscription service for the data needed by the user.
According to the transaction inquiry system provided by the embodiment of the application, through the data processing mechanism based on the spark architecture, the full-quantity data of each user can be obtained from the data lake, the full-quantity data of each user is classified, different types of historical transaction data inquiry services are provided for each user, different inquiry requirements of the user are met, and therefore the satisfaction degree of the user is improved.
Fig. 3 is a schematic diagram third of the structure of the transaction query system provided in the embodiment of the present application, and on the basis of the embodiments shown in fig. 1 and fig. 2, each module and function included in the transaction query system are further described, as shown in fig. 3:
the transaction inquiry system provided by the embodiment of the application comprises the steps of acquiring real-time transaction data to provide real-time transaction inquiry service for a user and acquiring historical transaction data in batches to provide historical transaction inquiry service for the user. As can be seen from the embodiments shown in fig. 1 and 2:
the transaction inquiry system provided by the embodiment of the application comprises a source data layer, a basic public layer and an application data layer. The source data layer comprises a first source data module and a second source data module, the basic public layer comprises a first basic public module and a second basic public module, and the application data layer comprises a first application data module and a second application data module.
And the first source data module, the first basic public module and the first application data module based on the Flink architecture process the acquired real-time transaction data to provide real-time transaction inquiry service for the user.
And the second source data module, the second basic public module and the second application data module based on the Spark architecture process the acquired historical transaction data to provide historical transaction inquiry service for the user.
The functions of the modules providing real-time transaction data processing will be described in turn.
The first source data module is specifically configured to:
acquiring real-time transaction data from the CMQ; performing data cleaning on the real-time transaction data to obtain a temporary transaction interface table and a temporary transaction information table, wherein the temporary transaction interface table comprises asset data in the real-time transaction data, and the temporary transaction information table comprises scene data in the real-time transaction data; and sending the temporary transaction interface table and the temporary transaction information table to the first basic public module.
In embodiments of the present application, the data system of the financial institution generally includes a core system and a peripheral system, for example, the core system may include a deposit system, a loan system, etc., and the peripheral system may include an accumulation system, an insurance system, etc. The temporary transaction interface table may also be referred to as a core completed transaction interface table, which includes the complete real-time transaction information of the user in the core system, and the temporary transaction information table may also be referred to as a peripheral transaction information table, which includes the real-time transaction information of the user in the peripheral system. The temporary transaction interface table and the temporary transaction information table are data tables temporarily stored in the source data layer, and after the first source data module sends the temporary transaction interface table and the temporary transaction information table to the first basic public module, or when the first basic public module reads the temporary transaction interface table and the temporary transaction information table from the first source data module, the first source data module can delete the temporary transaction interface table and the temporary transaction information table.
Optionally, in some embodiments, when the first source data module obtains the real-time transaction data, in order to improve accuracy of the data, the real-time transaction data may be cleaned, and the cleaned real-time transaction data may be sorted to generate the temporary transaction interface table and the temporary transaction information table. And stores the cleaned real-time transaction data.
Optionally, in some embodiments, when each peripheral product and each channel send data to CMQ, the message format can be changed from the fixed-length format to the JSON format, so that the universality of message reporting can be improved.
In this embodiment, the implementation manner of the first source data module to obtain the real-time transaction data from CMQ and send the temporary transaction interface table and the temporary transaction information table to the first basic public module is similar to the implementation manner in the embodiment shown in fig. 1, and is not described herein in detail.
It will be appreciated that the source data layer still stores the raw real-time transaction data obtained from CMQ by the first source data module. The data stored in the temporary transaction interface table and the temporary transaction information table are used to provide data support for the underlying transaction data area in the underlying public layer.
Exemplary, the first basic common module is specifically configured to:
And processing data of the temporary transaction interface table and the temporary transaction information table to obtain transaction data of the transaction detail information table, the transaction summary information and the plurality of query dimensions.
The transaction summary information comprises a transaction summary list and a transaction summary index list, and the transaction data of the plurality of query dimensions at least comprises organization data, product data, commodity data, currency data, merchant data and function data.
As shown in fig. 3, the basic public layer includes a basic transaction data area for storing data processed with real-time transaction information and a batch public data area for storing data processed with historical transaction data.
The basic transaction data area comprises a transaction data summarization area, a transaction data detail area and a dimension data area. The transaction data summarization area is used for storing a transaction summarization statement list obtained by data processing of the temporary transaction interface list and the temporary transaction information list, and the transaction summarization statement list comprises a multi-dimensional statement list and a transaction summarization index statement list. The transaction data detail area is used for storing a transaction detail information table obtained by processing data of the temporary transaction interface table and the temporary transaction information table. The dimension data area is used for storing each dimension data obtained by data processing of the temporary transaction interface table and the temporary transaction information table. The transaction detail information table is also called a complete transaction detail information table, and each dimension data is also called transaction data of multiple query dimensions.
In the embodiment of the application, the multi-dimensional detail table may be a data table of transaction detail data of each user summarized in different dimensions. The transaction summary index wide table may be a data table that summarizes each user according to different indexes, for example, a total transaction amount, a number of transactions, and the like. The complete transaction detail information table may be a data table storing all transaction detail data for each user. The respective dimension data may be a profile of user transaction data that is counted in different dimensions.
In this embodiment of the present application, when the first basic public module obtains the temporary transaction interface table and the temporary transaction information, the first basic public module may process the temporary transaction interface table according to different data processing policies, so as to generate each data table stored in the basic transaction data area.
In this embodiment of the present application, the implementation manner in which the first basic common module obtains each data table from the first source data module and sends each data table to the first application data module is similar to the implementation manner in the embodiment shown in fig. 1, and is not repeated herein.
It will be appreciated that the data tables in the underlying transaction data area are used to provide data support for transaction services in the underlying public layer.
The first application data module is specifically configured to:
And carrying out secondary processing on the transaction data summary table with the multiple data dimensions according to the generation time of each data in the transaction data summary table with the multiple data dimensions to obtain transaction data summary tables with at least two time spans with each data dimension, and storing the transaction data summary tables in the HATP database.
With continued reference to fig. 3, the application data layer may be a data processing layer that provides services to users. The application data layer includes a first application data module that provides transaction services to the user based on the data tables in the base transaction data area.
In order to provide transaction inquiry services of various time spans for users, when the first application data module obtains transaction data summary tables of various data dimensions, the transaction data summary tables of various data dimensions can be split according to the generation time of each data of the transaction data summary tables of various data dimensions, so as to obtain a plurality of corresponding data summary tables.
For example, the complete transaction information list may be classified into three types of a complete transaction information list (hot), a complete transaction information list (warm), and a complete transaction information list (cold) according to the time of data generation. When the user inquires the transaction information details, the first application data module can determine which detail table belongs to according to the inquiry time period selected by the user, and inquire corresponding data in the table, so that the inquiry efficiency is improved.
The multidimensional detail wide table, the transaction summary index wide table and the multidimensional data can be divided into a plurality of corresponding data tables according to the data generation time.
Optionally, in some embodiments, if the user performs the transaction query, the application data layer may further push the query result of the user to the user, so as to facilitate the user to check.
Illustratively, the application data layer further includes: a pushing module;
the first basic public module is further used for sending the real-time transaction data to the pushing module; the pushing module is used for pushing the real-time transaction data to the message downlink gateway; and the message downlink gateway sends the real-time transaction data to a user terminal corresponding to the real-time transaction data.
The first application data module is further used for carrying out batch secondary processing on real-time transaction data in the pushing period according to a preset pushing period to obtain a transaction summarizing result in the pushing period; the pushing module is further configured to batch push the transaction summary result in the pushing period to the message downlink gateway and/or the external sharing platform.
In this embodiment of the present application, the message downlink gateway may include at least one of a short message, an APP message, a mail, and a 5G message. When the user performs transaction, or the user performs transaction inquiry, the first application data module can push real-time transaction data or transaction inquiry results to the message downlink gateway, so that the message downlink gateway pushes the message downlink gateway to the terminal of the user, and the user can check conveniently.
In the embodiment of the application, the external sharing platform may be an external system connected to the transaction inquiry system, for example, a third party enterprise system. When the first application data module performs data pushing to the user, batch pushing can be performed according to a preset period, so that the user can conveniently know the periodic transaction summarizing result. That is, the first application data module may integrate the data in the preset pushing period, for example, after sorting according to time, form a transaction text, and push the transaction text to the message downlink gateway and/or the external sharing platform for the user to view. Such as business day end account balance information.
Optionally, in one possible implementation manner, when the user performs a transaction, the transaction query system may directly push real-time transaction information of the user to the message downlink gateway through CMQ, so that the message downlink gateway pushes the message to the terminal of the user, and timeliness of message pushing is improved.
The functions of the modules providing the historical transaction data processing will be described in turn.
The second source data module is specifically configured to:
acquiring historical transaction data from the data lakes in batches in a subscription mode; generating a historical transaction text, currency information of a historical transaction and historical transaction information except the historical transaction text and the currency information of the historical transaction according to the historical transaction data; and sending the historical transaction text, the currency information of the historical transaction and the historical transaction information to the second basic public module.
In the embodiment of the present application, the subscription may be to obtain the historical transaction data from the data lake in batches according to a fixed period. For example, all historical transaction data for 1 user is obtained from a data lake on a 10 minute cycle. The batch acquisition of the historical transaction data in the data lake can be that all historical transaction data of the user are acquired from the Hive area of the data lake in a mode of subscribing to the data lake table.
In this embodiment of the present application, when the second source data module obtains the historical transaction data, the historical transaction data may be classified and processed, so as to generate a historical transaction text, currency information of the historical transaction, and historical transaction information except for the historical transaction text and the currency information of the historical transaction. The historical transaction text is transaction text generated by a peripheral transaction system, and the currency information can be all information about the currency of the user, such as a subscription account, an asset, a credit card and the like.
In this embodiment, the manner in which the second source data module acquires the historical transaction data from the data lake in batches, and the specific implementation manner in which the historical transaction text, the currency information of the historical transaction, and the historical transaction information are sent to the second basic public module are similar to the implementation manner in the embodiment shown in fig. 2, and are not repeated herein.
The second basic public module is used for processing the acquired historical transaction text, the acquired currency information of the historical transaction and the acquired historical transaction information by adopting different data processing strategies to generate a historical transaction data summary table with multiple data dimensions for each user.
With continued reference to FIG. 3, a historical transaction data summary table for each user's various data dimensions is stored in a bulk common data area, including at least one of the following types:
individual dimension data, user billing, receipt data, user panoramic data, financial related data, complete transaction information details list, and user asset data.
Wherein each dimension data includes at least one of the following types:
card-to-household relationship data, subject data, card price data, contract data, product data, currency data, institution data, merchandise data, account data, deposit account data, user data, loan account data, institution public data, and credit card data.
The user bill and receipt data comprises at least one of the following types:
positive data, receipt verification data, receipt rule data, short message data, receipt affiliated information data, receipt basic data, subscription data, bill data, and receipt data.
The user panorama data comprises at least one of the following types:
user master data, address information data, financial status data, communication information data, identification data, web banking client data, employment information data, electronic information data, web banking user data, and other information data of the user.
The user asset data includes at least one of the following types:
periodic data, foreign exchange data, life data, loan data, fund data, bond data, financial data, paper gold data, daily asset data, and monthly asset data.
The finance-related data includes at least one of the following types:
financial calendar related data (expiration, repayment day), payroll data, and customer pre-sale data.
In the embodiment of the application, through multidimensional analysis and summarization of the client data, richer data transaction inquiry service can be provided for the user, so that the user can conveniently know own asset condition.
It will be appreciated that the above data may be stored in the form of a data table in a bulk common data area of the underlying common layer.
In this embodiment, the manner in which the second basic public module obtains the historical transaction text, the currency information of the historical transaction, and the specific implementation manner of the historical transaction information from the source data layer, and sends the historical transaction data summary table of multiple data dimensions of each user to the second application module is similar to the implementation manner in the embodiment shown in fig. 2, and will not be described herein again.
The second application data module is specifically configured to provide at least one of a subscription service, a billing-related service, a financial calendar service, and a non-financial service.
In the embodiment of the application, the second application data module can provide multiple functional services for the user by performing classification processing on the historical transaction data summary table with multiple data dimensions of each user.
With continued reference to fig. 3, the data of different dimensions used by the subscription service includes at least one of subscription data, card user relationship data, account data, and customer data.
Subscription services can be classified into data push type subscription and data processing type subscription. The data push type subscription can be completed by calling subscription service for the subscription channel side. Such as a dynamic account notification, a large transaction reminder. The data processing subscription can be that other business systems send subscription texts to a data lake through a data bus on the T-1 day, and subscription services splice appointed opponent information, transaction scene information and the like for appointed type transaction running water according to the subscription texts, such as financial special account subscription.
In one possible implementation, the second application data module may also synchronize subscription information to the data lake in real time through CMQ.
The bill related services include a receipt related service and a bill related service, and the data of different dimensions used include: receipt related data, account data, credit card data, and customer data. Through the billing related service, a full amount of billing query service may be provided to the user.
The data of different dimensions used by the financial calendar service include: expiration class data, repayment class data, authorization data, and user data. The financial calendar service can provide more detailed financial statistics service for the user, and statistical analysis is realized.
The data of different dimensions for non-financial services includes: non-financial consumption data, non-financial consumption behavior data, credit card data, and user data. The user can more comprehensively know the consumption condition of the user through the non-financial service.
In this embodiment of the present application, the specific implementation manner of the historical transaction data summary table of multiple data dimensions of each user of the second application data module from the second basic public module is similar to the implementation manner in the embodiment shown in fig. 2, and will not be repeated here.
The second application data module is further used for sending a historical transaction data summary table of multiple data dimensions of each user after secondary processing to a data lake for storage.
In this embodiment, please continue to refer to fig. 3, the application data layer further includes a data downloading area, where the data downloading area is configured to download historical transaction summary data processed by the application data layer to the data lake, and archive and backup the historical transaction summary data. To improve the reliability of data storage. The downloading of the historical transaction summary data of the secondary processing to the data lake can be realized through a second application data module.
Optionally, in some embodiments, the data downloading area is further configured to download the real-time transaction summary data processed secondarily by the application data layer to a data lake for archiving and backup. The downloading of the real-time transaction summary data of the secondary processing can be realized by a first application data module.
Optionally, in some embodiments, when the data downloading area downloads the data, the data to be downloaded may be further summarized and then downloaded to the data lake. For example, 1 day of data is collected and packaged and then downloaded into a data lake.
Optionally, in some embodiments, the data downloading area may also download data that is processed secondarily by the application data layer to other data systems. Other data systems may be acquired through subscription.
In summary, in the transaction query system provided by the embodiment of the application, by designing the architecture of the application data layer, the basic public layer and the source data layer, the data in the system is uniformly managed in a layered manner according to the preset data processing mode, so that different service scenes are not interfered with each other, and the data processing coupling degree and the public data sharing are reduced.
Aiming at a high concurrency warehouse-in query scene, high concurrency writing and query are realized by using an HTAP database, and pressure is dispersed by adopting a read-write separation mode aiming at hot data. And improving the response capability of the data.
And the online transaction data of the message middleware is received through the Flink, and the batch data of the data lake is received through the Spark/Hive, so that the timeliness and accuracy of data acquisition are improved.
By using the data storage mode of HTAP+data lake, the requirements on online, near-online and offline storage are realized, and the safety of data storage is improved.
The requirements of users on data query are met through real-time data pushing and batch data pushing, and the use experience of the users is improved.
Aiming at the multidimensional statistical analysis requirement of the user, the HTAP database is adopted for data storage and preprocessing, so that the multidimensional statistical analysis requirement of the user is met.
In this specification, each embodiment is described in a progressive manner, and all the embodiments are directly the same or similar parts referring to each other, and each embodiment mainly describes differences from other embodiments. In particular, for system embodiments, since they are substantially similar to method embodiments, the description is relatively simple, as relevant to see a section of the description of method embodiments. It should be noted that, any combination of the technical features of the foregoing embodiments may be used, and for brevity, all of the possible combinations of the technical features of the foregoing embodiments are not described, however, as long as there is no contradiction between the combinations of the technical features, they should be considered as the scope of the description.
Other embodiments of the present application will be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the application following, in general, the principles of the application and including such departures from the present disclosure as come within known or customary practice within the art to which the application pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the application being indicated by the following claims.
It is to be understood that the present application is not limited to the precise arrangements and instrumentalities shown in the drawings, which have been described above, and that various modifications and changes may be effected without departing from the scope thereof. The scope of the application is limited only by the appended claims.
Claims (10)
1. A transaction inquiry system, said transaction inquiry system comprising: a source data layer, a base common layer, an application data layer, and a distributed message queue CMQ; the source data layer comprises a first source data processing module based on a distributed processing engine (Flink) architecture, the basic public layer comprises a first basic public module based on the Flink architecture, the application data layer comprises a first application data module based on the Flink architecture, and the hybrid/analysis processing HATP database;
The CMQ is used for acquiring real-time transaction data of different transaction products and/or different transaction systems and/or different transaction channels;
the first source data module is configured to read real-time transaction data from the CMQ module and transfer the real-time transaction data to the first base common module;
the first basic public module is used for processing the real-time transaction data, generating a transaction data summary table with multiple data dimensions, and sending the transaction data summary table with the multiple data dimensions to the first application data module;
the first application data module is used for carrying out secondary processing on the transaction data summary tables with the multiple data dimensions, storing the transaction data summary tables in the HATP database, and providing real-time transaction data query service through the transaction data summary tables in the HATP database.
2. The transaction query system of claim 1, wherein the first source data module is configured to:
acquiring real-time transaction data from the CMQ;
performing data cleaning on the real-time transaction data to obtain a temporary transaction interface table and a temporary transaction information table, wherein the temporary transaction interface table comprises asset data in the real-time transaction data, and the temporary transaction information table comprises scene data in the real-time transaction data;
And sending the temporary transaction interface table and the temporary transaction information table to the first basic public module.
3. The transaction inquiry system according to claim 2, wherein the transaction data summary table of the plurality of data dimensions comprises: a transaction detail information table, transaction summary information and transaction data of a plurality of query dimensions; the transaction summary information comprises a transaction summary list and a transaction summary index list, and the transaction data of the plurality of query dimensions at least comprises mechanism data, product data, commodity data, currency data, merchant data and function data;
the first basic public module is specifically configured to:
and processing data of the temporary transaction interface table and the temporary transaction information table to obtain transaction data of the transaction detail information table, the transaction summary information and the plurality of query dimensions.
4. The transaction query system of claim 3, wherein the first application data module is specifically configured to:
and carrying out secondary processing on the transaction data summary table with the multiple data dimensions according to the generation time of each data in the transaction data summary table with the multiple data dimensions to obtain transaction data summary tables with at least two time spans with each data dimension, and storing the transaction data summary tables in the HATP database.
5. The transaction querying system of claim 1, wherein the application data layer further comprises: a pushing module;
the first basic public module is further used for sending the real-time transaction data to the pushing module;
the pushing module is used for pushing the real-time transaction data to the message downlink gateway; and the message downlink gateway sends the real-time transaction data to a user terminal corresponding to the real-time transaction data.
6. The transaction inquiry system according to claim 5, wherein,
the first application data module is further used for carrying out batch secondary processing on real-time transaction data in the pushing period according to a preset pushing period to obtain a transaction summarizing result in the pushing period;
the pushing module is further configured to batch push the transaction summary result in the pushing period to the message downlink gateway and/or the external sharing platform.
7. The transaction inquiry system according to any one of claims 1-6, wherein the transaction inquiry system further comprises: the data lake, the source data layer further comprises a second source data processing module based on a Spark architecture of a computing engine, the basic public layer comprises a second basic public module based on the Spark architecture, and the application data layer comprises a second application data module based on the Spark architecture;
The data lake is used for storing historical transaction data of the transaction inquiry system;
the second source data module is used for acquiring historical transaction data from the data lakes in batches and transmitting the historical transaction data to the second basic public module;
the second basic public module is used for processing the historical transaction data read in batches to generate a historical transaction data summary table of various data dimensions of each user;
the second application data module is used for performing secondary processing on historical transaction data summary tables of multiple data dimensions of each user, storing the processed historical transaction data summary tables in the HATP database, and providing historical transaction data query service for each user transaction data summary table in the HATP database.
8. The transaction query system of claim 7, wherein the second source data module is configured to:
acquiring historical transaction data from the data lakes in batches in a subscription mode;
generating a historical transaction text, currency information of a historical transaction and historical transaction information except the historical transaction text and the currency information of the historical transaction according to the historical transaction data;
And sending the historical transaction text, the currency information of the historical transaction and the historical transaction information to the second basic public module.
9. The transaction query system of claim 8, wherein the second application data module is specifically configured to provide at least one of a subscription service, a billing-related service, a financial calendar service, and a non-financial service.
10. The transaction query system of claim 9, wherein the second application data module is further configured to send a historical transaction data summary of the plurality of data dimensions for each user after the secondary processing to a data lake for storage.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310473486.XA CN116467352A (en) | 2023-04-27 | 2023-04-27 | Transaction inquiry system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310473486.XA CN116467352A (en) | 2023-04-27 | 2023-04-27 | Transaction inquiry system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116467352A true CN116467352A (en) | 2023-07-21 |
Family
ID=87173340
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310473486.XA Pending CN116467352A (en) | 2023-04-27 | 2023-04-27 | Transaction inquiry system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116467352A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116821192A (en) * | 2023-08-30 | 2023-09-29 | 中国中金财富证券有限公司 | Asset data query method and device and computer equipment |
-
2023
- 2023-04-27 CN CN202310473486.XA patent/CN116467352A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116821192A (en) * | 2023-08-30 | 2023-09-29 | 中国中金财富证券有限公司 | Asset data query method and device and computer equipment |
| CN116821192B (en) * | 2023-08-30 | 2023-11-24 | 中国中金财富证券有限公司 | Asset data query method and device and computer equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11373261B1 (en) | Automated analysis of data to generate prospect notifications based on trigger events | |
| US9466063B2 (en) | Cluster processing of an aggregated dataset | |
| CN111382150B (en) | Real-time computing method and system based on Flink | |
| US20080288522A1 (en) | Creating and storing a data field alteration datum using an analytic platform | |
| US20200028701A1 (en) | Systems and methods for alert services | |
| AU5110301A (en) | Electronic bill presentment and payment systems and processes | |
| US20060259423A1 (en) | Centralized payment processing system | |
| CN112506743A (en) | Log monitoring method and device and server | |
| CN115422173A (en) | Data management method and system in financial credit field | |
| CN112581290A (en) | Block chain-based equipment pledge financing method, system, electronic equipment and medium | |
| CN116467352A (en) | Transaction inquiry system | |
| CN114548963B (en) | Payment interaction processing method and device | |
| US20100070406A1 (en) | Integrated mortgage and real estate origination system | |
| Balakrishnan et al. | Implementing data strategy: design considerations and reference architecture for data-enabled value creation | |
| CN114565443B (en) | Data processing method, data processing device, computer equipment and storage medium | |
| CN111444212A (en) | Unified accounting method and device based on accounting system and accounting system | |
| HRP et al. | EXPENDITURE CYCLE: E-INVOICING AND E-PAYMENT TO SAVE TRANSACTION COSTS AND INCREASE THE NUMBER OF CUSTOMERS IN THE MANUFACTURING INDUSTRY | |
| CN112488860B (en) | Method and system for processing group list | |
| Neuman | A Historical Overview of the Computerization of the Internal Revenue Service | |
| CN112381514B (en) | Service data reminding method, device and server | |
| Konana et al. | Design of time cognizant electronic brokerages | |
| US20220164368A1 (en) | Management of data warehouse for electronic payment transaction processing networks | |
| CREMA | A graph approach to settlement instructions status prediction | |
| CN118689947A (en) | Cloud-based data processing and storing system and method | |
| Thomson | Practical studies in e-governance: an empirical exploration of enterprise resource planning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |