CN116303593A - Transaction data real-time query method and device, electronic equipment and storage medium - Google Patents
Transaction data real-time query method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN116303593A CN116303593A CN202310126989.XA CN202310126989A CN116303593A CN 116303593 A CN116303593 A CN 116303593A CN 202310126989 A CN202310126989 A CN 202310126989A CN 116303593 A CN116303593 A CN 116303593A
- Authority
- CN
- China
- Prior art keywords
- transaction
- file
- transaction data
- query
- real
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 53
- 238000004364 calculation method Methods 0.000 claims abstract description 85
- 238000004458 analytical method Methods 0.000 claims description 33
- 238000007726 management method Methods 0.000 claims description 16
- 238000004891 communication Methods 0.000 claims description 11
- 238000004590 computer program Methods 0.000 claims description 7
- 230000005540 biological transmission Effects 0.000 claims description 3
- 230000007246 mechanism Effects 0.000 claims description 3
- 230000008569 process Effects 0.000 description 10
- 238000010586 diagram Methods 0.000 description 6
- 238000012545 processing Methods 0.000 description 6
- 238000012546 transfer Methods 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000002159 abnormal effect Effects 0.000 description 1
- 230000009471 action Effects 0.000 description 1
- 125000004122 cyclic group Chemical group 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000000737 periodic effect Effects 0.000 description 1
- 239000010453 quartz Substances 0.000 description 1
- VYPSYNLAJGMNEJ-UHFFFAOYSA-N silicon dioxide Inorganic materials O=[Si]=O VYPSYNLAJGMNEJ-UHFFFAOYSA-N 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000011144 upstream manufacturing Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2228—Indexing structures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
- G06F16/2379—Updates performed during online database operations; commit processing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/248—Presentation of query results
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Abstract
The application discloses a transaction data real-time query method, a device, electronic equipment and a storage medium, wherein the device comprises: generating a calculation instruction of each transaction data index based on each field in the transaction detail; receiving a transaction detail file; analyzing the transaction detail file, extracting the value corresponding to each field, and generating the structured data corresponding to the transaction detail file; determining real-time calculation results of all transaction data indexes based on calculation instructions of all transaction data indexes and structured data corresponding to the transaction detail files; and generating a query result corresponding to the query request based on the transaction data indexes to be queried in the query request and real-time calculation results of all the transaction data indexes. The method and the device reduce the cost of each requester system, and the data queried by each requester system can keep real-time performance and consistency.
Description
Technical Field
The present application relates to the field of computer technologies, and in particular, to a method and apparatus for real-time query of transaction data, an electronic device, and a storage medium.
Background
After analysis, the transaction details of the banking institution can be widely applied to business scenes such as credit approval, post-credit management and the like. Specifically, a data summarizing platform such as a data warehouse sends a transaction detail file to each requester system, and each requester system respectively combines with service requirements to complete operations such as analysis of the transaction detail file, data index processing and the like.
In this way, the requestor systems analyze the transaction detail files independently, which is costly and has low technical reuse. And the data inquiry among the systems is realized through file interaction, so that the real-time performance is poor.
Disclosure of Invention
The application provides a transaction data real-time query method, a device, electronic equipment and a storage medium, which are used for solving the technical problems of high cost and poor real-time performance of the existing transaction data query method.
The application provides a transaction data real-time query method, which comprises the following steps:
generating a calculation instruction of each transaction data index based on each field in the transaction detail;
receiving a transaction detail file;
analyzing the transaction detail file, extracting the value corresponding to each field, and generating the structured data corresponding to the transaction detail file;
determining real-time calculation results of all transaction data indexes based on calculation instructions of all transaction data indexes and structured data corresponding to the transaction detail files;
and generating a query result corresponding to the query request based on the transaction data indexes to be queried in the query request and real-time calculation results of all the transaction data indexes.
In some embodiments, the receiving a transaction detail file includes:
determining a file storage catalog of the transaction detail file based on the issuing time of the transaction detail file;
and after the issuing of the transaction detail file is finished, generating an issuing finishing mark file corresponding to the transaction detail file in the file storage catalog.
In some embodiments, the parsing the transaction detail file, extracting values corresponding to respective fields, and generating structured data corresponding to the transaction detail file includes:
polling the file storage directory, and determining that the file storage directory contains the transaction detail file and a issuing end mark file corresponding to the transaction detail file;
determining that the transaction detail file is in a delivery completion state based on the delivery end mark file;
extracting the corresponding values of each field in the transaction detail file;
and generating structured data corresponding to the transaction detail file based on each field and the value corresponding to each field.
In some embodiments, the generating the query result corresponding to the query request based on the transaction data indexes to be queried in the query request and the real-time calculation result of each transaction data index includes:
receiving a current query request;
matching the current query request with each query request in a query waiting queue;
under the condition that the current query request is not matched with each query request in the query waiting queue, acquiring the analysis state of the transaction detail file and/or the execution state of a calculation instruction of each transaction data index;
generating a query result corresponding to the current query request based on real-time calculation results of all transaction data indexes under the conditions that the analysis state of the transaction detail file is analysis completion and the execution state of the calculation instruction is execution completion;
and adding the current query request to the query waiting queue when the analysis state of the transaction detail file is not analyzed or the execution state of any calculation instruction is not executed.
The application provides a transaction data real-time inquiry device, including:
the index configuration module is used for generating calculation instructions of all transaction data indexes based on all fields in the transaction detail;
the file analysis module is used for receiving the transaction detail file; analyzing the transaction detail file, extracting the value corresponding to each field, and generating the structured data corresponding to the transaction detail file;
the index management module is used for determining real-time calculation results of all transaction data indexes based on calculation instructions of all transaction data indexes and structured data corresponding to the transaction detail files; and generating a query result corresponding to the query request based on the transaction data indexes to be queried in the query request and real-time calculation results of all the transaction data indexes.
In some embodiments, the apparatus further comprises:
and the task scheduling module is connected with the index configuration module, the file analysis module and the index management module and used for controlling each module to perform data transmission based on a preset flow.
In some embodiments, the apparatus further comprises:
and the message queue module is connected with the index configuration module, the file analysis module and the index management module and is used for realizing asynchronous communication among the models based on a message queue mechanism.
In some embodiments, the index management module is connected with each requester system based on an Http interface and is configured to receive a query request sent by each requester system and return a query result to each requester system.
The application provides an electronic device comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the transaction data real-time query method through the computer program.
The application provides a computer readable storage medium, which comprises a stored program, wherein the program executes the transaction data real-time query method.
According to the transaction data real-time query method, the device, the electronic equipment and the storage medium, the received transaction detail file is analyzed, the value corresponding to each field is extracted, the structured data corresponding to the transaction detail file is generated, the real-time calculation result of each transaction data index is determined according to the calculation instruction of each transaction data index and the structured data corresponding to the transaction detail file, the query result corresponding to the query request is generated according to the transaction data index to be queried in the query request and the real-time calculation result of each transaction data index, the structured data is obtained by analyzing the transaction detail file, and the real-time calculation result of each transaction data index is obtained by calculating the structured data, so that each requesting party system can directly query each transaction data index without analyzing the transaction detail file independently, the cost of each requesting party system is reduced, the technical multiplexing degree of the analysis result of the transaction detail file is improved, and the real-time calculation result of each transaction data index in the process is kept updated in real time along with the acquisition of the transaction detail file, and the queried data of each requesting party system can keep real-time consistency.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the application and together with the description, serve to explain the principles of the application.
In order to more clearly illustrate the technical solutions of the present application or the prior art, the following description will briefly introduce the drawings used in the embodiments or the description of the prior art, and it is obvious that, in the following description, the drawings are some embodiments of the present application, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flow chart of a method for real-time query of transaction data according to one embodiment of the present application;
FIG. 2 is a flow chart of a transaction detail file parsing method according to one embodiment of the present application;
FIG. 3 is a flow chart of a method for real-time query of transaction data according to another embodiment of the present application;
fig. 4 is a schematic structural diagram of a transaction data real-time query device according to an embodiment of the present application;
fig. 5 is a schematic structural diagram of a transaction data real-time query device according to another embodiment of the present application;
fig. 6 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
In order to make the present application solution better understood by those skilled in the art, the following description will be made in detail and with reference to the accompanying drawings in the embodiments of the present application, it is apparent that the described embodiments are only some embodiments of the present application, not all embodiments. All other embodiments, which can be made by one of ordinary skill in the art based on the embodiments herein without making any inventive effort, shall fall within the scope of the present application.
It should be noted that the terms "first," "second," and the like herein are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that embodiments of the present application described herein may be implemented in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or units or modules is not necessarily limited to those steps or units or modules that are expressly listed or inherent to such process, method, article, or apparatus.
In the technical scheme of the application, the related processes of collecting, storing, using, processing, transmitting, providing, disclosing and the like of the client personal information all accord with the regulations of related laws and regulations, necessary security measures are adopted, and the public order harmony is not violated.
Fig. 1 is a flow chart of a method for real-time query of transaction data according to an embodiment of the present application, as shown in fig. 1, the method includes steps 110, 120, 130, 140 and 150.
Specifically, the execution subject of the method provided by the embodiment of the application is a transaction data real-time query device. The transaction data real-time inquiry device can be realized by software, such as a transaction data real-time inquiry program running in a computer; the transaction data real-time query method can also be a device for executing the transaction data real-time query method, such as a mobile terminal, a tablet computer, a desktop computer or a server.
Transaction details refer to a detailed record of a customer's transactions. For example, a customer's transaction details at a banking institution refer to a detailed record of the customer's entry and exit of funds in the banking institution's account, which may also be referred to as banking pipelining.
A field refers to the name of a data category having the same attribute in the transaction detail. For example, the fields in the transaction details may be transaction time, transaction amount, transaction type, and the like.
The transaction data indicator refers to a parameter that may be used to measure a customer's transaction characteristics. For example, the transaction data indicators may include abnormal funds flow, back flow of funds, upstream and downstream customer changes, close related transactions, frequent funds transfers, large fluctuations in funds, payroll anomalies, utility rate anomalies and various types of homonymies, cyclic ratios, periodic changes, and the like.
And calculating the numerical values corresponding to all or part of fields in the transaction details according to a certain operation rule to obtain the numerical values corresponding to the transaction data indexes. For example, the transaction amount for each transaction period in the transaction detail may be accumulated to obtain the transaction total. For some transaction data indexes, it may be necessary to solve the intermediate values according to the values of the partial fields, and then calculate the values corresponding to the transaction data indexes. The fields corresponding to these intermediate values are not present in the transaction details and may be referred to as derivative fields. For another example, the transaction increment rate can be obtained by calculating the transaction amount of the current year, and then subtracting the transaction amount of the previous year from the transaction amount of the previous year to obtain the transaction amount increment, and then comparing the transaction amount increment with the transaction amount of the previous year.
The operation rules are determined according to the association relation between each field and each transaction data index in the transaction detail. The calculation instruction is a command for executing a corresponding operation rule, and may be embodied in a code form.
Specifically, the transaction detail file is a file recording customer transaction details. And a data summarization platform such as a data warehouse and the like can be arranged for receiving the transaction detail files generated by each transaction system, so that unified management of the transaction detail files is realized.
And the data summarizing platform transmits the collected transaction detail files to the transaction data real-time inquiring device.
Here, the transaction data real-time inquiry device may receive the transaction detail files sequentially in order, or may receive the transaction detail files in batch.
And 130, analyzing the transaction detail file, extracting the values corresponding to the fields, and generating the structured data corresponding to the transaction detail file.
Specifically, the transaction data real-time query device can analyze the transaction detail file after receiving the transaction detail file, and can sequentially extract the values corresponding to the fields according to the sequence of the fields in the transaction detail file.
These data are combined into structured data in fields and values, which are stored in a relational database.
And 140, determining a real-time calculation result of each transaction data index based on the calculation instruction of each transaction data index and the structured data corresponding to the transaction detail file.
Specifically, the calculation instruction of each transaction data index includes a field required to participate in calculation and a corresponding calculation rule.
Since the fields and values involved in the calculation are stored in the relational database in the form of structured data, the values of these fields can be read from the relational database by calculation instructions and calculated according to calculation rules to obtain the calculation result of each transaction data index.
Because the transaction detail file is generated and issued in real time, and the analysis of the transaction detail file is also performed in real time, the calculation result of each transaction data index is a real-time calculation result.
Specifically, the transaction data real-time query device may be connected to a plurality of requester systems, and receive query requests sent by the requester systems. Each inquiry request comprises a client number to be inquired, a transaction data index to be inquired and a requester system number. The data structure of the query request may be in JSON format.
The transaction data real-time query device can match the transaction data index to be queried with each transaction data index, extract the real-time calculation result of the matched transaction data index, generate a query result corresponding to the query request, and return the query result to the requester system.
Two databases, namely a relational database and a non-relational database (NoSQL), can be arranged in the transaction data real-time query device. The relational database can be used for storing structured data corresponding to the transaction detail file and real-time calculation results of various transaction data indexes. The non-relational database may be used as a cache database for real-time calculation of individual transaction data metrics. After the real-time calculation result is generated, the calculation result can be stored in a non-relational database. And after meeting a certain time limit requirement, storing the real-time calculation result into a relational database. Because the data searching speed of the non-relational database is high, the transaction data real-time searching device can quickly search the real-time calculation result of each transaction data index and quickly generate the searching result corresponding to the searching request. The non-relational database may be implemented by an open source database Redis.
According to the transaction data real-time query method provided by the embodiment of the application, the received transaction detail file is analyzed, the value corresponding to each field is extracted to generate the structured data corresponding to the transaction detail file, the real-time calculation result of each transaction data index is determined according to the calculation instruction of each transaction data index and the structured data corresponding to the transaction detail file, the query result corresponding to the query request is generated according to the transaction data index to be queried in the query request and the real-time calculation result of each transaction data index, and the structured data is obtained by analyzing the transaction detail file and then calculating the structured data, so that each requester system can directly query and obtain each transaction data index without analyzing the transaction detail file independently, the cost of each requester system is reduced, the technical multiplexing degree of analysis results of the transaction detail file is improved, and the real-time calculation result of each transaction data index in the process is kept updated in real time along with the acquisition of the transaction detail file, so that the queried data of each requester system can keep real-time and consistency.
It should be noted that each embodiment of the present application may be freely combined, permuted, or executed separately, and does not need to rely on or rely on a fixed execution sequence.
In some embodiments, step 120 comprises:
determining a file storage catalog of the transaction detail file based on the issuing time of the transaction detail file;
after the issuing of the transaction detail file is finished, generating an issuing end mark file corresponding to the transaction detail file in the file storage directory.
Specifically, the file storage directory of the transaction detail file may be determined according to the issuing time of the transaction detail file. For example, the transaction data real-time inquiry device may create a file storage directory with the date of the issue time as the name according to the issue time of the transaction detail file.
In order to determine that the transaction detail file is successfully issued, that is, determine that the transaction data real-time query device can receive the complete transaction detail file, an issue end mark file corresponding to the transaction detail file can be generated in the file storage directory. The issuing end flag file may use a preset number or letter as a file name or an extension, etc. as a flag for ending the issuing action of the transaction detail file.
The file storage directory may use the date (year, month, and day) corresponding to the delivery time of the transaction detail file as the directory name, and the following date is 2022, 7, and 1, and the file storage directory name may be 2022-7-1. After the transaction detail file is issued, the transaction detail file can be named as a file name consistent with the directory name. The format of the transaction detail file may be a text format or other editable document format. After the issuing is finished, generating an issuing finishing mark file corresponding to the transaction detail file in the file storage directory. The name of the issue end flag file may be added with ". OK" as an extension, for example, "2022-7-1.OK", on the basis of the file name of the transaction detail file. The file content of the issuing end flag file may be no content.
According to the transaction data real-time query method, after the transaction detail file is issued, the issuing end mark file corresponding to the transaction detail file is generated in the file storage directory, so that the complete transaction detail file can be confirmed to be received, and the accuracy of calculation of the transaction data index is improved.
Fig. 2 is a flow chart illustrating a transaction detail file parsing method according to an embodiment of the present application, as shown in fig. 2, in some embodiments, step 130 includes:
Specifically, the file storage directory may be polled to determine whether the file storage directory includes a transaction detail file, and an end-of-issue flag file corresponding to the transaction detail file.
If the issuing end flag file exists in the file storage directory, it may be determined that the transaction detail file is in the issuing complete state, that is, that a complete transaction detail file has been received.
The analysis of the transaction detail file can be started, the values corresponding to the fields in the transaction detail file are extracted, and the data are combined in a field and value combination mode to generate the structured data corresponding to the transaction detail file.
For example, after confirming that the transaction detail file is issued, a file reading flow is started, a payer, a payee, a transaction amount, a transaction date, a fund use, a serial number and the like are extracted, and written into a structured database, and the structured database can use a relational database MySQL.
According to the transaction data real-time query method provided by the embodiment of the application, the issuing state of the transaction detail file can be determined in real time by polling the file storage directory, so that the structured data corresponding to the transaction detail file is updated in real time.
Fig. 3 is a flow chart of a transaction data real-time query method according to another embodiment of the present application, as shown in fig. 3, in some embodiments, step 150 includes:
Specifically, the transaction data real-time query device may receive query requests sent by each requester system, and construct a query waiting queue to process a plurality of query requests.
After receiving the current query request, the current query request may be matched with each query request in the query wait queue. The matched content comprises the client number to be queried, the transaction data index and the requester system number in the current query request.
If the current query request matches any query request in the query waiting queue, the current query request is a repeated request, and the query state can be returned to the requester system as "in-query".
If the current query request does not match each query request in the query wait queue, then the current query request is indicated as a new query request. At this time, the analysis state of the transaction detail file and/or the execution state of the calculation instruction of each transaction data index may be acquired.
If the analysis state of the transaction detail file is analysis completion and the execution state of the calculation instruction is execution completion, the fact that the transaction data real-time query device does not receive a new transaction detail file at the moment, and no transaction data index is being calculated, namely the real-time calculation result of each transaction data index in the transaction data real-time query device is the latest result, and at the moment, the query result corresponding to the current query request can be generated according to the real-time calculation result of each transaction data index.
If the analysis state of the transaction detail file is not analyzed or the execution state of any calculation instruction is not executed, the fact that the transaction detail file is received possibly exists in the transaction data real-time query device at the moment, or the transaction data index is being calculated, that is, the real-time calculation result of each transaction data index in the transaction data real-time query device is not the latest result, at the moment, the completion of file analysis and the completion of data index calculation are required to be waited, the current query request can be added into a query waiting queue, meanwhile, the query state is returned to the requester system as 'in query', and a specific query result is returned to the requester system after the file analysis and the index calculation are completed.
According to the transaction data real-time query method, the query waiting queue is constructed to process a plurality of query requests, repeated queries can be processed, and the transaction data real-time query efficiency is improved.
The apparatus provided in the embodiments of the present application will be described below, and the apparatus described below and the method described above may be referred to correspondingly.
Fig. 4 is a schematic structural diagram of a transaction data real-time query device according to an embodiment of the present application, as shown in fig. 4, where the device includes:
an index configuration module 410, configured to generate calculation instructions of each transaction data index based on each field in the transaction detail;
a file parsing module 420, configured to receive a transaction detail file; analyzing the transaction detail file, extracting the values corresponding to the fields, and generating structured data corresponding to the transaction detail file;
the index management module 430 is configured to determine a real-time calculation result of each transaction data index based on the calculation instruction of each transaction data index and the structured data corresponding to the transaction detail file; and generating a query result corresponding to the query request based on the transaction data indexes to be queried in the query request and the real-time calculation results of all the transaction data indexes.
Specifically, the index configuration module may add or delete transaction data indexes in addition to setting field tags, data types, field lengths, etc. in the transaction detail file. The index configuration module can be realized through a Web front end.
The file analysis module is used for monitoring the generation state of the new transaction detail file, supporting file analysis and state transfer according to preset time and based on external requests, and completing the structured storage of data. The file parsing module can be realized through an open source framework SpringBatch.
The index management module reads the transaction detail data which is stored in a structured mode according to the index labels and the processing instructions in the index configuration module, completes processing, storage and cache updating of the index by combining the history detail data, and provides functions of adding, deleting, updating and reading the index data.
According to the transaction data real-time query device provided by the embodiment of the application, the received transaction detail file is analyzed, the value corresponding to each field is extracted to generate the structured data corresponding to the transaction detail file, the real-time calculation result of each transaction data index is determined according to the calculation instruction of each transaction data index and the structured data corresponding to the transaction detail file, the query result corresponding to the query request is generated according to the transaction data index to be queried in the query request and the real-time calculation result of each transaction data index, and the structured data is obtained by analyzing the transaction detail file and then calculating the structured data, so that each requester system can directly query and obtain each transaction data index without analyzing the transaction detail file independently, the cost of each requester system is reduced, the technical multiplexing degree of analysis results of the transaction detail file is improved, and the real-time calculation result of each transaction data index in the process is kept updated in real time along with the acquisition of the transaction detail file, so that the queried data of each requester system can keep real-time and consistency.
Fig. 5 is a schematic structural diagram of a transaction data real-time query device according to another embodiment of the present application, as shown in fig. 5, in some embodiments, the device further includes:
the task scheduling module 440 is connected to the index configuration module 410, the file parsing module 420, and the index management module 430, and is configured to control each module to perform data transmission based on a preset flow.
Specifically, the preset flow may be a data processing flow corresponding to the transaction data real-time query method. The task scheduling module can receive and process external requests and requests among internal modules, and the overall scheduling modules process, store data and access data according to preset flows and with independent task type completion indexes. The task scheduling module may be implemented by an open source framework Quartz.
A message queue module 450, coupled to the metrics configuration module 410, the file parsing module 420, and the metrics management module 430, is configured to implement asynchronous communications between the various models based on a message queue mechanism.
Specifically, the message queue module can realize asynchronous communication between each module and also can realize asynchronous communication between each module and an external system, and realize decoupling between systems and between modules.
In some embodiments, the index management module is connected with each requester system based on an Http interface and is used for receiving a query request sent by each requester system and returning a query result to each requester system.
Specifically, the index management module and each requester system adopt an Http interface, and follow the Http protocol and adopt REST (Resource Representational State Transfer) API (Application Programming Interface) design principles, so that the interface has expandability, flexibility and independence.
According to the transaction data real-time query device provided by the embodiment of the application, the mode of querying data among systems is changed from file interaction to an Http interface, common data indexes are designed to be default automatic processing, the front-end Web interface is provided for personalized configuration, and after transaction detail files are analyzed and processed, the data indexes are stored in the non-relational cache database, so that the query efficiency of the data indexes and the convenience of data configuration are improved.
Fig. 6 is a schematic structural diagram of an electronic device according to an embodiment of the present application, as shown in fig. 6, the electronic device may include: processor (Processor) 610, communication interface (Communications Interface) 620, memory (Memory) 630, and communication bus (Communications Bus) 640, wherein Processor 610, communication interface 620, and Memory 630 communicate with each other via communication bus 640. The processor 610 may invoke logic commands in the memory 630 to perform the following method:
generating a calculation instruction of each transaction data index based on each field in the transaction detail; receiving a transaction detail file; analyzing the transaction detail file, extracting the values corresponding to the fields, and generating structured data corresponding to the transaction detail file; determining a real-time calculation result of each transaction data index based on the calculation instruction of each transaction data index and the structured data corresponding to the transaction detail file; and generating a query result corresponding to the query request based on the transaction data indexes to be queried in the query request and the real-time calculation results of all the transaction data indexes.
In addition, the logic commands in the memory described above may be implemented in the form of software functional units and may be stored in a computer readable storage medium when sold or used as a stand alone product. Based on such understanding, the technical solution of the present application may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution, in the form of a software product stored in a storage medium, comprising several commands for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
The processor in the electronic device provided by the embodiment of the present application may call the logic instruction in the memory to implement the above method, and the specific implementation manner of the processor is consistent with the implementation manner of the foregoing method, and may achieve the same beneficial effects, which are not described herein again.
The present application also provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, is implemented to perform the methods provided by the above embodiments.
The specific embodiment is consistent with the foregoing method embodiment, and the same beneficial effects can be achieved, and will not be described herein.
Embodiments of the present application provide a computer program product comprising a computer program which, when executed by a processor, implements a method as described above.
The apparatus embodiments described above are merely illustrative, wherein the elements illustrated as separate elements may or may not be physically separate, and the elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art will understand and implement the present invention without undue burden.
From the above description of the embodiments, it will be apparent to those skilled in the art that the embodiments may be implemented by means of software plus necessary general hardware platforms, or of course may be implemented by means of hardware. Based on this understanding, the foregoing technical solution may be embodied essentially or in a part contributing to the prior art in the form of a software product, which may be stored in a computer readable storage medium, such as ROM/RAM, a magnetic disk, an optical disk, etc., including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method described in the respective embodiments or some parts of the embodiments.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present application, and are not limiting thereof; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the corresponding technical solutions.
Claims (10)
1. A method for real-time query of transaction data, comprising:
generating a calculation instruction of each transaction data index based on each field in the transaction detail;
receiving a transaction detail file;
analyzing the transaction detail file, extracting the value corresponding to each field, and generating the structured data corresponding to the transaction detail file;
determining real-time calculation results of all transaction data indexes based on calculation instructions of all transaction data indexes and structured data corresponding to the transaction detail files;
and generating a query result corresponding to the query request based on the transaction data indexes to be queried in the query request and real-time calculation results of all the transaction data indexes.
2. The method of claim 1, wherein receiving the transaction detail file comprises:
determining a file storage catalog of the transaction detail file based on the issuing time of the transaction detail file;
and after the issuing of the transaction detail file is finished, generating an issuing finishing mark file corresponding to the transaction detail file in the file storage catalog.
3. The method of claim 2, wherein the parsing the transaction detail file, extracting values corresponding to respective fields, and generating structured data corresponding to the transaction detail file comprises:
polling the file storage directory, and determining that the file storage directory contains the transaction detail file and a issuing end mark file corresponding to the transaction detail file;
determining that the transaction detail file is in a delivery completion state based on the delivery end mark file;
extracting the corresponding values of each field in the transaction detail file;
and generating structured data corresponding to the transaction detail file based on each field and the value corresponding to each field.
4. The method for real-time query of transaction data according to claim 1, wherein the generating the query result corresponding to the query request based on the transaction data indexes to be queried in the query request and the real-time calculation results of the transaction data indexes comprises:
receiving a current query request;
matching the current query request with each query request in a query waiting queue;
under the condition that the current query request is not matched with each query request in the query waiting queue, acquiring the analysis state of the transaction detail file and/or the execution state of a calculation instruction of each transaction data index;
generating a query result corresponding to the current query request based on real-time calculation results of all transaction data indexes under the conditions that the analysis state of the transaction detail file is analysis completion and the execution state of the calculation instruction is execution completion;
and adding the current query request to the query waiting queue when the analysis state of the transaction detail file is not analyzed or the execution state of any calculation instruction is not executed.
5. A transaction data real-time query device, comprising:
the index configuration module is used for generating calculation instructions of all transaction data indexes based on all fields in the transaction detail;
the file analysis module is used for receiving the transaction detail file; analyzing the transaction detail file, extracting the value corresponding to each field, and generating the structured data corresponding to the transaction detail file;
the index management module is used for determining real-time calculation results of all transaction data indexes based on calculation instructions of all transaction data indexes and structured data corresponding to the transaction detail files; and generating a query result corresponding to the query request based on the transaction data indexes to be queried in the query request and real-time calculation results of all the transaction data indexes.
6. The transaction data real-time query device according to claim 5, wherein the device further comprises:
and the task scheduling module is connected with the index configuration module, the file analysis module and the index management module and used for controlling each module to perform data transmission based on a preset flow.
7. The transaction data real-time query device according to claim 5, wherein the device further comprises:
and the message queue module is connected with the index configuration module, the file analysis module and the index management module and is used for realizing asynchronous communication among the models based on a message queue mechanism.
8. The apparatus according to claim 5, wherein the index management module is connected to each of the requester systems based on an Http interface, and is configured to receive a query request sent by each of the requester systems, and return a query result to each of the requester systems.
9. An electronic device comprising a memory and a processor, wherein the memory has stored therein a computer program, the processor being arranged to execute the transaction data real-time querying method according to any of claims 1 to 4 by means of the computer program.
10. A computer-readable storage medium, characterized in that the computer-readable storage medium comprises a stored program, wherein the program when run performs the transaction data real-time query method of any one of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310126989.XA CN116303593A (en) | 2023-02-16 | 2023-02-16 | Transaction data real-time query method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310126989.XA CN116303593A (en) | 2023-02-16 | 2023-02-16 | Transaction data real-time query method and device, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116303593A true CN116303593A (en) | 2023-06-23 |
Family
ID=86837001
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310126989.XA Pending CN116303593A (en) | 2023-02-16 | 2023-02-16 | Transaction data real-time query method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116303593A (en) |
-
2023
- 2023-02-16 CN CN202310126989.XA patent/CN116303593A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11886503B2 (en) | Method and apparatus for implementing a search index generator | |
| JP5624674B2 (en) | How to improve queries for searching databases | |
| CN111382279A (en) | Order examination method and device | |
| CN105095424A (en) | Method for realizing high-concurrency retrieval of medicine in hospital HIS system | |
| CN109086433A (en) | A kind of file management method and server based on big data analysis | |
| CN110659971B (en) | Transaction data processing method and device | |
| US11913811B2 (en) | Enhanced meter management solution | |
| CN110942392A (en) | Service data processing method, device, equipment and medium | |
| CN111191153A (en) | Information technology consultation service display device | |
| CN108959514B (en) | Data processing method and device | |
| JP4938813B2 (en) | Transaction data processing method, system and program | |
| CN112163948A (en) | Method, system, equipment and storage medium for separately-moistening calculation | |
| CN110781235A (en) | Big data based purchase data processing method and device, terminal and storage medium | |
| CN110704486A (en) | Data processing method, device, system, storage medium and server | |
| CN116167860A (en) | Account checking processing method and device, electronic equipment and storage medium | |
| CN116303593A (en) | Transaction data real-time query method and device, electronic equipment and storage medium | |
| CN110689951A (en) | Customer information management system of medical institution | |
| EP4348441A1 (en) | Systems and methods for ensuring quality of search system data | |
| CN114969071A (en) | Data updating method and device | |
| JP5850546B1 (en) | Government bond principal and interest distribution system and method | |
| JP4927150B2 (en) | Trade settlement related data management system and method | |
| KR102148152B1 (en) | Processing method and apparatus for financial instrument information | |
| CN114549211A (en) | Data analysis method, device and equipment based on random forest and storage medium | |
| CN116166641A (en) | Mapping method, device, equipment and medium for data model and data standard | |
| CN115167855A (en) | Front-end page generation method, device and equipment applied to matching transaction system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |