CN111158934A - Method and device for realizing different-level delay splicing based on buffer queue received data - Google Patents
Method and device for realizing different-level delay splicing based on buffer queue received data Download PDFInfo
- Publication number
- CN111158934A CN111158934A CN201911406478.3A CN201911406478A CN111158934A CN 111158934 A CN111158934 A CN 111158934A CN 201911406478 A CN201911406478 A CN 201911406478A CN 111158934 A CN111158934 A CN 111158934A
- Authority
- CN
- China
- Prior art keywords
- queue
- transaction information
- transaction
- information
- delay
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/54—Interprogram communication
- G06F9/546—Message passing systems or structures, e.g. queues
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/54—Interprogram communication
- G06F9/544—Buffers; Shared memory; Pipes
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention provides a method and a device for realizing different-level delay splicing based on buffer queue received data, wherein the method comprises the following steps: monitoring the MQ message queue to obtain transaction information; forwarding the transaction information to a buffer queue; the buffer queue is provided with a buffer duration; after the buffering duration of the buffering queue is expired, the transaction information is forwarded to the actual consumption queue; taking out the transaction information in the actual consumption queue, and performing matching judgment on the transaction information and a peripheral information table; when the transaction information is matched with the peripheral information table, splicing and processing the transaction information to determine complete transaction information; and when the transaction information is not matched with the peripheral information table, forwarding the transaction information to the multistage delay queue, and after the delay time of the multistage delay queue is expired, forwarding the transaction information to the actual consumption queue. The invention can relieve backlog conditions in the MQ message queue and the database and realize delayed splicing of the transaction information.
Description
Technical Field
The invention belongs to the technical field of data information splicing processing, and particularly relates to a method and a device for realizing different-level delay splicing based on buffer queue received data.
Background
In the process of processing the message received by the MQ in real time, when transaction information of financial account change is received, in order to meet the requirement of real-time processing, the transaction information needs to be associated and completed according to information provided in another peripheral information table in real time, the information in the peripheral information table needs to be waited for to be received as much as possible through a mechanism of time lag of entering the table and repeated query, and if the information cannot be completed, the information can only be considered to be not spliced and forced to be put in storage for providing external query.
The method comprises the steps of screening data meeting conditions through a table entering time equal to the current time minus a configured time waiting interval of a database when the data are obtained, repeatedly scanning a peripheral information supplement table according to configuration times by the data needing to be spliced, and adding one to the scanning times if no relevant data exists in the supplement table until the scanning times are full.
When the method in the prior art is adopted, the database time is used for controlling, so that part of data always belongs to an unprocessed state, and the table query load is increased; in the case of repeated scanning, the table needs to be updated repeatedly, which affects the overall processing performance and increases the database load, thereby affecting the speed of data receiving processing.
Disclosure of Invention
The embodiment of the invention provides a method for realizing different-level delay splicing based on buffer queue received data, which realizes delay splicing processing of transaction information and a peripheral information table by using a buffer queue and a multi-level delay queue, and comprises the following steps:
monitoring the MQ message queue to obtain transaction information;
forwarding the transaction information to a buffer queue; the buffer queue is provided with a buffer duration;
after the buffering duration of the buffering queue is expired, the transaction information is forwarded to the actual consumption queue;
taking out the transaction information in the actual consumption queue, and performing matching judgment on the transaction information and a peripheral information table;
when the transaction information is matched with the peripheral information table, splicing and processing the transaction information to determine complete transaction information;
and when the transaction information is not matched with the peripheral information table, forwarding the transaction information to the multistage delay queue, and after the delay time of the multistage delay queue is expired, forwarding the transaction information to the actual consumption queue.
The embodiment of the invention also provides a device for realizing different levels of delay splicing based on the data received by the buffer queue, which comprises:
the transaction information acquisition module is used for monitoring the MQ message queue and acquiring transaction information;
the transaction information forwarding and buffering queue module is used for forwarding the transaction information to a buffering queue; the buffer queue is provided with a buffer duration;
the transaction information forwarding actual consumption queue module is used for forwarding the transaction information to the actual consumption queue after the buffering duration of the buffering queue is expired;
the matching judgment module is used for taking out the transaction information in the actual consumption queue and carrying out matching judgment on the transaction information and the peripheral information table;
the complete transaction information determining module is used for splicing and processing the transaction information to determine complete transaction information when the transaction information is matched with the peripheral information table;
and the delay forwarding module is used for forwarding the transaction information to the multistage delay queue when the transaction information is not matched with the peripheral information table, and forwarding the transaction information to the actual consumption queue after the delay time of the multistage delay queue is expired.
The embodiment of the invention also provides computer equipment which comprises a memory, a processor and a computer program which is stored on the memory and can run on the processor, wherein the processor realizes the method for realizing different levels of delay splicing based on the data received by the buffer queue when executing the computer program.
An embodiment of the present invention further provides a computer-readable storage medium, where the computer-readable storage medium stores a computer program for implementing the method for implementing different levels of delay splicing based on buffer queue received data.
The method and the device for realizing different-level delay splicing based on the data received by the buffer queue provided by the embodiment of the invention can relieve the backlog condition in the MQ message queue and the database by monitoring the MQ message queue to acquire the transaction information and forwarding the transaction information to the buffer queue, then after the buffer duration of the buffer queue is expired, the transaction information is forwarded to the actual consumption queue, then the transaction information in the actual consumption queue is taken out and is matched and judged with the peripheral information table, in the process of data processing, when the matching is successful, the transaction information is directly spliced with the peripheral information table to determine that the complete transaction is new, and when the matching is unsuccessful, the transaction information is distributed to the multistage delay queues with different delay levels, and after the delay time of the multistage delay queues is expired, the transaction information is forwarded to the actual consumption queue to realize the delayed splicing of the transaction information. The embodiment of the invention does not depend on the operation of inserting the database, removes the operation of repeatedly updating by polling for many times, reduces the whole load of the database and can improve the speed of data receiving and processing.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts. In the drawings:
fig. 1 is a schematic diagram illustrating a method for implementing different levels of delay splicing based on data received by a buffer queue according to an embodiment of the present invention.
Fig. 2 is a flowchart of a method for implementing different levels of delay splicing based on data received by a buffer queue according to an embodiment of the present invention.
Fig. 3 is a schematic diagram of an apparatus for implementing different levels of delay splicing based on data received by a buffer queue according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the embodiments of the present invention are further described in detail below with reference to the accompanying drawings. The exemplary embodiments and descriptions of the present invention are provided to explain the present invention, but not to limit the present invention.
As shown in fig. 1, a schematic diagram of a method for implementing different-level delay splicing based on data received by a buffer queue according to an embodiment of the present invention is shown, where the embodiment of the present invention provides a method for implementing different-level delay splicing based on data received by a buffer queue, and a delayed splicing process between transaction information and a peripheral information table is implemented by using a buffer queue and a multi-level delay queue, and the method includes:
step 101: monitoring the MQ message queue to obtain transaction information;
step 102: forwarding the transaction information to a buffer queue; the buffer queue is provided with a buffer duration;
step 103: after the buffering duration of the buffering queue is expired, the transaction information is forwarded to the actual consumption queue;
step 104: taking out the transaction information in the actual consumption queue, and performing matching judgment on the transaction information and a peripheral information table;
step 105: when the transaction information is matched with the peripheral information table, splicing and processing the transaction information to determine complete transaction information;
step 106: and when the transaction information is not matched with the peripheral information table, forwarding the transaction information to the multistage delay queue, and after the delay time of the multistage delay queue is expired, forwarding the transaction information to the actual consumption queue.
The method for realizing different-level delay splicing based on the data received by the buffer queue, provided by the embodiment of the invention, can relieve the backlog condition in the MQ message queue and the database by monitoring the MQ message queue, acquiring the transaction information and forwarding the transaction information to the buffer queue, then after the buffer duration of the buffer queue is expired, the transaction information is forwarded to the actual consumption queue, then the transaction information in the actual consumption queue is taken out and is matched and judged with the peripheral information table, in the process of data processing, when the matching is successful, the transaction information is directly spliced with the peripheral information table to determine that the complete transaction is new, and when the matching is unsuccessful, the transaction information is distributed to the multistage delay queues with different delay levels, and after the delay time of the multistage delay queues is expired, the transaction information is forwarded to the actual consumption queue to realize the delayed splicing of the transaction information. The embodiment of the invention does not depend on the operation of inserting the database, removes the operation of repeatedly updating by polling for many times, reduces the whole load of the database and can improve the speed of data receiving and processing.
In a specific implementation of the method for implementing different levels of delay splicing based on data received by a buffer queue according to an embodiment of the present invention, as shown in fig. 1, a schematic diagram of a method for implementing different levels of delay splicing based on data received by a buffer queue according to an embodiment of the present invention, and fig. 2, a flowchart of a method for implementing different levels of delay splicing based on data received by a buffer queue according to an embodiment of the present invention, the method may include:
monitoring the MQ message queue to obtain transaction information; forwarding the transaction information to a buffer queue; the buffer queue is provided with a buffer duration; after the buffering duration of the buffering queue is expired, the transaction information is forwarded to the actual consumption queue; taking out the transaction information in the actual consumption queue, and performing matching judgment on the transaction information and a peripheral information table; when the transaction information is matched with the peripheral information table, splicing and processing the transaction information to determine complete transaction information; and when the transaction information is not matched with the peripheral information table, forwarding the transaction information to the multistage delay queue, and after the delay time of the multistage delay queue is expired, forwarding the transaction information to the actual consumption queue.
In a specific embodiment of the present invention, when implementing a method for implementing different levels of delayed concatenation based on data received by a buffer queue, in an embodiment, the monitoring an MQ message queue to obtain transaction information may include:
and starting the JAVA resident daemon, continuously monitoring the MQ message queue, and taking the transaction information out of the MQ message queue when the transaction information reaches the MQ message queue.
In the embodiment, the JAVA resident daemon is started as a long suspended process, the MQ message queue is monitored continuously, and the transaction information is taken out from the MQ message queue when reaching the MQ message queue, so that the backlog risk of the MQ message queue is reduced. In an embodiment, the MQ message queue may be an IBM MQ message queue.
In an embodiment of the invention, when implementing a method for implementing different levels of delayed concatenation based on data received by a buffer queue, the forwarding transaction information to the buffer queue may include: the transaction information is forwarded to the buffer queue by calling the JAVA API.
In an embodiment, the buffer queue is provided with a buffer duration, and in a specific implementation, the buffer duration may be set according to an actual requirement, for example, in an example, the buffer duration of the buffer queue is set to 3s, that is, the survival time of the transaction information in the buffer queue is 3s, and after 3s is finished, the transaction information is forwarded from the buffer queue to the actual consumption queue; in an embodiment, the aforementioned buffer queue may employ a buffer queue of a rabbitmq message cluster of an internal cluster.
In a specific embodiment of the method for implementing different levels of delayed concatenation based on data received by a buffer queue according to an embodiment of the present invention, in an embodiment, after a buffer duration of the buffer queue expires, forwarding the transaction information to an actual consumption queue may include:
after the buffer duration of the buffer queue expires, the transaction information is removed from the buffer queue by a Dead-letter message Exchange middleware (Dead-letter Exchange) and forwarded to the actual consumption queue.
In an embodiment, for example, the set buffer duration is 3s, and after the transaction information reaches 3s in the buffer queue, the message exchange middleware moves the transaction information out of the buffer queue and forwards the transaction information to the actual consumption queue.
In a specific embodiment of the present invention, when a method for implementing different levels of delayed concatenation based on data received by a buffer queue is implemented, in one embodiment, the peripheral information table includes attached information;
the aforementioned taking out the transaction information in the actual consumption queue, and performing matching judgment with the accessory information includes:
and taking out the transaction information in the actual consumption queue by using the long-hanging multithread service, and searching whether auxiliary information which belongs to the same transaction with the transaction information exists in a peripheral information table.
In the embodiment, the peripheral information table may be searched in a scanning manner, whether there is additional information belonging to the same transaction as the transaction information is found in the peripheral information table, if there is additional information belonging to the same transaction as the transaction information in the peripheral information table, splicing processing is performed, and if there is no additional information belonging to the same transaction as the transaction information in the peripheral information table, the transaction information may be forwarded to the multistage delay queue, and delayed splicing is performed after the peripheral information table is updated.
In an embodiment, the auxiliary information may include: and the contents such as the information of the transaction opponents, the abstract postscript of the purposes and the like are used for complementing the transaction information. The peripheral information table is reported in real time by other systems, so that a certain delay exists.
In one embodiment, after the transaction information in the actual consumption queue is taken out, the transaction information is distributed to each independent thread to be processed independently.
In a specific embodiment of the method for implementing different levels of delayed concatenation based on data received by a buffer queue according to an embodiment of the present invention, when the transaction information matches with the peripheral information table, the processing of concatenation on the transaction information to determine complete transaction information may include:
and when the transaction information and the auxiliary information in the peripheral information table belong to the same transaction, splicing and processing the auxiliary information and the transaction information according to the service main key to determine complete transaction information.
In one embodiment, after the complete transaction information is determined, the method further includes changing the state of the complete transaction information to be completed, and storing the complete transaction information in the final database for storage.
In a specific embodiment of the method for implementing different levels of delay splicing based on data received by a buffer queue according to an embodiment of the present invention, in an embodiment, when the transaction information is not matched with the peripheral information table, the transaction information is forwarded to the multi-level delay queue, and after the delay duration of the multi-level delay queue expires, the forwarding the transaction information to the actual consumption queue may include:
when the transaction information and the auxiliary information in the peripheral information table do not belong to the same transaction, forwarding the transaction information to a multi-stage delay queue according to the transaction timeliness requirement of the transaction information;
and after the delay time of the multi-stage delay queue is expired, the transaction information is transmitted back to the actual consumption queue by the deadlock message exchange middleware.
In the embodiment, because the auxiliary information in the peripheral information table is reported by other systems, and a certain delay exists, the peripheral information table is continuously updated, in the updating process, if the transaction information does not scan the auxiliary information belonging to the same transaction with the transaction information in the peripheral information table, the transaction information needs to be forwarded to a multi-stage delay queue, set delay is performed in the multi-stage delay queue to wait for updating of the peripheral information table, and after the delay time of the multi-stage delay queue expires, the transaction information is transmitted back to an actual consumption queue, and scanning and splicing processing are performed again; by continuously circulating the process, the splicing processing of all the auxiliary information and the transaction information can be realized.
In a specific embodiment of the present invention, when implementing a method for implementing different levels of delay splicing based on data received by a buffer queue, in an embodiment, the multi-level delay queue may include: a first stage delay queue, a second stage delay queue and a third stage delay queue;
the delay time of the first-stage delay queue is shorter than that of the second-stage delay queue, and the delay time of the second-stage delay queue is shorter than that of the third-stage delay queue;
the aforementioned transaction timeliness requirements may include: high trading processing timeliness, medium trading processing timeliness and low trading processing timeliness;
the aforementioned forwarding the transaction information to the multi-stage delay queue according to the transaction timeliness requirement of the transaction information may include:
when the transaction timeliness of the transaction information requires high transaction processing timeliness, the transaction information is forwarded to a first-stage delay queue;
when the transaction timeliness requirement of the transaction information is the processing timeliness of the medium transaction, the transaction information is forwarded to the second-stage delay queue;
and when the transaction timeliness of the transaction information requires low transaction processing timeliness, forwarding the transaction information to the third-stage delay queue.
In an example of the embodiment of the present invention, the delay duration of the first-stage delay queue may be set to 1s, the delay duration of the second-stage delay queue may be set to 2s, and the delay duration of the third-stage delay queue may be set to 3 s; when the transaction timeliness of the transaction information requires high transaction processing timeliness, the transaction information is forwarded to a first-stage delay queue, and the transaction information is delayed for 1 second; when the transaction timeliness requirement of the transaction information is medium transaction processing timeliness, the transaction information is forwarded to a second-stage delay queue, and the transaction information is delayed for 2 seconds; when the transaction timeliness requirement of the transaction information is low transaction processing timeliness, the transaction information is forwarded to a third-stage delay queue, and the transaction information is delayed for 3 seconds;
if it is desired to perform delay processing or repeatedly query supplementary information without using a database, a buffer queue must be used first, and if delay time is to be controlled hierarchically, the degree of delay can be controlled hierarchically by means of the expiration times of different buffer queues. The buffer queue of the Rabbit MQ is used for relieving backlog conditions in an IBM MQ message queue and a database, then in the processing process, data which cannot be spliced in real time are distributed to delay queues with different delay times according to different delay times, and the data meeting the delay time are forwarded to an actual work consumption queue by DLX (Dead-letter Exchange). The invention uses JAVA hanging service program, firstly solves the risk of reducing the backlog of IBM MQ message queue, does not excessively depend on the operation of inserting the database, secondly removes the operation of repeatedly polling and repeatedly updating, and further reduces the whole load of the database.
The embodiment of the invention also provides computer equipment which comprises a memory, a processor and a computer program which is stored on the memory and can run on the processor, wherein the processor realizes the method for realizing different levels of delay splicing based on the data received by the buffer queue when executing the computer program.
An embodiment of the present invention further provides a computer-readable storage medium, where the computer-readable storage medium stores a computer program for implementing the method for implementing different levels of delay splicing based on buffer queue received data.
The embodiment of the invention also provides a device for realizing different levels of delay splicing based on the data received by the buffer queue, which is described in the following embodiment. Because the principle of the device for solving the problems is similar to a method for realizing different levels of delay splicing based on the data received by the buffer queue, the implementation of the device can refer to the implementation of the method for realizing different levels of delay splicing based on the data received by the buffer queue, and repeated parts are not described again.
As shown in fig. 3, an apparatus for implementing different levels of delay splicing based on data received by a buffer queue according to an embodiment of the present invention is schematically illustrated, and an apparatus for implementing different levels of delay splicing based on data received by a buffer queue according to an embodiment of the present invention may include:
the transaction information acquisition module 301 is configured to monitor the MQ message queue and acquire transaction information;
a transaction information forwarding buffer queue module 302, configured to forward the transaction information to a buffer queue; the buffer queue is provided with a buffer duration;
the transaction information forwarding actual consumption queue module 303 is configured to forward the transaction information to the actual consumption queue after the buffering duration of the buffering queue expires;
the matching judgment module 304 is used for taking out the transaction information in the actual consumption queue and performing matching judgment with the peripheral information table;
a complete transaction information determining module 305, configured to splice and process the transaction information to determine complete transaction information when the transaction information matches the peripheral information table;
the delay forwarding module 306 is configured to forward the transaction information to the multistage delay queue when the transaction information is not matched with the peripheral information table, and forward the transaction information to the actual consumption queue after the delay duration of the multistage delay queue expires.
In an embodiment of the invention, when implementing a device for implementing different levels of delayed concatenation based on data received by a buffer queue, the transaction information obtaining module is specifically configured to:
and starting the JAVA resident daemon, continuously monitoring the MQ message queue, and taking the transaction information out of the MQ message queue when the transaction information reaches the MQ message queue.
In an embodiment of the invention, when implementing a device for implementing different levels of delayed splicing based on data received by a buffer queue, the transaction information forwarding buffer queue module is specifically configured to: the transaction information is forwarded to the buffer queue by calling the JAVA API.
In a specific embodiment of the present invention, when implementing a device for implementing different levels of delayed concatenation based on data received by a buffer queue, in one embodiment, the transaction information forwarding actual consumption queue module is specifically configured to:
and after the buffering duration of the buffering queue is expired, the transaction information is moved out of the buffering queue by the deadlock message exchange middleware and forwarded to the actual consumption queue.
In a specific embodiment of the present invention, when a device for implementing different levels of delayed concatenation based on data received by a buffer queue is implemented, in one embodiment, the peripheral information table includes auxiliary information;
the matching judgment module is specifically configured to:
and taking out the transaction information in the actual consumption queue by using the long-hanging multithread service, and searching whether auxiliary information which belongs to the same transaction with the transaction information exists in a peripheral information table.
In an embodiment of the invention, when implementing a device for implementing different levels of delayed concatenation based on data received by a buffer queue, the complete transaction information determining module is specifically configured to:
and when the transaction information and the auxiliary information in the peripheral information table belong to the same transaction, splicing and processing the auxiliary information and the transaction information according to the service main key to determine complete transaction information.
In a specific embodiment of the present invention, when implementing a device for implementing different levels of delay concatenation based on data received by a buffer queue, in an embodiment, the delay forwarding module is specifically configured to:
when the transaction information and the auxiliary information in the peripheral information table do not belong to the same transaction, forwarding the transaction information to a multi-stage delay queue according to the transaction timeliness requirement of the transaction information;
and after the delay time of the multi-stage delay queue is expired, the transaction information is transmitted back to the actual consumption queue by the deadlock message exchange middleware.
In a specific embodiment of the present invention, when implementing a device for implementing different levels of delay concatenation based on data received by a buffer queue, in an embodiment, a multi-level delay queue includes: a first stage delay queue, a second stage delay queue and a third stage delay queue;
the delay time of the first-stage delay queue is shorter than that of the second-stage delay queue, and the delay time of the second-stage delay queue is shorter than that of the third-stage delay queue;
transaction timeliness requirements, including: high trading processing timeliness, medium trading processing timeliness and low trading processing timeliness;
the aforementioned delay forwarding module is further configured to:
when the transaction timeliness of the transaction information requires high transaction processing timeliness, the transaction information is forwarded to a first-stage delay queue;
when the transaction timeliness requirement of the transaction information is the processing timeliness of the medium transaction, the transaction information is forwarded to the second-stage delay queue;
and when the transaction timeliness of the transaction information requires low transaction processing timeliness, forwarding the transaction information to the third-stage delay queue.
To sum up, the method and apparatus for implementing different levels of delayed concatenation based on data received by a buffer queue according to embodiments of the present invention first obtain transaction information and forward the transaction information to the buffer queue by monitoring the MQ message queue, can alleviate backlog in the MQ message queue and the database, then after the buffer duration of the buffer queue is expired, the transaction information is forwarded to the actual consumption queue, then the transaction information in the actual consumption queue is taken out and is matched and judged with the peripheral information table, in the process of data processing, when the matching is successful, the transaction information is directly spliced with the peripheral information table to determine that the complete transaction is new, and when the matching is unsuccessful, the transaction information is distributed to the multistage delay queues with different delay levels, and after the delay time of the multistage delay queues is expired, the transaction information is forwarded to the actual consumption queue to realize the delayed splicing of the transaction information. The embodiment of the invention does not depend on the operation of inserting the database, removes the operation of repeatedly updating by polling for many times, reduces the whole load of the database and can improve the speed of data receiving and processing.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The above-mentioned embodiments are intended to illustrate the objects, technical solutions and advantages of the present invention in further detail, and it should be understood that the above-mentioned embodiments are only exemplary embodiments of the present invention, and are not intended to limit the scope of the present invention, and any modifications, equivalent substitutions, improvements and the like made within the spirit and principle of the present invention should be included in the scope of the present invention.
Claims (18)
1. A method for realizing different-level delay splicing based on data received by a buffer queue is characterized by comprising the following steps:
monitoring the MQ message queue to obtain transaction information;
forwarding the transaction information to a buffer queue; the buffer queue is provided with a buffer duration;
after the buffering duration of the buffering queue is expired, the transaction information is forwarded to the actual consumption queue;
taking out the transaction information in the actual consumption queue, and performing matching judgment on the transaction information and a peripheral information table;
when the transaction information is matched with the peripheral information table, splicing and processing the transaction information to determine complete transaction information;
and when the transaction information is not matched with the peripheral information table, forwarding the transaction information to the multistage delay queue, and after the delay time of the multistage delay queue is expired, forwarding the transaction information to the actual consumption queue.
2. The method as claimed in claim 1, wherein listening to the MQ message queue to obtain the transaction information comprises:
and starting the JAVA resident daemon, continuously monitoring the MQ message queue, and taking the transaction information out of the MQ message queue when the transaction information reaches the MQ message queue.
3. The method of claim 1, wherein forwarding transaction information to a buffer queue comprises: the transaction information is forwarded to the buffer queue by calling the JAVA API.
4. The method of claim 1, wherein forwarding the transaction information to the actual consumption queue after the buffer duration of the buffer queue expires comprises:
and after the buffering duration of the buffering queue is expired, the transaction information is moved out of the buffering queue by the deadlock message exchange middleware and forwarded to the actual consumption queue.
5. The method of claim 1, wherein the peripheral information table includes auxiliary information;
taking out the transaction information in the actual consumption queue, and performing matching judgment with the auxiliary information, wherein the method comprises the following steps:
and taking out the transaction information in the actual consumption queue by using the long-hanging multithread service, and searching whether auxiliary information which belongs to the same transaction with the transaction information exists in a peripheral information table.
6. The method of claim 5, wherein when the transaction information matches the peripheral information table, stitching the transaction information to determine complete transaction information comprises:
and when the transaction information and the auxiliary information in the peripheral information table belong to the same transaction, splicing and processing the auxiliary information and the transaction information according to the service main key to determine complete transaction information.
7. The method of claim 5, wherein forwarding the transaction information to the multi-stage delay queue when the transaction information does not match the peripheral information table, and forwarding the transaction information to the actual consumption queue after the delay duration of the multi-stage delay queue expires comprises:
when the transaction information and the auxiliary information in the peripheral information table do not belong to the same transaction, forwarding the transaction information to a multi-stage delay queue according to the transaction timeliness requirement of the transaction information;
and after the delay time of the multi-stage delay queue is expired, the transaction information is transmitted back to the actual consumption queue by the deadlock message exchange middleware.
8. The method of claim 7,
a multi-stage delay queue comprising: a first stage delay queue, a second stage delay queue and a third stage delay queue;
the delay time of the first-stage delay queue is shorter than that of the second-stage delay queue, and the delay time of the second-stage delay queue is shorter than that of the third-stage delay queue;
transaction timeliness requirements, including: high trading processing timeliness, medium trading processing timeliness and low trading processing timeliness;
according to the transaction timeliness requirement of the transaction information, the transaction information is forwarded to a multi-stage delay queue, and the method comprises the following steps:
when the transaction timeliness of the transaction information requires high transaction processing timeliness, the transaction information is forwarded to a first-stage delay queue;
when the transaction timeliness requirement of the transaction information is the processing timeliness of the medium transaction, the transaction information is forwarded to the second-stage delay queue;
and when the transaction timeliness of the transaction information requires low transaction processing timeliness, forwarding the transaction information to the third-stage delay queue.
9. A device for realizing different levels of delay splicing based on data received by a buffer queue is characterized by comprising:
the transaction information acquisition module is used for monitoring the MQ message queue and acquiring transaction information;
the transaction information forwarding and buffering queue module is used for forwarding the transaction information to a buffering queue; the buffer queue is provided with a buffer duration;
the transaction information forwarding actual consumption queue module is used for forwarding the transaction information to the actual consumption queue after the buffering duration of the buffering queue is expired;
the matching judgment module is used for taking out the transaction information in the actual consumption queue and carrying out matching judgment on the transaction information and the peripheral information table;
the complete transaction information determining module is used for splicing and processing the transaction information to determine complete transaction information when the transaction information is matched with the peripheral information table;
and the delay forwarding module is used for forwarding the transaction information to the multistage delay queue when the transaction information is not matched with the peripheral information table, and forwarding the transaction information to the actual consumption queue after the delay time of the multistage delay queue is expired.
10. The apparatus of claim 9, wherein the transaction information acquisition module is specifically configured to:
and starting the JAVA resident daemon, continuously monitoring the MQ message queue, and taking the transaction information out of the MQ message queue when the transaction information reaches the MQ message queue.
11. The apparatus of claim 9, wherein the transaction message forwarding buffer queue module is specifically configured to: the transaction information is forwarded to the buffer queue by calling the JAVA API.
12. The apparatus of claim 9, wherein the transaction information forwarding actual consumption queue module is specifically configured to:
and after the buffering duration of the buffering queue is expired, the transaction information is moved out of the buffering queue by the deadlock message exchange middleware and forwarded to the actual consumption queue.
13. The apparatus of claim 9, wherein the peripheral information table includes auxiliary information;
the matching judgment module is specifically used for:
and taking out the transaction information in the actual consumption queue by using the long-hanging multithread service, and searching whether auxiliary information which belongs to the same transaction with the transaction information exists in a peripheral information table.
14. The apparatus of claim 13, wherein the complete transaction information determination module is specifically configured to:
and when the transaction information and the auxiliary information in the peripheral information table belong to the same transaction, splicing and processing the auxiliary information and the transaction information according to the service main key to determine complete transaction information.
15. The apparatus of claim 13, wherein the delay forwarding module is specifically configured to:
when the transaction information and the auxiliary information in the peripheral information table do not belong to the same transaction, forwarding the transaction information to a multi-stage delay queue according to the transaction timeliness requirement of the transaction information;
and after the delay time of the multi-stage delay queue is expired, the transaction information is transmitted back to the actual consumption queue by the deadlock message exchange middleware.
16. The apparatus of claim 15,
a multi-stage delay queue comprising: a first stage delay queue, a second stage delay queue and a third stage delay queue;
the delay time of the first-stage delay queue is shorter than that of the second-stage delay queue, and the delay time of the second-stage delay queue is shorter than that of the third-stage delay queue;
transaction timeliness requirements, including: high trading processing timeliness, medium trading processing timeliness and low trading processing timeliness;
a delay forwarding module further configured to:
when the transaction timeliness of the transaction information requires high transaction processing timeliness, the transaction information is forwarded to a first-stage delay queue;
when the transaction timeliness requirement of the transaction information is the processing timeliness of the medium transaction, the transaction information is forwarded to the second-stage delay queue;
and when the transaction timeliness of the transaction information requires low transaction processing timeliness, forwarding the transaction information to the third-stage delay queue.
17. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the method of any one of claims 1 to 8 for implementing different levels of delay splicing based on buffered queue received data when executing the computer program.
18. A computer-readable storage medium storing a computer program for executing a method for implementing different levels of delay splicing based on data received from a buffer queue according to any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911406478.3A CN111158934B (en) | 2019-12-31 | 2019-12-31 | Method and device for realizing different-level delay splicing based on data received by buffer queue |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911406478.3A CN111158934B (en) | 2019-12-31 | 2019-12-31 | Method and device for realizing different-level delay splicing based on data received by buffer queue |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111158934A true CN111158934A (en) | 2020-05-15 |
| CN111158934B CN111158934B (en) | 2023-05-02 |
Family
ID=70559717
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911406478.3A Active CN111158934B (en) | 2019-12-31 | 2019-12-31 | Method and device for realizing different-level delay splicing based on data received by buffer queue |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111158934B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115328680A (en) * | 2022-09-28 | 2022-11-11 | 天津卓朗昆仑云软件技术有限公司 | Message queue consumption abnormity assisting method and device and electronic equipment |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20160285812A1 (en) * | 2015-03-24 | 2016-09-29 | Xinyu Xingbang Information Industry Co., Ltd. | Method of issuing messages of a message queue and a message issuing device |
| CN106210021A (en) * | 2016-07-05 | 2016-12-07 | 中国银行股份有限公司 | The method for real-time monitoring of financial application system online business and supervising device |
| CN108076098A (en) * | 2016-11-16 | 2018-05-25 | 北京京东尚科信息技术有限公司 | A kind of method for processing business and system |
| WO2018232490A1 (en) * | 2017-06-22 | 2018-12-27 | Zeu Crypto Networks Inc. | Multilevel queue based transaction traffic shaping for blockchains |
| CN109271417A (en) * | 2018-11-23 | 2019-01-25 | 福建榕基软件股份有限公司 | A kind of lightweight messages queue implementing method and storage equipment based on database |
-
2019
- 2019-12-31 CN CN201911406478.3A patent/CN111158934B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20160285812A1 (en) * | 2015-03-24 | 2016-09-29 | Xinyu Xingbang Information Industry Co., Ltd. | Method of issuing messages of a message queue and a message issuing device |
| CN106210021A (en) * | 2016-07-05 | 2016-12-07 | 中国银行股份有限公司 | The method for real-time monitoring of financial application system online business and supervising device |
| CN108076098A (en) * | 2016-11-16 | 2018-05-25 | 北京京东尚科信息技术有限公司 | A kind of method for processing business and system |
| WO2018232490A1 (en) * | 2017-06-22 | 2018-12-27 | Zeu Crypto Networks Inc. | Multilevel queue based transaction traffic shaping for blockchains |
| CN109271417A (en) * | 2018-11-23 | 2019-01-25 | 福建榕基软件股份有限公司 | A kind of lightweight messages queue implementing method and storage equipment based on database |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115328680A (en) * | 2022-09-28 | 2022-11-11 | 天津卓朗昆仑云软件技术有限公司 | Message queue consumption abnormity assisting method and device and electronic equipment |
| CN115328680B (en) * | 2022-09-28 | 2023-01-31 | 天津卓朗昆仑云软件技术有限公司 | Auxiliary method and device for message queue consumption abnormity and electronic equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111158934B (en) | 2023-05-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106802826B (en) | Service processing method and device based on thread pool | |
| CN105677469B (en) | Timed task execution method and device | |
| US20180365254A1 (en) | Method and apparatus for processing information flow data | |
| CN110196761B (en) | Delayed task processing method and device | |
| US20130074095A1 (en) | Handling and reporting of object state transitions on a multiprocess architecture | |
| CN113238843B (en) | Task execution method, device, equipment and storage medium | |
| US8898126B1 (en) | Method and apparatus for providing concurrent data insertion and updating | |
| CN114428674A (en) | Task scheduling method, device, equipment and storage medium | |
| CN111158934A (en) | Method and device for realizing different-level delay splicing based on buffer queue received data | |
| CN114138894A (en) | Distributed transaction data synchronization method, device, equipment and readable storage medium | |
| CN107844363B (en) | Business transaction processing method, device, storage medium and equipment | |
| CN113761052A (en) | Database synchronization method and device | |
| CN111767126A (en) | System and method for distributed batch processing | |
| CN116383207A (en) | Data tag management method and device, electronic equipment and storage medium | |
| US6291757B1 (en) | Apparatus and method for processing music data | |
| CN114443313A (en) | Method and system for realizing delay queue by multiple strategies | |
| CN111431892B (en) | Accelerator management architecture and method and accelerator interface controller | |
| CN113220869A (en) | Text processing device and method for bank peripheral system | |
| US10546257B2 (en) | Optimizing event aggregation in an event-driven system | |
| CN112748855A (en) | Method and device for processing high-concurrency data request | |
| CN115987905B (en) | Multichannel flow control method, system, equipment and storage medium | |
| CN114090626A (en) | Method and device for acquiring database data | |
| CN112100192A (en) | Database lock waiting processing method and device | |
| CN116541374A (en) | Data processing method, device, equipment and computer readable storage medium | |
| CN111831425B (en) | Data processing method, device and equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |