CN111884767B - Method and system for solving 1-hop cost time full-order multicast segmentation problem - Google Patents
Method and system for solving 1-hop cost time full-order multicast segmentation problem Download PDFInfo
- Publication number
- CN111884767B CN111884767B CN202010579369.8A CN202010579369A CN111884767B CN 111884767 B CN111884767 B CN 111884767B CN 202010579369 A CN202010579369 A CN 202010579369A CN 111884767 B CN111884767 B CN 111884767B
- Authority
- CN
- China
- Prior art keywords
- index information
- message
- input
- input index
- host
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L1/00—Arrangements for detecting or preventing errors in the information received
- H04L1/12—Arrangements for detecting or preventing errors in the information received by using return channel
- H04L1/16—Arrangements for detecting or preventing errors in the information received by using return channel in which the return channel carries supervisory signals, e.g. repetition request signals
- H04L1/1607—Details of the supervisory signal
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/06—Management of faults, events, alarms or notifications
- H04L41/0654—Management of faults, events, alarms or notifications using network fault recovery
Abstract
The invention relates to the technical field of information data processing, in particular to a method and a system for solving the problem of full-order multicast segmentation at 1-hop cost.A primary message is sent to a host by an input source, a response message of the primary message is sent to corresponding downstream machines by the host in a full-order multicast mode, the host generates input index information T0' according to the received primary message and sends the input index information T0' to all the downstream machines, and the input index information T0' comprises message source information of the primary message and a unique sequence number seqNo of the primary message. Compared with the prior art, the invention has the advantages that: the method comprises the steps of sending index information of an input original message to a downstream, obtaining an input full-sequence stream index from the downstream during failover, requesting the original message to the upstream according to the index and restoring the original message into the full-sequence input stream, solving the segmentation problem of full-sequence multicast at 1-hop cost, and ensuring that a standby machine can reconstruct a message queue with complete information after taking over.
Description
Technical Field
The invention relates to the technical field of information data processing, in particular to a method and a system for solving the problem of full-order multicast segmentation in 1-hop cost.
Background
"full-order multicast" means: a mechanism that ensures that messages are delivered to multiple independent receivers in the same order when they are issued independently by multiple senders in a reliable multicast. The full-order multicast is the key of the core architecture of the transaction system, and the high availability of the transaction system is realized by depending on the full-order multicast. Meanwhile, the full-order multicast which is meaningful for constructing a high-availability core architecture also requires that a single point of failure can be prevented.
The 'hop count' is the network transmission frequency of the message between two devices, has great influence on the time delay of the system, and the 2-3 hop design is mostly adopted when the design transaction system of the mainstream exchange at home and abroad is highly available, such as a new transaction system of a large exchange, a deep exchange V5 transaction system and the like; but there are individual exchanges that build their trading systems based on 1-hop cost full-order multicasting. The idea of 1 jump complete sequence is as follows: the message sequencing result of the main receiving node is asynchronously copied to the slave nodes, and the gap recovery of the slave nodes after being upgraded to the main node can be completed by means of the next link in service. On the premise of only processing single point of failure, the technology realizes the lowest time delay, but has great side effect.
Taking fig. 1 as an example, after the TH (transaction host) processes the input order, the output is sent to the CS (front access machine) by means of OC (commission confirmation) and TC (transaction confirmation), the CS stores the OC/TC, and the OC/TC carries information sufficient to restore the input order queue. After the slave node TH' (transaction host standby) is upgraded to the master node, the missing part of the order queue can be recovered by means of the information stored in the CS. In fig. 1, the hop of the OC/TC from TH back to CS is a part of the critical path, and it is not counted in the full-order multicast cost in other schemes, so the cost of the full-order multicast is still 1 hop. Therefore, by deploying CS and TH cross-site, the method can resist single-site faults, and the number of cross-site crossing times introduced at the moment is 1.
However, the inventors of the present application have found that, although the latency cost is the lowest, the 1-hop cost full-order multicast still has a serious problem without further design, and is referred to as "split problem" herein.
The "slicing problem" is caused by slicing a single queue that must be processed sequentially into several slices that are disjoint from each other and saved by different hosts. Taking fig. 2 as an example, different messages have different CS attributions, where messages 1, 5, and 7 are attributed to one CS, messages 2 and 9 are attributed to another CS, messages 3 and 8 are attributed to another CS, messages 4 and 6 are attributed to another CS, and messages No. 1 to 9 (OC/TC) generated by the master TH are sent to the respective CS for storage. Since the primary TH issues the OC/TC via reliable multicast, the loss of the downlink message can be recognized by the CS according to the message gap and request retransmission, but this takes time. If the message 7 that should be stored by a CS is lost and not yet complemented when the primary TH crashes, a hole will appear in the message queue reconstructed by the takeover TH', and the scene cannot be reproduced.
Disclosure of Invention
The invention aims to solve the defects of the prior art, provides a method and a system for solving the problem of full-order multicast segmentation in 1-hop cost, and is particularly suitable for a trading system.
In order to achieve the purpose, a method for solving the problem of full-order multicast segmentation at 1-hop cost is designed, an input source sends an original message to a host, the host sends a response message of the original message to corresponding downstream machines in a full-order multicast mode, the host generates input index information T0' according to the received original message and sends the input index information T0' to all the downstream machines, the input index information T0' comprises message source information of the original message and a unique sequence number seqNo of the original message, and the message source information is used for positioning the source of the original message.
The invention also has the following preferable technical scheme:
the method comprises the following steps: step s1, each input source respectively endows the sent original message with a unique sequence number seqNo; step s2, the host generates input index information T0 'according to the received original message, and sends the input index information T0' to all downstream machines, and the host sends the response message of the original message to the corresponding downstream machine; and S3, when the host fails, the standby machine acquires input index information T0 'from the downstream machine and acquires the original message from the input source again according to the input index information T0'.
The host merges all received original messages into a sequenced input index stream TO, which contains the original messages and input index information T0'.
After the host generates the input index information T0', the input index information T0' is synchronized to the standby machine in different places.
When the host fails, the standby machine acquires all input index information T0' starting from the breakpoint from the downstream machine.
After receiving the original message, the host processes the original message to obtain a response message and generates a unique ordered response flow F0, where the unique ordered response flow F0 includes the response message and input index information T0'.
The host splits the response message in the unique ordered response flow F0 to send the response message to the corresponding downstream machine, and simultaneously sends the input index information T0' to all the downstream machines.
The method specifically comprises the following steps: generating input index information T0' through an index module; generating a response message of the original message through a processing module; the response message is split according to the corresponding downstream machine through a splitting module; sending the input index information T0' to all downstream machines through a distribution module, and sending response messages to the corresponding downstream machines; and acquiring input index information T0 'from a downstream machine through a backup module, and acquiring the original message from the input source again according to the input index information T0'.
The invention also relates to a system adopting the method for solving the problem of full-order multicast segmentation in the 1-hop cost, which comprises an index module for generating input index information T0'; a processing module for generating a response message of the original message; the splitting module is used for splitting the response message according to the corresponding downstream machine; the distribution module is used for sending the input index information T0' to all downstream machines and sending response messages to the corresponding downstream machines; and the backup module is used for acquiring the input index information T0 'from the downstream machine and acquiring the original message from the input source again according to the input index information T0'.
Compared with the prior art, the invention has the advantages that: the index information of the input original message is sent to the downstream, so that the input full-sequence stream index is obtained from the downstream during the fault switching, the original message is requested upstream according to the index and is restored into the full-sequence input stream, the segmentation problem of the full-sequence multicast at the 1-hop cost is solved, and the message queue with complete information can be reconstructed after the standby machine takes over.
Drawings
Fig. 1 is a schematic diagram of full-order multicast at 1-hop cost in the prior art.
Fig. 2 is a schematic diagram illustrating the segmentation problem of full-order multicast at 1-hop cost in the prior art.
FIG. 3 is a diagram of an input index stream TO in one embodiment.
FIG. 4 is a diagram of response flow generation in one embodiment.
FIG. 5 is a diagram illustrating splitting of a response flow, in accordance with an embodiment.
FIG. 6 is a flow diagram of data recovery upon failure, according to an embodiment.
Detailed Description
The present invention is further described in conjunction with the appended drawings, the structure and principles of such apparatus and methods being apparent to those skilled in the art. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
The inventor researches to find that the slicing problem can be solved by a mode of enabling all CS to store TH full-quantity output, but the scalability of the system is sacrificed by the mode. How to adopt a general processing mode to solve the segmentation problem generated by full-order multicast with 1-hop cost is a key for reducing the system time delay on the basis of ensuring the expandability, consistency and high availability of the exchange, especially when the system fault is recovered. Therefore, the present embodiment provides a method and a system for solving the problem of full-order multicast segmentation at 1-hop cost, which are preferable to solve the existing problem.
The system comprises an index module for generating input index information T0'; a processing module for generating a response message to the original message; the splitting module is used for splitting the response message according to the corresponding downstream machine; the distribution module is used for sending the input index information T0' to all downstream machines and sending response messages to corresponding downstream machines; and the backup module is used for acquiring the input index information T0 'from the downstream machine and acquiring the original message from the input source again according to the input index information T0'.
The method comprises the following steps:
and step s1, each input source respectively endows the sent original message with a unique sequence number seqNo.
And step s2, the host generates input index information T0 'according to the received original message, sends the input index information T0' to all downstream machines, and sends the response message of the original message to the corresponding downstream machines.
And S3, when the host fails, the standby machine acquires input index information T0 'from the downstream machine and acquires the original message from the input source again according to the input index information T0'.
Example 1
The method is specifically illustrated below, as shown in fig. 3, for a cluster, there are several input sources, which are named TOPIC [ i ] in this embodiment, i is the serial number of the input source, and the original messages on each input source are ordered, and each original message has its corresponding unique serial number seqNo. After necessary filtering, the master node merges all the original messages of the input sources into an input index stream T0 after sequencing, and the content contained in T0 retains the message source information of the original message, i.e., [ TOPIC, seqNo ] index, in addition to the original message, and is used for uniquely locating the source of the original message.
The full-order multicast T0 is used as input to the service process and asynchronously and synchronously provided to the standby machine, after the service process receives T0, the service process outputs the input index information T0 'in T0, and then outputs the response message thereof, and constructs a complete unique and ordered response stream F0, which includes the full amount of input index key values, and the generation flow of the response stream is shown in fig. 4, wherein the input index information T0' includes the message source information of the original message and the unique sequence number seqNo of the original message.
Since the input index information T0' is only increased and output, the traffic increase is not significant in response to the stream compared to the case without the index. As required, the output is split and sent to different downstream, and different response flows F [ i ] are generated, but each downstream response flow F [ i ] includes the full amount of input index information T0', as shown in fig. 5, at this time, any downstream has the full amount of index information, and when failover occurs, the node can acquire the full amount of index information from the downstream and restore the input index T0' therefrom. The original message can be restored from the upstream by the information of T0'.
When a failure occurs, all downstream T0' flow messages from the breakpoint are collected and the original information is pulled back upstream. Original information and T0' stream messages are combined, and then the original input stream T0 can be restored, and a site is restored. The flow of the failure recovery is shown in fig. 6. By restoring the complete original input stream T0, the processing sequence of the application process is consistent with that before the fault happens, so that the effects of completely restoring the site and continuing to serve the external service can be achieved.
The above description is only for the specific embodiment of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art should be considered to be within the technical scope of the present invention, and the technical solutions and novel concepts according to the present invention should be considered to be covered by the scope of the present invention. Also, it should be noted that the present embodiment is a design at the technical architecture level, and is not related to the service. Therefore, the method can be used for a transaction system and can also be used in other service scenes to construct a low-latency high-availability architecture.
Claims (8)
1. A method for solving the segmentation problem of full-order multicast at 1-hop cost, an input source sends an original message to a host, and the host sends a response message of the original message to corresponding downstream machines in a full-order multicast mode, characterized in that the host generates input index information T0' according to the received original message and sends the input index information T0' to all the downstream machines, the input index information T0' comprises message source information of the original message and a unique sequence number seqNo of the original message, and the message source information is used for positioning the source of the original message;
the method comprises the following steps:
step s1, each input source respectively endows the sent original message with a unique sequence number seqNo;
step s2, the host generates input index information T0 'according to the received original message, and sends the input index information T0' to all downstream machines, and the host sends the response message of the original message to the corresponding downstream machines;
and S3, when the host fails, the standby machine acquires input index information T0 'from the downstream machine and acquires the original message from the input source again according to the input index information T0'.
2. The method according TO claim 1, wherein the host merges all received original messages into a sequenced input index stream TO, and the input index stream TO comprises the original messages and input index information T0'.
3. The method according to claim 1, wherein the host remotely synchronizes the input index information T0 'to the standby after generating the input index information T0'.
4. The method according to claim 3, wherein when the host fails, the standby machine obtains all input index information T0' from the downstream machine starting from the breakpoint.
5. The method according to claim 1, wherein the host processes an original message after receiving the original message to obtain a response message and generates a unique ordered response stream F0, and the unique ordered response stream F0 includes the response message and input index information T0'.
6. The method according to claim 5, wherein the host splits the response message in the unique ordered response stream F0 to send the response message to the corresponding downstream machine, and sends the input index information T0' to all downstream machines at the same time.
7. The method for solving the problem of 1-hop cost time full-order multicast segmentation as claimed in claim 1, wherein the method specifically comprises the following steps:
generating input index information T0' through an index module;
generating a response message of the original message through a processing module;
splitting the response message according to the corresponding downstream machine through a splitting module;
sending the input index information T0' to all downstream machines through a distribution module, and sending response messages to corresponding downstream machines;
and acquiring input index information T0 'from a downstream machine through a backup module, and acquiring the original message from the input source again according to the input index information T0'.
8. A system using the method for solving the 1-hop cost time full-order multicast segmentation problem of claim 7, comprising:
an index module for generating input index information T0';
a processing module for generating a response message of the original message;
the splitting module is used for splitting the response message according to the corresponding downstream machine;
the distribution module is used for sending the input index information T0' to all downstream machines and sending response messages to corresponding downstream machines;
and the backup module is used for acquiring the input index information T0 'from the downstream machine and acquiring the original message from the input source again according to the input index information T0'.
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2020102174179 | 2020-03-25 | ||
| CN202010217417 | 2020-03-25 |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111884767A CN111884767A (en) | 2020-11-03 |
| CN111884767B true CN111884767B (en) | 2023-01-20 |
Family
ID=73156928
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010579369.8A Active CN111884767B (en) | 2020-03-25 | 2020-06-23 | Method and system for solving 1-hop cost time full-order multicast segmentation problem |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111884767B (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103634411A (en) * | 2013-12-16 | 2014-03-12 | 上海证券交易所 | Real-time market data broadcasting system and real-time market data broadcasting method with state consistency |
| WO2016177231A1 (en) * | 2015-07-10 | 2016-11-10 | 中兴通讯股份有限公司 | Dual-control-based active-backup switching method and device |
| CN106982138A (en) * | 2017-03-20 | 2017-07-25 | 国网福建省电力有限公司 | A kind of method that service consumer based on wide area Service-Oriented Architecture Based ensures data reliability in active-standby switch |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103647669B (en) * | 2013-12-16 | 2017-04-05 | 上海证券交易所 | It is a kind of to ensure the conforming system and method for distributed data processing |
| US10360118B2 (en) * | 2016-10-17 | 2019-07-23 | Cboe Exchange, Inc. | Low latency system having high availability computer architecture |
| CN109739804B (en) * | 2018-12-19 | 2022-09-23 | 卡斯柯信号有限公司 | Security platform input consistency comparison method based on 2-by-2-out-of-2 architecture |
-
2020
- 2020-06-23 CN CN202010579369.8A patent/CN111884767B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103634411A (en) * | 2013-12-16 | 2014-03-12 | 上海证券交易所 | Real-time market data broadcasting system and real-time market data broadcasting method with state consistency |
| WO2016177231A1 (en) * | 2015-07-10 | 2016-11-10 | 中兴通讯股份有限公司 | Dual-control-based active-backup switching method and device |
| CN106982138A (en) * | 2017-03-20 | 2017-07-25 | 国网福建省电力有限公司 | A kind of method that service consumer based on wide area Service-Oriented Architecture Based ensures data reliability in active-standby switch |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111884767A (en) | 2020-11-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106254094B (en) | A kind of method of data synchronization and system | |
| CN110209726A (en) | Distributed experiment & measurement system system, method of data synchronization and storage medium | |
| CN110581782B (en) | Disaster tolerance data processing method, device and system | |
| CN107241430A (en) | A kind of enterprise-level disaster tolerance system and disaster tolerant control method based on distributed storage | |
| CN104679614B (en) | A kind of Database Disaster Backup/Recover | |
| CN102467508A (en) | Method for providing database service and database system | |
| US20050022046A1 (en) | Method and system for enabling recovery of data stored in a computer network; a method and a system for recovering data stored in a computer network | |
| CN110392120B (en) | Method and device for recovering fault in message pushing process | |
| CN106656589A (en) | Server dual hot backup system | |
| WO2009081657A1 (en) | Node system, server switching method, server device, and data transfer method | |
| CN103209210A (en) | Method for improving erasure code based storage cluster recovery performance | |
| CN104077181A (en) | Status consistent maintaining method applicable to distributed task management system | |
| CN114050904B (en) | Consensus system and method based on two-level leader node fragmentation structure | |
| CN101267392A (en) | A realizing method for notifying downstream device in case of switch of uplink link status | |
| CN108390919A (en) | A kind of message synchronization system and method for highly reliable two-node cluster hot backup | |
| CN110213359A (en) | A kind of car networking networking data delivery system and method based on D2D | |
| CN114297296A (en) | Server active-standby switching system for broadcasting | |
| CN111884767B (en) | Method and system for solving 1-hop cost time full-order multicast segmentation problem | |
| CN109347655A (en) | Fault recovery system and method and storage medium based on network data | |
| CN110351122B (en) | Disaster recovery method, device, system and electronic equipment | |
| Nsaif et al. | RURT: A novel approach for removing the unnecessary redundant traffic in any HSR closed-loop network type | |
| JP2019510302A (en) | Send data across regions | |
| CN112948484A (en) | Distributed database system and data disaster recovery drilling method | |
| CN111083074A (en) | High availability method and system for main and standby dual OSPF state machines | |
| Almeida et al. | Using light-weight groups to handle timing failures in quasi-synchronous systems |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |