CN114124668A - System and method for ensuring consistency of market quotation slices of multiple hosts - Google Patents
System and method for ensuring consistency of market quotation slices of multiple hosts Download PDFInfo
- Publication number
- CN114124668A CN114124668A CN202111291635.8A CN202111291635A CN114124668A CN 114124668 A CN114124668 A CN 114124668A CN 202111291635 A CN202111291635 A CN 202111291635A CN 114124668 A CN114124668 A CN 114124668A
- Authority
- CN
- China
- Prior art keywords
- market
- quotation
- slice
- server
- main server
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 51
- 230000015654 memory Effects 0.000 claims abstract description 25
- 230000002159 abnormal effect Effects 0.000 claims description 14
- 238000004891 communication Methods 0.000 claims description 12
- 238000012545 processing Methods 0.000 claims description 10
- 238000012163 sequencing technique Methods 0.000 claims description 8
- 230000002085 persistent effect Effects 0.000 claims description 6
- 238000012856 packing Methods 0.000 claims description 4
- 238000004364 calculation method Methods 0.000 claims description 3
- 238000004806 packaging method and process Methods 0.000 claims description 2
- 238000012423 maintenance Methods 0.000 abstract description 2
- 238000005457 optimization Methods 0.000 abstract description 2
- 108700009949 PTP protocol Proteins 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 210000004556 brain Anatomy 0.000 description 1
- 230000006835 compression Effects 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000012407 engineering method Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000003860 storage Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/06—Management of faults, events, alarms or notifications
- H04L41/0654—Management of faults, events, alarms or notifications using network fault recovery
- H04L41/0663—Performing the actions predefined by failover planning, e.g. switching to standby network elements
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q40/00—Finance; Insurance; Tax strategies; Processing of corporate or income taxes
- G06Q40/04—Trading; Exchange, e.g. stocks, commodities, derivatives or currency exchange
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/10—Active monitoring, e.g. heartbeat, ping or trace-route
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/10—Active monitoring, e.g. heartbeat, ping or trace-route
- H04L43/103—Active monitoring, e.g. heartbeat, ping or trace-route with adaptive polling, i.e. dynamically adapting the polling rate
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1001—Protocols in which an application is distributed across nodes in the network for accessing one among a plurality of replicated servers
- H04L67/1004—Server selection for load balancing
- H04L67/1008—Server selection for load balancing based on parameters of servers, e.g. available memory or workload
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/104—Peer-to-peer [P2P] networks
- H04L67/1044—Group management mechanisms
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
- H04L69/16—Implementation or adaptation of Internet protocol [IP], of transmission control protocol [TCP] or of user datagram protocol [UDP]
Landscapes
- Engineering & Computer Science (AREA)
- Signal Processing (AREA)
- Computer Networks & Wireless Communication (AREA)
- Business, Economics & Management (AREA)
- Theoretical Computer Science (AREA)
- Accounting & Taxation (AREA)
- Finance (AREA)
- General Health & Medical Sciences (AREA)
- Cardiology (AREA)
- Health & Medical Sciences (AREA)
- Physics & Mathematics (AREA)
- Marketing (AREA)
- Computing Systems (AREA)
- Computer Security & Cryptography (AREA)
- General Business, Economics & Management (AREA)
- Technology Law (AREA)
- Computer Hardware Design (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Strategic Management (AREA)
- Economics (AREA)
- Development Economics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Financial Or Insurance-Related Operations Such As Payment And Settlement (AREA)
Abstract
The invention relates to the technical field of stock market generating systems, in particular to a system and a method for ensuring consistency of market slices of multiple hosts, wherein the system consists of a plurality of servers which are responsible for generating market slices, one of the servers is used as a main server, the other servers are used as standby servers, each standby server is communicated with the main server through UDP and TCP protocols, the main server is responsible for generating maintenance quotation slice marks, the standby servers request the main server for synchronizing the quotation slice marks into respective shared memories, if the standby servers cannot receive data and heartbeat in a certain interval, starting the quotation slice control engines to respectively generate quotation slice marks to be stored in the shared memory, enabling the data receiving module to support a bypass UDP to receive multi-channel data, and the optimization is carried out, so that the real-time performance of the market quotation calculated by each server is improved, and the availability of the market quotation system is also improved by receiving the multi-channel data.
Description
Technical Field
The invention relates to the technical field of stock market quotation generating systems, in particular to a system and a method for ensuring consistency of quotation slices of multiple hosts.
Background
At present, each large stock exchange is used as a stock centralized trading platform, which not only needs to provide an open, fair and safe stock centralized trading platform for the market, but also provides real-time, efficient and accurate stock market quotation data for investors. The quality of market data constantly influences the behavior of investors, so that each large stock exchange is dedicated to creating a market generating system with low time delay, high throughput, strong expansibility and high availability.
In the face of a large amount of transaction data, uncontrollable networks and other problems, a market system of a single host often cannot ensure that market slices can be generated safely and stably within a transaction day. Therefore, in consideration of the security and availability of the system, each exchange usually deploys a multi-host market quotation generation system with multiple hosts and multiple standby machines, and the host can be switched to the standby machine to send market quotation slices when single-point exception occurs. The market is a discrete market data because the market continuously changes in real time with the generation of the transaction data, and the market slice is a market data which is produced by the host computer at a fixed point under a specific frequency (for example, 3 s/frame). When the hosts cannot receive the same transaction data at the same time or the generation time of the first market section is inconsistent, the market sections of the hosts cannot be guaranteed to be consistent. At this time, if the host fails, the system may not be able to switch to the standby machine in time and continue to provide accurate market conditions to the outside. Therefore, on the premise of improving safety and usability, in order to ensure seamless switching of multiple hosts and continuous accuracy of market quotation slicing, a method for ensuring homologous consistency of the market quotation slicing of the multiple hosts needs to be designed.
At present, many engineering methods can solve the problem of the homology of the market quotations of multiple hosts, such as cluster thought in a distributed system, consensus algorithm in a block chain and the like. The algorithms solve the problem of split brain in the switching process, but the algorithm is complex to realize and low in fault tolerance, and the availability of the multi-host market system is greatly reduced.
Therefore, it is necessary to design a method that is easy to implement and ensures the homology of market conditions slices during switching of multiple hosts, so as to improve the availability of switching of multiple hosts and the continuity accuracy of market conditions slices.
Disclosure of Invention
The invention aims to solve the problem that the market quotation system of the existing single host often cannot ensure that market quotation slices can be generated safely and stably in a trading day when the existing market quotation system of the single host faces a large amount of trading data, an uncontrollable network and the like, and provides a system and a method for ensuring the consistency of the market quotation slices of multiple hosts.
In order to achieve the purpose, a system for ensuring the consistency of the quotation slices of a plurality of hosts is designed, and is characterized in that the system consists of a plurality of servers which are responsible for generating the quotation slices, wherein one server is used as a main server, the rest servers are standby servers, each standby server is communicated with the main server through UDP and TCP protocols, the main server is responsible for generating maintenance quotation slice marks, the standby servers request the main server for synchronizing the quotation slice marks to respective shared memories, and if the standby servers cannot receive data and heartbeats within a certain interval, the quotation slice control engines are started to respectively generate the quotation slice marks to be stored in the shared memories; the system is characterized in that each server comprises an initialization module, a data receiving and forwarding module, a market slicing control module and a market slicing generation module from top to bottom, after the system is initialized, the data receiving and forwarding module sequences received transaction data streams and stores the transaction data streams into a shared memory in sequence, because each host receives the same transaction data streams and each data has a unique serial number identifier in a corresponding product set (set), the transaction data stored into the shared memory after sequencing are completely consistent, the market slicing control module polls the transaction data in the shared memory in real time and generates market slicing marks of each code according to rules, a standby server synchronizes marks of a main server through a TCP and stores the marks into the shared memory in sequence, the market generation module of the standby server acquires information in the market slicing marks in real time and calculates the transaction data in the shared memory in real time by taking the mark position as a cut-off position, and slicing the stock codes according to the quotation slice types of the stock codes and then storing the stock codes into an issuing queue, and sequentially taking the data packets of the issuing queue of the main server by the quotation issuing platform and sending the data packets to the subscribed users.
Preferably, the market information publishing system of the market information publishing platform only subscribes the market information delivery of one market information server, under normal conditions, the market information publishing system only subscribes the market information of the main server and delivers the market information, and when the market information cannot be received after the set time, the market information publishing system considers that the communication is abnormal or the main server is abnormal and switches to one of the configured standby servers in sequence to subscribe the market information: if the communication between the standby server and the main server is smooth, the generated quotation is consistent with that of the main server, and the quotation subscribed after the switching of the quotation issuing system can be regarded as the same quotation; if the communication between the standby server and the main server fails, the backup server with set time can autonomously generate quotation slice marks and quotation slices of all stock codes, and before the standby server is disconnected with the main server, the coded quotation is completely consistent with that of the main server, so that when the quotation issuing system is switched to the standby server, if the latest quotation slice serial number received is greater than the breakpoint serial number, the latest quotation slice serial number is immediately issued to ensure that the quotation does not flow back, and the quotations of the main server and the standby server can be regarded as the same quotation after being merged.
Meanwhile, a method for ensuring the consistence of the quotation slices of the multiple hosts by adopting the system is designed, and is characterized by comprising the following steps:
s1, a plurality of servers perform time synchronization on time based on a communication protocol;
s2, after a server is started, a data receiving and distributing subsystem of each host starts different processes to subscribe theme data according to different channel numbers, each sub-process receives transaction data streams of each channel in real time and multicasts the transaction data streams to other hosts in real time, simultaneously receives multicast of data streams of the other hosts with the same channel number, screens out the fastest transaction data streams according to a first-receiving first-processing principle to analyze the transaction data streams, sorts the transaction data according to execution serial numbers, each sub-process puts the sorted transaction data into a corresponding shared memory container according to a strip, and a lastPos position newly recorded in a set is added with one, and the number of records is also added with one;
s3, in a quotation slice control subsystem of the main server, starting a plurality of sub-processes according to the set number, wherein each sub-process corresponds to a quotation slice control object, each quotation slice control object periodically acquires lastPos in the corresponding set, polls quotation slice controllers of all stock codes in the set in the period by taking the lastPos as a cut-off position, and each quotation slice controller calculates the time difference between the current time and the previous quotation slice timestamp of the stock codes: if the transaction data is changed after exceeding 3 seconds, generating incremental market slicing in the code book period, otherwise, not slicing; if the time difference exceeds 1 minute, generating a full amount of market quotation slices in the code book period, and simultaneously, sequentially storing the market quotation slice types of all the codes and lastPos of the set of the code book period as market quotation slice marks into market quotation slice control shared memory containers corresponding to the set in a strip-by-strip manner;
s4, continuously subscribing market condition slice marks to the main server by market condition slice control subsystems of the other standby servers, and synchronizing the market condition slice marks to respective snapCtrl in sequence according to serial numbers of the market condition slice marks;
s5, in the market generating subsystem of the server, each sub-process maintains a set, and the sub-process mainly performs three operations:
reading elements in the snapCtrl in real time, and acquiring lastPos and corresponding market slice types in each element;
calculating a corresponding statistical value in real time according to the read lastPos in a certain element in the snapCrl;
obtaining a corresponding market slice object according to the code, generating market slices according to the corresponding market slice types, and storing the market slices into a corresponding issuing queue and persisting the market slices to a file after packaging and FAST (FIX attached for streaming) protocol coding;
and S6, after the system operation is finished, decoding FAST codes of the persistent files of the servers into market data, sequencing the market data according to the serial numbers of market slices, and comparing all market slices of all codes after sequencing so as to verify the correctness of the multi-host market slice consistency method, wherein finally, the market slices generated by all the servers are completely consistent.
Preferably, step S4 can be replaced with: when the standby server does not receive the message within the set time, heartbeat is actively sent to the main server, if the heartbeat reply cannot be received within the set time, the connection with the main server is considered to be abnormal or the main server is considered to be abnormal and disconnected, and at the moment, each market condition slice control subsystem of the standby server is changed from a passive market condition slice processing mark to an active market condition slice generating mark; polling set, polling transaction data from the first of the set until the latest position, determining the current lastPos; a quotation slice control object of the subsystem polls quotation slice controllers corresponding to all stock codes in the set, and each quotation slice controller calls a quotation slice control algorithm to determine the quotation slice type of the code in the current period; and storing the information such as the code, the type of the market quotation slice, the lastPos and the like as a market quotation slice mark record into the snapCtrl maintained by the subsystem.
Preferably, the method further comprises step S5, the market conditions generating subsystem of the standby server starts a plurality of sub-processes according to set, each sub-process polls the market conditions slice mark in snapCtrl, polls the transaction data in set for real-time calculation of market conditions with lastPos as a cut-off position, polls the market conditions slice object after polling the transaction data, generates market conditions slices according to the types of the market conditions slices of each code, packs the market conditions according to the same packing strategy, stores the market conditions into the corresponding issuing queue via FAST coding, and persists the market conditions slice object to the file.
Compared with the prior art, the invention has the advantages that:
1. the data receiving module supports bypass UDP to receive multi-channel data and performs optimization, so that the real-time performance of calculation of quotations of each server is improved, and the usability of a quotation system is also improved by receiving the multi-channel data;
2. compared with cluster type multi-host market quotation isogenism, the method can ensure that market quotations among the multiple hosts are completely consistent under a normal scene, and when downstream needs to be switched, the system can be switched to any host to receive the market quotation at will, so that the availability of the system is improved;
3. the TCP back-up is supported, and when the standby server is reconnected, if the market condition slice mark is lost, the information of the lost part can be subscribed from the main server;
4. the service is processed in a multi-process mode, a locking mechanism is avoided, and the parallel processing capacity of the service is improved;
5. and the shared memory is used as a means for data interaction between processes, so that the data access performance is improved.
Drawings
Fig. 1 is a schematic diagram illustrating normal operation of a multi-host market segment reconciliation method according to embodiment 1 of the present disclosure.
Detailed Description
The present invention is further described below in conjunction with the following drawings, the structure and principle of which will be apparent to those skilled in the art. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
The method is respectively deployed on different servers, all the applications receive the same transaction data stream, and the received transaction data streams are respectively processed and the real-time quotation is calculated. During normal operation, each application stores the market information slice mark of each market information slice and each market information slice to the shared memory. And after the operation is finished, comparing the quotation slices in the shared memory of all the quotation hosts and judging whether the quotation slices are completely consistent. Under the condition that the main server operates abnormally, the standby server can generate the quotation slices according to the mechanism of the standby server, and the quotation slices cannot reflow.
The following embodiments were performed in a test environment. And taking four servers provided with market system applications as a tester, determining one server as a main server and the other three servers as standby servers. And after the system is operated, comparing all the market information slices generated by each server. Fig. 1 shows a normal flow of the present embodiment, and as shown in the figure, the system includes:
step a, four servers carry out time synchronization and time synchronization based on a PTP protocol.
And b, after the four servers are started, the data receiving and distributing subsystem of each host starts different processes to subscribe theme data according to different channel numbers (in order to support distributed processing and improve the processing performance of the system, the trading system divides different stocks into different channels to respectively carry out trading processing, and each channel processes the trading of a group of stocks to generate trading data corresponding to the group of stocks, wherein the trading data comprise orders and deals). Each subprocess receives the transaction data stream of each channel in real time and multicasts the transaction data stream to other hosts in real time, and simultaneously receives the data stream multicast of the same channel number of other hosts, screens out the fastest transaction data stream by a first-receiving first-processing principle to analyze the transaction data stream (the binary stream data is analyzed into structural body data with understandable codes), and sorts the transaction data stream according to the execution serial number (all transaction data in each channel have the unique execution serial number and increase the execution serial number in an increasing way). And each subprocess puts the sequenced transaction data into a corresponding shared memory container (one subprocess maintains a data storage container, hereinafter referred to as set) according to a bar, and when one transaction data is added into the set, the last recorded position lastPos in the set is added with one (the position is sequentially increased from 0), and the number of records is also added with one.
And c, starting a plurality of sub-processes according to the set quantity in the quotation slice control subsystem of the main server, wherein each sub-process corresponds to a quotation slice control object, each quotation slice control object periodically acquires lastPos in the corresponding set, and polls quotation slice controllers of all stock codes in the set in the period (one stock code maintains one quotation slice controller) by taking the lastPos as a cut-off position. Each market slice controller calculates the time difference between the current time and the timestamp of a market slice on the stock code: if the transaction data is changed after exceeding 3 seconds, generating incremental market slicing in the code book period, otherwise, not slicing; if the time difference exceeds 1 minute, a full market slice is generated within the codebook period. Meanwhile, the market slice type (0 is not generated, 1 is generated full, 2 is generated increment) of each code and lastPos of the period set are sequentially stored as market slice marks into a market slice control shared memory container (a subprocess maintains a market slice control storage container, hereinafter referred to as snapCtrl) corresponding to the set in a strip-by-strip manner.
And d, continuously subscribing market condition slice marks to the main server by market condition slice control subsystems of the other three standby servers, and synchronizing the market condition slice marks to respective snapCtrl in sequence according to serial numbers of the market condition slice marks.
Step e, in the market generating subsystem of the four servers, each sub-process maintains a set, and the sub-process mainly performs three operations:
1. reading the elements in the snapCtrl in real time, and obtaining lastPos and the corresponding market slice type in each element.
2. And calculating a corresponding statistical value in real time according to the read lastPos in a certain element in the snapCrl.
3. And acquiring a corresponding market slice object according to the codes (one stock code corresponds to one market slice object, and stock codes of four hosts are completely consistent), and generating market slices according to corresponding market slice types. And the data is packaged, encoded by FAST (a compression encoding), stored in a corresponding issuing queue and persisted to a file.
And f, after the system operation is finished, decoding FAST from the persistent files of the four servers into market data, and sequencing according to the serial numbers of market slices. And after sequencing, comparing all the market quotation slices of all the codes to verify the correctness of the multi-host market quotation slice consistency method, and finally, completely consistent market quotation slices generated by all the four servers.
And ending the operation flow in the normal scene.
In embodiment 2, in consideration of the abnormal communication between the multiple host systems, once the standby server loses contact with the primary server, it needs to ensure that the standby server can independently perform production. For the operation of the step d, the standby server synchronizes the market information slice marking, when the connection is abnormal, the following adjustment needs to be made:
and d, when the standby server cannot receive the message within a certain time (such as 3 s), actively sending heartbeat to the main server. If the heartbeat reply cannot be received within a certain time, the connection with the main server is considered to be abnormal or the main server is considered to be abnormal and disconnected. At the moment, each market slice control subsystem of the standby server is converted from a passive processing market slice mark into an active generation market slice mark:
1. polling set, polling transaction data from the first of the set until the latest position, determining the current lastPos;
2. a quotation slice control object of the subsystem polls quotation slice controllers corresponding to all stock codes in a set, and each quotation slice controller calls a quotation slice control algorithm to determine the quotation slice type of the code in the current period (since the quotation slice control algorithm is called by the standby server for the first time at this moment, the quotation slice types of all the codes are all 1, and then full quotation slices are generated);
3. and storing the information such as the code, the type of the market quotation slice, the lastPos and the like as a market quotation slice mark record into the snapCtrl maintained by the subsystem.
And e, the market generating subsystem of the standby server starts a plurality of sub-processes according to the set, each sub-process polls market slice marks in the snapCrl, and polls the transaction data in the set to calculate the market in real time by taking lastPos as a cut-off position. And after the polling transaction data is finished, polling the market quotation slice object, and generating a quotation slice according to the quotation slice type of each code. Packing the quotation according to the same packing strategy, storing the quotation into a corresponding issuing queue after FAST coding, and persisting the quotation to a file.
Example 3, the invention involves two parties: the system comprises a market generating engine and a market publishing system. The above describes a process of generating a consistent quotation slice by a multi-host quotation, and the following describes a process of issuing a homologous quotation by a quotation issuing system.
The market publishing system only subscribes market publishing of one market server. Under normal conditions, the market information publishing system only subscribes the market information of the main server and sends the market information. When the market can not be received within a certain time (such as 3 s), the market publishing system considers that the communication is abnormal or the primary server is abnormal, and switches to one of the configured standby servers to subscribe the market in sequence:
1. if the communication between the standby server and the main server is smooth, the generated quotation is consistent with that of the main server, and the quotation subscribed after the switching of the quotation issuing system can be regarded as the same quotation;
2. if the communication between the standby server and the main server fails, the standby server can autonomously generate quotation slice marks and quotation slices of all stock codes within a certain time (for example, 3 s). Before the standby server is disconnected with the main server, the coding market is completely consistent with that of the main server. Therefore, when the market publishing system is switched to the standby server, if the latest market slice serial number received is greater than the breakpoint serial number, the market slice serial number is immediately issued to ensure that the market does not reflow. After the quotations of the main server and the standby server are combined, the quotations can be regarded as the same quotation.
Claims (5)
1. A system for ensuring the consistency of quotation slices of a plurality of hosts is characterized in that the system consists of a plurality of servers which are responsible for generating the quotation slices, wherein one server is used as a main server, the rest servers are standby servers, each standby server is communicated with the main server through UDP and TCP protocols, the main server is responsible for generating and maintaining quotation slice marks, the standby servers request the main server to synchronize the quotation slice marks to respective shared memories, and if the standby servers cannot receive data and heartbeat within a certain interval, a quotation slice control engine is started to respectively generate the quotation slice marks to be stored in the shared memories; the system is characterized in that each server comprises an initialization module, a data receiving and forwarding module, a market slicing control module and a market slicing generation module from top to bottom, after the system is initialized, the data receiving and forwarding module sequences received transaction data streams and stores the transaction data streams into a shared memory in sequence, because each host receives the same transaction data streams and each data has a unique serial number identifier in a corresponding product set (set), the transaction data stored into the shared memory after sequencing are completely consistent, the market slicing control module polls the transaction data in the shared memory in real time and generates market slicing marks of each code according to rules, a standby server synchronizes marks of a main server through a TCP and stores the marks into the shared memory in sequence, the market generation module of the standby server acquires information in the market slicing marks in real time and calculates the transaction data in the shared memory in real time by taking the mark position as a cut-off position, and slicing the stock codes according to the quotation slice types of the stock codes and then storing the stock codes into an issuing queue, and sequentially taking the data packets of the issuing queue of the main server by the quotation issuing platform and sending the data packets to the subscribed users.
2. The system according to claim 1, wherein the market publishing platform includes a market publishing system, the market publishing system only subscribes to market delivery of one market server, the market publishing system only subscribes to market delivery of the primary server and delivers the market delivery of the primary server under normal conditions, and when the market is not received for a set time, the market publishing system considers that communication is abnormal or the primary server is abnormal and switches to one of the configured backup servers in sequence to subscribe to market delivery: if the communication between the standby server and the main server is smooth, the generated quotation is consistent with that of the main server, and the quotation subscribed after the switching of the quotation issuing system can be regarded as the same quotation; if the communication between the standby server and the main server fails, the backup server with set time can autonomously generate quotation slice marks and quotation slices of all stock codes, and before the standby server is disconnected with the main server, the coded quotation is completely consistent with that of the main server, so that when the quotation issuing system is switched to the standby server, if the latest quotation slice serial number received is greater than the breakpoint serial number, the latest quotation slice serial number is immediately issued to ensure that the quotation does not flow back, and the quotations of the main server and the standby server can be regarded as the same quotation after being merged.
3. A method for ensuring consistency of market segments for multiple hosts using the system of claim 1, comprising the steps of:
s1, a plurality of servers perform time synchronization on time based on a communication protocol;
s2, after a server is started, a data receiving and distributing subsystem of each host starts different processes to subscribe theme data according to different channel numbers, each sub-process receives transaction data streams of each channel in real time and multicasts the transaction data streams to other hosts in real time, simultaneously receives multicast of data streams of the other hosts with the same channel number, screens out the fastest transaction data streams according to a first-receiving first-processing principle to analyze the transaction data streams, sorts the transaction data according to execution serial numbers, each sub-process puts the sorted transaction data into a corresponding shared memory container according to a strip, and a lastPos position newly recorded in a set is added with one, and the number of records is also added with one;
s3, in a quotation slice control subsystem of the main server, starting a plurality of sub-processes according to the set number, wherein each sub-process corresponds to a quotation slice control object, each quotation slice control object periodically acquires lastPos in the corresponding set, polls quotation slice controllers of all stock codes in the set in the period by taking the lastPos as a cut-off position, and each quotation slice controller calculates the time difference between the current time and the previous quotation slice timestamp of the stock codes: if the transaction data is changed after exceeding 3 seconds, generating incremental market slicing in the code book period, otherwise, not slicing; if the time difference exceeds 1 minute, generating a full amount of market quotation slices in the code book period, and simultaneously, sequentially storing the market quotation slice types of all the codes and lastPos of the set of the code book period as market quotation slice marks into market quotation slice control shared memory containers corresponding to the set in a strip-by-strip manner;
s4, continuously subscribing market condition slice marks to the main server by market condition slice control subsystems of the other standby servers, and synchronizing the market condition slice marks to respective snapCtrl in sequence according to serial numbers of the market condition slice marks;
s5, in the market generating subsystem of the server, each sub-process maintains a set, and the sub-process mainly performs three operations:
reading elements in the snapCtrl in real time, and acquiring lastPos and corresponding market slice types in each element;
calculating a corresponding statistical value in real time according to the read lastPos in a certain element in the snapCrl;
obtaining a corresponding market slice object according to the code, generating market slices according to the corresponding market slice types, and storing the market slices into a corresponding issuing queue and persisting the market slices to a file after packaging and FAST (FIX attached for streaming) protocol coding;
and S6, after the system operation is finished, decoding FAST codes of the persistent files of the servers into market data, sequencing the market data according to the serial numbers of market slices, and comparing all market slices of all codes after sequencing so as to verify the correctness of the multi-host market slice consistency method, wherein finally, the market slices generated by all the servers are completely consistent.
4. The method according to claim 3, wherein the step S4 can be replaced by: when the standby server does not receive the message within the set time, heartbeat is actively sent to the main server, if the heartbeat reply cannot be received within the set time, the connection with the main server is considered to be abnormal or the main server is considered to be abnormal and disconnected, and at the moment, each market condition slice control subsystem of the standby server is changed from a passive market condition slice processing mark to an active market condition slice generating mark; polling set, polling transaction data from the first of the set until the latest position, determining the current lastPos; a quotation slice control object of the subsystem polls quotation slice controllers corresponding to all stock codes in the set, and each quotation slice controller calls a quotation slice control algorithm to determine the quotation slice type of the code in the current period; and storing the information such as the code, the type of the market quotation slice, the lastPos and the like as a market quotation slice mark record into the snapCtrl maintained by the subsystem.
5. The method according to claim 4, further comprising a step S5, wherein the market generating subsystem of the standby server starts a plurality of sub-processes according to set, each sub-process polls market slice mark in snapCtrl, polls transaction data in set for real-time calculation of market with lastPos as a cut-off position, polls market slice object after completing the polling transaction data, generates market slices according to market slice types of each code, packs market according to the same packing strategy, stores into corresponding issue queue after FAST encoding, and persists to files.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111291635.8A CN114124668B (en) | 2021-11-03 | 2021-11-03 | System and method for guaranteeing consistency of multi-host quotation slices |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111291635.8A CN114124668B (en) | 2021-11-03 | 2021-11-03 | System and method for guaranteeing consistency of multi-host quotation slices |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114124668A true CN114124668A (en) | 2022-03-01 |
| CN114124668B CN114124668B (en) | 2024-08-02 |
Family
ID=80380529
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111291635.8A Active CN114124668B (en) | 2021-11-03 | 2021-11-03 | System and method for guaranteeing consistency of multi-host quotation slices |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114124668B (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7603316B1 (en) * | 2004-11-01 | 2009-10-13 | American Express Travel Related Services Company, Inc. | Message delivery delegate system and method |
| CN103532753A (en) * | 2013-10-11 | 2014-01-22 | 中国电子科技集团公司第二十八研究所 | Double-computer hot standby method based on memory page replacement synchronization |
| CN103647668A (en) * | 2013-12-16 | 2014-03-19 | 上海证券交易所 | Host group decision system in high availability cluster and switching method for host group decision system |
| CN103838640A (en) * | 2012-11-23 | 2014-06-04 | 景幂机械（上海）有限公司 | Data backup system for securities trading |
| CN107247729A (en) * | 2017-05-03 | 2017-10-13 | 中国银联股份有限公司 | A kind of document handling method and device |
| CN110415027A (en) * | 2019-07-16 | 2019-11-05 | 上海金融期货信息技术有限公司 | A kind of big data market plateform system |
| CN112965860A (en) * | 2021-03-11 | 2021-06-15 | 中科驭数（北京）科技有限公司 | Snapshot market distribution method, device, equipment and storage medium |
-
2021
- 2021-11-03 CN CN202111291635.8A patent/CN114124668B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7603316B1 (en) * | 2004-11-01 | 2009-10-13 | American Express Travel Related Services Company, Inc. | Message delivery delegate system and method |
| CN103838640A (en) * | 2012-11-23 | 2014-06-04 | 景幂机械（上海）有限公司 | Data backup system for securities trading |
| CN103532753A (en) * | 2013-10-11 | 2014-01-22 | 中国电子科技集团公司第二十八研究所 | Double-computer hot standby method based on memory page replacement synchronization |
| CN103647668A (en) * | 2013-12-16 | 2014-03-19 | 上海证券交易所 | Host group decision system in high availability cluster and switching method for host group decision system |
| CN107247729A (en) * | 2017-05-03 | 2017-10-13 | 中国银联股份有限公司 | A kind of document handling method and device |
| CN110415027A (en) * | 2019-07-16 | 2019-11-05 | 上海金融期货信息技术有限公司 | A kind of big data market plateform system |
| CN112965860A (en) * | 2021-03-11 | 2021-06-15 | 中科驭数（北京）科技有限公司 | Snapshot market distribution method, device, equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114124668B (en) | 2024-08-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9819733B2 (en) | Peer-to-peer exchange of data resources in a control system | |
| US10362323B2 (en) | Methods and apparatus of encoding real time media content | |
| US7966514B2 (en) | Scalable fault tolerant system | |
| US8169856B2 (en) | Time synchronization in cluster systems | |
| JP2004070712A (en) | Data delivery method, data delivery system, split delivery data receiving method, split delivery data receiving device and split delivery data receiving program | |
| CN112272291B (en) | Video storage method, device, management equipment and readable storage medium | |
| CN106254094A (en) | A kind of method of data synchronization and system | |
| CN107153660A (en) | The fault detect processing method and its system of distributed data base system | |
| WO2014197963A1 (en) | Failover system and method | |
| CN109639512B (en) | Hot backup method of VTS multi-sensor information comprehensive processing system | |
| CN113704354A (en) | Data synchronization method and device, computer equipment and storage medium | |
| CN106850255A (en) | A kind of implementation method of multi-computer back-up | |
| CN111726388A (en) | Cross-cluster high-availability implementation method, device, system and equipment | |
| CN114625489B (en) | Access request response method, device and distributed system | |
| CN113220473B (en) | Data storage method and system | |
| CN114124668A (en) | System and method for ensuring consistency of market quotation slices of multiple hosts | |
| AU2012327228B2 (en) | A system for high reliability and high performance application message delivery | |
| CN114554593A (en) | Data processing method and device | |
| CN107147708A (en) | A kind of peer-to-peer network redundant data balance method and system | |
| CN113794541A (en) | Main/standby switching management method, device, equipment and machine readable storage medium | |
| CN113821362A (en) | Data copying method and device | |
| CN115276710B (en) | Transponder transmission system, transponder positioning compensation method, device and medium | |
| CN113722125B (en) | Data processing method and system based on distributed subscription and release network | |
| CN117667531B (en) | Four-machine hot standby redundancy processing method, system, electronic equipment and storage medium | |
| US7587623B2 (en) | Systems and methods for recovering a trading system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |