CN111314217B - Multilink transmission system - Google Patents
Multilink transmission system Download PDFInfo
- Publication number
- CN111314217B CN111314217B CN201911250991.8A CN201911250991A CN111314217B CN 111314217 B CN111314217 B CN 111314217B CN 201911250991 A CN201911250991 A CN 201911250991A CN 111314217 B CN111314217 B CN 111314217B
- Authority
- CN
- China
- Prior art keywords
- publishing
- message
- subscriber
- subscription
- topic
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L45/00—Routing or path finding of packets in data switching networks
- H04L45/24—Multipath
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L47/00—Traffic control in data switching networks
- H04L47/10—Flow control; Congestion control
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L47/00—Traffic control in data switching networks
- H04L47/10—Flow control; Congestion control
- H04L47/37—Slow start
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L49/00—Packet switching elements
- H04L49/90—Buffering arrangements
- H04L49/9005—Buffering arrangements using dynamic buffer space allocation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L49/00—Packet switching elements
- H04L49/90—Buffering arrangements
- H04L49/9057—Arrangements for supporting packet reassembly or resequencing
Abstract
The invention discloses a multilink transmission system which can improve throughput, reduce time delay and ensure reliable transmission of data. The technical scheme is as follows: the system comprises an issuing end, a plurality of subscribing ends, a plurality of channels and a theme flow, wherein the issuing end is used for executing a publishing task of a specified theme flow; the channel is used for task publishing at a publishing terminal and message receiving at a subscribing terminal; the topic flow message is stored in the topic flow and is published to a plurality of subscribing terminals through a plurality of channels in parallel; the subscription end is used for receiving the theme flow messages through a plurality of channels and transmitting the received theme flow messages to the application layer for processing.
Description
Technical Field
The invention relates to the field of financial transaction software, in particular to a high-performance multi-link transmission system with low delay and high reliability.
Background
Financial transaction software is essentially a streaming processing system and therefore inevitably involves data streaming between different modules, which is generally handled by a transport communication component, also referred to as communication middleware.
The transmitting communication component generally employs a "publish-subscribe" mechanism for message transmission. A typical scenario is shown in fig. 1. The components shown in fig. 1 include: the system comprises a theme flow, a channel, a publishing terminal and a subscribing terminal.
The topic flow describes the organizational form of the message. In the transmission communication component, each transmitted message must have a subject, and the messages under the same subject form a message subject flow. Messages within a subject stream are ordered and may be understood as a type of queue formed by the messages.
Data transmission needs to pass through a communication channel. According to data transmission among different machines, channels can be divided into four types, namely TCP, UDP broadcast, UDP unicast and UDP multicast. In addition to the above four communication channels, the shared memory may also be used as a data communication channel for data transmission in the same machine.
The publishing end, also known as the producer, is responsible for publishing a topic stream. After the application layer generates a new message, the publishing terminal puts the message into the corresponding theme flow and is responsible for sending the theme flow to the subscribing terminal through the communication channel.
The subscriber, also called a consumer, is responsible for receiving a topic stream. The subscribing terminal is responsible for transmitting the subscribed messages to the application layer according to the original publishing sequence of the subscribed messages, and generally needs to have the capability of detecting packet loss.
Due to the characteristics of low delay and high reliability of the financial transaction software, the transmission communication component in the financial transaction software is required to have the following characteristics:
(1) low latency. The delay is measured by the time for sending a message from the issuing end to the outside until the receiving end confirms that the message is successfully received, and since the financial transaction software is very sensitive to the delay in the system, the communication component is required to have lower delay.
(2) High reliability. For the message generated by the distribution end, it is necessary to ensure that the receiving end can receive the message in order and provide a packet loss retransmission mechanism.
There is a need in the art to develop a transmission communication device that meets the above-mentioned needs.
Disclosure of Invention
The following presents a simplified summary of one or more aspects in order to provide a basic understanding of such aspects. This summary is not an extensive overview of all contemplated aspects, and is intended to neither identify key or critical elements of all aspects nor delineate the scope of any or all aspects. Its sole purpose is to present some concepts of one or more aspects in a simplified form as a prelude to the more detailed description that is presented later.
The invention aims to solve the problems and provides a multilink transmission system, which can realize that a distribution end supports multichannel parallel distribution of a theme flow, improves throughput and reduces overall delay; message delay is reduced through mechanisms such as flow control, packet combination and the like.
The technical scheme of the invention is as follows: the invention discloses a multilink transmission system, which comprises a publishing terminal, a plurality of subscribing terminals, a plurality of channels and a theme flow, wherein:
the issuing end is used for executing an issuing task of a specified theme flow;
the channel is used for task publishing at a publishing terminal and message receiving at a subscribing terminal;
the topic flow message is stored in the topic flow and is published to a plurality of subscribing terminals through a plurality of channels in parallel;
the subscription end is used for receiving the theme flow messages through a plurality of channels and transmitting the received theme flow messages to the application layer for processing.
In an embodiment of the multilink transmission system according to the present invention, the types of channel support include, but are not limited to: and TCP, UDP broadcast, UDP unicast, UDP multicast and shared memory are supported.
According to an embodiment of the multilink transmission system of the present invention, an issuing session list is maintained in the issuing terminal, the issuing session list includes a plurality of issuing sessions, the issuing sessions in the issuing session list correspond to the channels one to one, wherein the issuing sessions are configured to monitor the subject stream messages on the respective corresponding channels, and issue the subject stream messages on the respective corresponding channels at the same time.
According to an embodiment of the multilink transmission system of the present invention, a subscription session list is maintained in the subscriber, the subscription session list includes a plurality of subscription sessions, the subscription sessions in the subscription session list correspond to channels one to one, wherein the subscription sessions are configured to process topic flow messages on the respective corresponding channels.
According to an embodiment of the multilink transmission system of the present invention, each subscription session runs in a different thread entity, one subscription topic stream is shared among the subscription sessions, the subscription sessions receive topic stream messages on each channel in parallel, and write the received topic stream messages into the shared subscription topic stream.
According to an embodiment of the multilink transmission system of the present invention, the multilink transmission system is configured to initiate a topic subscription request on a channel by a subscriber, after a publishing session on a channel corresponding to a publishing terminal monitors the topic subscription request, a new subscriber node is established according to a subscriber node number and a subscription starting position in the topic subscription request, the publishing session of the publishing terminal adds the newly established subscriber node to a current subscriber node list maintained by the publishing session, wherein the subscriber node list includes a plurality of subscriber nodes, and the subscriber node is a basic unit for message publishing and is used for storing messages of the subscriber; the publishing terminal also maintains a publishing identifier list, wherein the publishing identifier list is composed of a plurality of publishing identifiers, each publishing identifier corresponds to one subscribing terminal and is used for recording the publishing condition of the subscribing terminal in real time.
According to an embodiment of the multilink transmission system of the present invention, if a publishing session does not include any subscribing node, the publishing session is in a dormant state and does not publish data, and when a new subscribing node joins the publishing session, the publishing session is activated and begins to publish messages; the subscriber messages maintained by the subscriber node include, but are not limited to: the number of the subscriber end node, the current publishing position and the subscriber end publishing identification pointer are used for recording the publishing state corresponding to each message.
According to an embodiment of the multilink transmission system of the present invention, the multilink transmission system is configured to maintain a distribution topic stream in a distribution terminal, the distribution topic stream is a thread-safe message queue and is composed of a series of messages, each message in the distribution topic stream has a unique message number, and after a new message is generated by an application layer, the distribution terminal assigns a new message number to the new message and puts the new message into the distribution topic stream.
According to an embodiment of the multilink transmission system of the present invention, the multilink transmission system is further configured to execute a process flow of a publishing task, the publishing task is to publish a message to a channel, and the process flow is:
in a publishing task, a publishing session traverses each subscriber node to execute message publishing, and in the traversing process, for each subscriber node, a corresponding subscriber publishing identifier pointer and a current publishing position are taken out, and meanwhile, a sending buffer area is emptied;
if the sending buffer is not full, the publishing session continues to read messages in the stream according to the current publishing position: if the current release position is larger than the size of the theme flow, the message of the sending buffer area is sent and then the release task is finished; if the current publishing position is set to be published in the subscribing terminal publishing identification pointer, skipping the message; if the state of the current publishing position in the subscribing terminal publishing identification pointer is to be published, the message is put into a sending buffer area, and the state of the subscribing terminal publishing identification pointer is modified to be published, wherein before the message is added into the sending buffer area, a publishing packet header is added to the message by the publishing terminal, and the publishing packet header comprises a message sequence number, namely the current publishing position, a subject number and publishing terminal number information;
and if the sending buffer area is full, immediately sending the message in the sending buffer area to the channel, and finishing the task of the issuing.
According to an embodiment of the multilink transmission system of the present invention, each publishing session adjusts the size of the sending buffer in real time according to the flow control policy, and finds the maximum value of the transmission rate matching the publishing end and the subscribing end as the flow control.
According to an embodiment of the multilink transmission system of the present invention, before sending the data of the sending buffer to the corresponding channel, the publishing session of the publishing terminal merges a plurality of messages to be sent in the sending buffer into a data packet for one-time sending.
According to an embodiment of the multilink transmission system of the present invention, a subscription session of a subscriber receives a data packet from a channel, and then checks the data packet, and adds the data packet which is qualified by the check to a subscription topic stream, and if packet loss is checked, packet loss retransmission processing is performed, wherein the data packet check includes topic check and sequence number check, the topic check is to check whether a topic number in the received data packet is consistent with a subscribed topic number, and the sequence number check is to ensure the order of messages transmitted to an application layer.
According to an embodiment of the multilink transmission system of the present invention, the packet loss retransmission configuration is that the subscriber side sends a retransmission request to the publisher side after confirming the packet loss range, information of the retransmission request includes but is not limited to a topic number, a subscriber side number, and a data retransmission range, a publishing session of the publisher side analyzes the retransmission request after receiving the retransmission request, confirms the data retransmission range, then takes out a corresponding subscriber side node according to the subscriber side number in the retransmission request, sets a message state to be retransmitted in a subscriber side publishing identification pointer as unpublished, and finally preferentially reads and publishes the message to be retransmitted when reading the message from the topic stream in a publishing task.
Compared with the prior art, the invention has the following beneficial effects: the invention has the following innovation points:
(1) the high performance multilink reliable transport communication component supports a multi-channel parallel "publish-subscribe" mechanism. For a topic flow, a publishing terminal can publish data of the topic flow through a plurality of channels, and a receiving terminal can access the plurality of channels to receive topic flow messages. Providing a thread pool mechanism, wherein each issuing session of an issuing end can independently run in different thread entities to realize the parallel issuing of messages; similarly, each receiving session of the receiving end can also independently run in different thread entities, so as to realize the parallel receiving of the messages. For the environment with multiple network cards, a transmission channel can be allocated on each network card, the transmission capability of the multiple network cards is fully utilized, and the tasks of data release are distributed to each channel, so that the overall throughput is improved, and the message delay is reduced.
(2) And a subscription check mechanism, a packet loss retransmission mechanism and a heartbeat detection mechanism are provided to ensure the reliable transmission of data, and the message issuing rate is improved through a flow control and packet merging mechanism.
(3) The communication channel supports multiple mechanisms such as TCP, UDP broadcast, UDP unicast, UDP multicast, shared memory and the like, and the same theme flow can simultaneously use multiple channels of different types in parallel. The number and type of communication channels can be flexibly configured.
Drawings
The above features and advantages of the present disclosure will be better understood upon reading the detailed description of embodiments of the disclosure in conjunction with the following drawings. In the drawings, components are not necessarily drawn to scale, and components having similar relative characteristics or features may have the same or similar reference numerals.
Fig. 1 shows the principle of message transmission of communication middleware.
Fig. 2 shows a block diagram of an embodiment of the multilink transmission system of the present invention.
Fig. 3 is a block diagram showing the components of the publishing terminal and the subscribing terminal in an embodiment of the multilink transmission system of the present invention.
Fig. 4 shows a flow chart of an embodiment of the multilink transmission system of the present invention performing a single publishing task.
Detailed Description
The invention is described in detail below with reference to the figures and specific embodiments. It is noted that the aspects described below in connection with the figures and the specific embodiments are only exemplary and should not be construed as imposing any limitation on the scope of the present invention.
Fig. 2 shows the structure of an embodiment of the multilink transmission system of the present invention. Referring to fig. 2, the system of the present embodiment includes: the system comprises a publishing terminal, a plurality of subscribing terminals, a plurality of channels configured for the publishing terminal and a theme flow.
The publishing terminal is used for executing a publishing task of a specified theme flow. The subscription end is used for receiving the messages of the theme flow. The channels are used for task publishing at the publishing end, and support channel types such as TCP, UDP broadcast, UDP unicast, UDP multicast, shared memory and the like, and messages in tasks (hereinafter referred to as topic stream messages) are stored in the topic stream and are published to each subscribing end in parallel through a plurality of channels. And the subscriber receives the messages in the theme flow through one or more channels and transmits the messages confirmed to be received to the application layer for processing. The number of subscribers is not limited.
The internal components of the publisher and the subscriber are shown in fig. 3.
The publishing terminal maintains a publishing session (PubSession) list, and the list includes a plurality of publishing sessions, such as the publishing session1 and the publishing session2 shown in the figure. The number of publishing sessions in the list is consistent with the number of channels, one for each publishing session. The publishing sessions are configured to listen for the subject stream messages on the respective channel while publishing the subject stream messages to that channel. Each publishing session can run in different thread entities, a publishing subject stream object is shared among the publishing sessions, and the publishing sessions read messages in the publishing subject stream in parallel and send the messages to corresponding channels.
A subscription session (subscription) list is maintained in the subscriber, the subscription session list includes a plurality of subscription sessions, the number of the subscription sessions is equal to the number of channels accessed by the subscriber, each subscription session corresponds to one channel, and the subscription sessions are configured to process messages (i.e., data) on the respective corresponding channels. Each subscription conversation can run in different thread entities, one subscription topic flow is shared among the subscription conversations, topic flow messages on each channel are received in parallel by the subscription conversations, and the topic flow messages confirmed to be received are written into the shared subscription topic flow.
And after the subscription end is started, according to the configured channel address, sending a topic subscription request to the publishing end through each channel, wherein the topic subscription request is provided with an identifier of a subscription end node number. A list of subscribing nodes is maintained in each publishing session of the publisher, and the subscribing Node list is composed of a series of subscribing nodes (nodes). The method comprises the steps that a subscription end initiates a topic subscription request on a certain channel, after a publishing session on the channel corresponding to the publishing end monitors the topic subscription request, a new subscription end node is established according to a subscription end node number and a subscription starting position in the topic subscription request, and the newly established subscription end node is added into a current subscription end node list. In addition, the publishing terminal also maintains a publishing identifier list, the publishing identifier list is composed of a series of publishing identifiers (PubFlag), each publishing identifier corresponds to one subscribing terminal, and the publishing identifiers are used for recording the publishing condition of the subscribing terminal in real time.
In the example shown in FIG. 2, the publisher registers two channels, generating two publication sessions. There are three subscribers subscribing to the topic stream, so there are three publication identities (PubFlag) in the publication identity list of the publisher. For example, in fig. 2, the subscriber 1 only subscribes to channel 1, and the subscribers 2 and 3 both subscribe to channels 1 and 2. Therefore, the publishing session 1(PubSession1) includes 3 subscriber nodes (nodes), and the publishing session 2(PubSession2) includes 2 subscriber nodes (nodes).
The publishing terminal internally maintains a publish topic stream, which is a thread-safe message queue. In the published topic stream, each message has a unique message number (Seqno), which is incremented from 0. When the application layer generates a new message, the publishing terminal assigns a new Seqno to the newly generated message and puts the newly generated message into the publishing topic stream.
And each publishing session PubSession at the publishing terminal independently publishes the message. If a publish session PubSession does not contain any subscriber Node, it indicates that there is no subscriber on the channel, so the publish session PubSession will be in dormant state and no data publication will be performed. When a new subscribing end Node is added into the pubSession, the pubSession is activated to start message publishing.
The basic unit of message publishing is a subscriber Node, and the subscriber Node structure of the publishing terminal is used for storing the information related to the subscriber, and specifically includes: (1) subscribing the end node number SubID; (2) current release position Seqno: identifying the maximum message sequence number published to the subscriber; (3) the subscriber terminal issues an identifier PubFlag pointer: PubFlag is a multithreading read-write safe array, and is used to record the issue state corresponding to each message, including two states of "not issued" and "issued". The publisher maintains a PubFlag pointer for each subscriber, and the PubFlag pointer corresponding to the subscriber is stored in the subscriber Node.
The PubSession in the active state continues to execute the publishing task in each thread loop.
The execution flow of a single issue task of the multilink transmission system of the present embodiment is shown in fig. 4, and the issue task is a process of issuing a message (also referred to as data) onto a channel.
Within a publish task, a publish session PubSession traverses each of the subscribing end nodes to perform data publishing. And for each subscriber Node, taking out the corresponding subscriber publishing identifier PubFlag pointer and the current publishing position Seqno, and emptying the sending buffer area. Under the condition that the sending buffer is not full, the publishing session PubSession continues to read the message in the stream according to the current publishing position Seqno: if the current release position Seqno is larger than the size of the theme stream, the current release position Seqno indicates that the current release position Seqno is released to the latest position, and the release task can be ended after the message of the sending buffer area is sent; if the current publish position Seqno is already set to the "published" state in the publish flag PubFlag of the subscriber, it indicates that the other publish sessions PubSession have already published the message, so that the message can be skipped; if the state of the current publishing position Seqno in the publishing identifier PubFlag of the subscriber is "to be published", it indicates that the message has not been published yet, so that the message is put into the sending buffer, and the state of the publishing identifier PubFlag of the subscriber is modified to be "published". Before adding the message into the sending buffer area, the issuing end is responsible for adding an issuing header to the message, and the issuing header contains the message sequence number Seqno (namely the current issuing position), the subject number and issuing end number information. And if the sending buffer area is full, immediately sending the message in the sending buffer area to the channel, and finishing the task of the issuing.
The size of the sending buffer in the message publishing process is not fixed, and each publishing session PubSession needs to adjust the size of the sending buffer in real time according to a flow control strategy.
The main purpose of flow control is to find the maximum value of the transmission rate matched between the publishing terminal and the subscribing terminal, thereby ensuring the maximization of the transmission efficiency and ensuring that network congestion does not occur. The transmission rate is related to the channel performance, and the transmission rates of different channels are different, for example, if the channel a runs ten-gigabit network card and the channel B runs gigabit network card, the transmission rate of the channel a is ten times the transmission rate of the channel B. Considering that the transmission rate should not only identify the network capacity of the network channel, but also identify the message amount that the application layer program can process, the number of data packets transmitted per second is used as the calculation index of the rate.
The PubSession of the publishing terminal PubSession uses a slow start mode to find the optimal transmission rate: giving an initial rate, and if no packet loss event exists on the initial rate, upgrading the rate to 1.5 times at fixed time intervals; if a packet loss event occurs at the current rate, the current rate is reduced to 0.75 times.
After a sending buffer area in a PubSession of a publishing end is full, data needs to be sent to a corresponding channel. When the PubSession sends data to the TCP/UDP channel, the data transmission needs to be completed by calling a system call interface provided by Linux, and the more the execution times of the system call, the greater the delay of the data transmission. Therefore, when data is actually transmitted, before the publishing session PubSession executes system call, a plurality of messages to be transmitted in the transmission buffer are merged into one large data packet (no more than 1024 bytes), and the merged data packet is transmitted at one time, so that the system call frequency is reduced, and the message transmission delay is reduced.
The subscriber internally maintains a list of subscription sessions (subsessions) and a subscription topic stream. After each subscription session subscion receives a data packet from the accessed channel, the data packet is verified according to the head of the publishing packet, the data packet passing the verification is added into the subscription theme flow, and the application layer is informed that a new message can be processed. The data packet checking comprises two stages of subject checking and sequence number checking.
The purpose of the theme check is to check whether the theme number in the received data packet is consistent with the subscribed theme number, and if not, the data packet is directly discarded.
The purpose of sequence number verification is to ensure that messages transmitted to an application layer are strictly ordered, and if a packet loss event is detected, packet loss retransmission processing is required. Assuming that the sequence number of the currently received data packet is K, the maximum sequence number of the topic stream that the subscriber has confirmed to receive is N (the sequence number of the data packet expected to be received next is N + 1): (1) if K < N +1, it indicates that the current message has been acknowledged, directly discarding the current data packet (2), if K > N +1, then detecting a packet loss event, where the packet loss sequence number range is [ N +1, K-1], entering packet loss retransmission processing, and meanwhile, placing the currently received data packet into a receiving buffer by the subscription session SubSession (3), if K is N +1, it indicates that the currently received data packet is an expected data packet, adding the data packet into the subject stream by the subscription session SubSession, and detecting whether the data packet in the receiving buffer can be acknowledged.
If the channel type is UDP broadcast, UDP unicast, or UDP multicast, since the UDP protocol itself does not guarantee reliable transmission of data, the communication component is required to perform packet loss detection and retransmission. And if the subscription session of the subscription terminal detects a packet loss event, after confirming the packet loss range, sending a retransmission request to the publishing terminal. The retransmission request comprises information such as topic number, subscriber number, data retransmission range and the like.
After receiving the retransmission request, the publishing session PubSession at the publishing terminal is processed according to the following steps: (1) analyzing the retransmission request and confirming the retransmission range of the data; (2) according to the number of the subscription end in the retransmission request, taking out the corresponding subscription end Node, and setting the state of the message to be retransmitted as 'unpublished' in the publication identifier PubFlag of the subscription end; (3) in the publishing task, when reading the message from the theme flow, the message to be retransmitted is preferentially read and published.
While, for purposes of simplicity of explanation, the methodologies are shown and described as a series of acts, it is to be understood and appreciated that the methodologies are not limited by the order of acts, as some acts may, in accordance with one or more embodiments, occur in different orders and/or concurrently with other acts from that shown and described herein or not shown and described herein, as would be understood by one skilled in the art.
Those of skill would further appreciate that the various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
The various illustrative logical blocks, modules, and circuits described in connection with the embodiments disclosed herein may be implemented or performed with a general purpose processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. A general purpose processor may be a microprocessor, but in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. A processor may also be implemented as a combination of computing devices, e.g., a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration.
The steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor such the processor can read information from, and write information to, the storage medium. In the alternative, the storage medium may be integral to the processor. The processor and the storage medium may reside in an ASIC. The ASIC may reside in a user terminal. In the alternative, the processor and the storage medium may reside as discrete components in a user terminal.
In one or more exemplary embodiments, the functions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software as a computer program product, the functions may be stored on or transmitted over as one or more instructions or code on a computer-readable medium. Computer-readable media includes both computer storage media and communication media including any medium that facilitates transfer of a computer program from one place to another. A storage media may be any available media that can be accessed by a computer. By way of example, and not limitation, such computer-readable media can comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer. Any connection is properly termed a computer-readable medium. For example, if the software is transmitted from a web site, server, or other remote source using a coaxial cable, fiber optic cable, twisted pair, Digital Subscriber Line (DSL), or wireless technologies such as infrared, radio, and microwave, then the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of medium. Disk (disk) and disc (disc), as used herein, includes Compact Disc (CD), laser disc, optical disc, Digital Versatile Disc (DVD), floppy disk and blu-ray disc where disks (disks) usually reproduce data magnetically, while discs (discs) reproduce data optically with lasers. Combinations of the above should also be included within the scope of computer-readable media.
The previous description of the disclosure is provided to enable any person skilled in the art to make or use the disclosure. Various modifications to the disclosure will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other variations without departing from the spirit or scope of the disclosure. Thus, the disclosure is not intended to be limited to the examples and designs described herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (10)
1. A multilink transmission system, comprising a distribution end, a plurality of subscription ends, a plurality of channels and a theme flow, wherein:
the publishing terminal is used for executing a publishing task of a specified theme flow, wherein a publishing session list is maintained in the publishing terminal, the publishing session list comprises a plurality of publishing sessions, the publishing sessions in the publishing session list are in one-to-one correspondence with the channels, and the publishing sessions are configured to monitor theme flow messages on the respective corresponding channels and publish the theme flow messages to the respective corresponding channels;
the channel is used for task publishing at a publishing terminal and message receiving at a subscribing terminal;
the topic flow message is stored in the topic flow and is published to a plurality of subscribing terminals through a plurality of channels in parallel;
the subscription terminal is used for receiving the topic flow messages through a plurality of channels and transmitting the received topic flow messages to the application layer for processing, wherein a subscription session list is maintained in the subscription terminal, the subscription session list comprises a plurality of subscription sessions, the subscription sessions in the subscription session list are in one-to-one correspondence with the channels, and the subscription sessions are configured to process the topic flow messages on the corresponding channels; the subscription sessions run in different thread entities, share one subscription theme flow, receive theme flow messages on each channel in parallel and write the received theme flow messages into the shared subscription theme flow.
2. The multi-link transmission system according to claim 1, wherein the types of channel support include, but are not limited to: and TCP, UDP broadcast, UDP unicast, UDP multicast and shared memory are supported.
3. The multilink transmission system according to claim 1, wherein the multilink transmission system is configured such that a subscriber initiates a topic subscription request on a channel, after a publishing session on a channel corresponding to the publisher monitors the topic subscription request, a new subscriber node is created according to a subscriber node number and a subscription start position in the topic subscription request, the publishing session of the publisher adds the new created subscriber node to a current subscriber node list maintained by the publishing session, wherein the subscriber node list includes a plurality of subscriber nodes, and the subscriber node is a basic unit for message publishing and is used for storing messages of the subscriber; the publishing terminal also maintains a publishing identifier list, wherein the publishing identifier list is composed of a plurality of publishing identifiers, each publishing identifier corresponds to one subscribing terminal and is used for recording the publishing condition of the subscribing terminal in real time.
4. The multi-link transmission system according to claim 3, wherein if a publishing session does not include any subscribing node, the publishing session is dormant and does not publish data, and when a new subscribing node joins the publishing session, the publishing session is activated and begins to publish messages; the subscriber messages maintained by the subscriber node include, but are not limited to: the number of the subscriber end node, the current publishing position and the subscriber end publishing identification pointer are used for recording the publishing state corresponding to each message.
5. The multi-link transmission system according to claim 1, wherein the multi-link transmission system is configured to maintain a distribution topic stream in the distribution terminal, the distribution topic stream being a thread-safe message queue composed of a series of messages, each message having a unique message number in the distribution topic stream, and when a new message is generated by the application layer, the distribution terminal assigns a new message number to the new message and puts the new message into the distribution topic stream.
6. The multi-link transmission system according to claim 4, wherein the multi-link transmission system is further configured to execute a process of issuing a task, wherein the issue task is issuing a message to the channel, and the process comprises:
in a publishing task, a publishing session traverses each subscriber node to execute message publishing, and in the traversing process, for each subscriber node, a corresponding subscriber publishing identifier pointer and a current publishing position are taken out, and meanwhile, a sending buffer area is emptied;
if the sending buffer is not full, the publishing session continues to read messages in the stream according to the current publishing position: if the current release position is larger than the size of the theme flow, the message of the sending buffer area is sent and then the release task is finished; if the current publishing position is set to be published in the subscribing terminal publishing identification pointer, skipping the message; if the state of the current publishing position in the subscribing terminal publishing identification pointer is to be published, the message is put into a sending buffer area, and the state of the subscribing terminal publishing identification pointer is modified to be published, wherein before the message is added into the sending buffer area, a publishing packet header is added to the message by the publishing terminal, and the publishing packet header comprises a message sequence number, namely the current publishing position, a subject number and publishing terminal number information;
and if the sending buffer area is full, immediately sending the message in the sending buffer area to the channel, and finishing the task of the issuing.
7. The multilink transmission system according to claim 6, wherein each distribution session adjusts the size of the transmission buffer in real time according to the flow control policy, and finds the maximum value of the transmission rate at which the distribution end matches the subscriber end as the flow control.
8. The multilink transmission system according to claim 7, wherein the publishing session at the publishing end combines the messages to be sent in the sending buffer into a data packet for one-time transmission before sending the data in the sending buffer to the corresponding channel.
9. The multilink transmission system according to claim 8, wherein the subscription session of the subscriber checks the received data packet from the channel, and adds the qualified data packet to the subscription topic stream, and if the packet loss is checked, the retransmission processing of the packet loss is performed, wherein the data packet check includes a topic check for checking whether the topic number in the received data packet is consistent with the subscribed topic number, and the sequence number check is for ensuring the order of the message transmitted to the application layer.
10. The multilink transmission system according to claim 9, wherein the packet loss retransmission configuration is that the subscriber confirms a packet loss range and then sends a retransmission request to the publisher, information of the retransmission request includes but is not limited to a topic number, a subscriber number, and a data retransmission range, a publishing session of the publisher analyzes the retransmission request after receiving the retransmission request, confirms the data retransmission range, then takes out a corresponding subscriber node according to the subscriber number in the retransmission request, sets a message state to be retransmitted in a subscriber publishing identification pointer as not published, and finally preferentially reads and publishes a message to be retransmitted when reading the message from the topic stream in a publishing task.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911250991.8A CN111314217B (en) | 2019-12-09 | 2019-12-09 | Multilink transmission system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911250991.8A CN111314217B (en) | 2019-12-09 | 2019-12-09 | Multilink transmission system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111314217A CN111314217A (en) | 2020-06-19 |
| CN111314217B true CN111314217B (en) | 2021-12-17 |
Family
ID=71161257
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911250991.8A Active CN111314217B (en) | 2019-12-09 | 2019-12-09 | Multilink transmission system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111314217B (en) |
Families Citing this family (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112751943A (en) * | 2021-01-19 | 2021-05-04 | 国网江苏省电力有限公司无锡供电分公司 | Data communication method, intermediate device and data communication system |
| CN113193974B (en) * | 2021-07-02 | 2021-11-09 | 深圳华云信息系统有限公司 | Multicast-based market information pushing method, system, equipment and medium |
| CN113778674A (en) * | 2021-08-31 | 2021-12-10 | 上海弘积信息科技有限公司 | Lock-free implementation method of load balancing equipment configuration management under multi-core |
| CN114297109A (en) * | 2021-12-28 | 2022-04-08 | 中汽创智科技有限公司 | Data processing method and device based on subscription and publishing mode, electronic equipment and storage medium |
| CN115065733B (en) * | 2022-04-14 | 2024-04-16 | 中国银行股份有限公司 | Data publishing and subscribing system, data publishing method and data subscribing method |
| CN115190164B (en) * | 2022-06-24 | 2023-11-03 | 弥费科技(上海)股份有限公司 | Network communication method, device, computer equipment and storage medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103220328A (en) * | 2013-03-05 | 2013-07-24 | 迈普通信技术股份有限公司 | Data synchronous distribution method and device |
| CN103268567A (en) * | 2013-03-25 | 2013-08-28 | 中国科学院沈阳自动化研究所 | Manufacturing management system-oriented high-efficiency large-scale event detection and handling method |
| CN103561479A (en) * | 2013-10-17 | 2014-02-05 | 东软熙康健康科技有限公司 | Bluetooth device communication method and device |
| CN108306852A (en) * | 2017-12-05 | 2018-07-20 | 上海金融期货信息技术有限公司 | A kind of message-oriented middleware system and method based on simple binary coding |
| CN110121863A (en) * | 2016-09-23 | 2019-08-13 | 萨托里环球有限责任公司 | For providing the system and method for message to multiple subscribers |
| CN110266783A (en) * | 2019-06-13 | 2019-09-20 | 中国铁道科学研究院集团有限公司通信信号研究所 | A kind of railway CTC system communications platform based on DDS |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20180367583A1 (en) * | 2017-06-15 | 2018-12-20 | Satori Worldwide, Llc | System and method for providing a video stream and metadata over parallel channels |
| US10607012B2 (en) * | 2017-12-29 | 2020-03-31 | Delphian Systems, LLC | Bridge computing device control in local networks of interconnected devices |
-
2019
- 2019-12-09 CN CN201911250991.8A patent/CN111314217B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103220328A (en) * | 2013-03-05 | 2013-07-24 | 迈普通信技术股份有限公司 | Data synchronous distribution method and device |
| CN103268567A (en) * | 2013-03-25 | 2013-08-28 | 中国科学院沈阳自动化研究所 | Manufacturing management system-oriented high-efficiency large-scale event detection and handling method |
| CN103561479A (en) * | 2013-10-17 | 2014-02-05 | 东软熙康健康科技有限公司 | Bluetooth device communication method and device |
| CN110121863A (en) * | 2016-09-23 | 2019-08-13 | 萨托里环球有限责任公司 | For providing the system and method for message to multiple subscribers |
| CN108306852A (en) * | 2017-12-05 | 2018-07-20 | 上海金融期货信息技术有限公司 | A kind of message-oriented middleware system and method based on simple binary coding |
| CN110266783A (en) * | 2019-06-13 | 2019-09-20 | 中国铁道科学研究院集团有限公司通信信号研究所 | A kind of railway CTC system communications platform based on DDS |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111314217A (en) | 2020-06-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111314217B (en) | Multilink transmission system | |
| KR102047142B1 (en) | Transporting operations of arbitrary size over remote direct memory access | |
| CN110336702B (en) | System and implementation method of message middleware | |
| WO2019148563A1 (en) | Data sending method and apparatus, and data receiving method and apparatus | |
| US9843524B2 (en) | Priority data transmission using fibre channel over ethernet | |
| CN108306852B (en) | Message middleware system and method based on simple binary coding | |
| WO2019109513A1 (en) | Resource updating method and system, electronic device, and medium | |
| US20160330128A1 (en) | Queue scheduling method and device, and computer storage medium | |
| US10862767B2 (en) | Data packet prediction | |
| CN114338651A (en) | File transmission method and device, electronic equipment and readable storage medium | |
| US20140281019A1 (en) | Network Transmission Adjustment Based On Application-Provided Transmission Metadata | |
| CN113157465B (en) | Message sending method and device based on pointer linked list | |
| WO2019109902A1 (en) | Queue scheduling method and apparatus, communication device, and storage medium | |
| US9317678B2 (en) | System and method for managing logins in a network interface | |
| WO2017071430A1 (en) | Message processing method, network card, system, information update method, and server | |
| CN112398754A (en) | Data transmission method, device, medium, electronic equipment and network access equipment | |
| JP7074181B2 (en) | Control devices, communication control methods, and programs | |
| CN114186163A (en) | Application layer network data caching method | |
| CN112702624B (en) | Method, system, medium and device for optimizing short video playing efficiency | |
| CN113645283A (en) | Multilink communication method, device, storage medium and electronic equipment | |
| CN110290215B (en) | Signal transmission method and device | |
| CN114489730A (en) | Remote upgrading method, terminal equipment and computer readable storage medium | |
| CN111813795A (en) | Method and apparatus for confirming transactions in a blockchain network | |
| US9674282B2 (en) | Synchronizing SLM statuses of a plurality of appliances in a cluster | |
| WO2023226730A1 (en) | Data transmission method and apparatus, and storage medium and program product |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |