CN112988411A - Information transmission method, device and system - Google Patents
Information transmission method, device and system Download PDFInfo
- Publication number
- CN112988411A CN112988411A CN202110133176.4A CN202110133176A CN112988411A CN 112988411 A CN112988411 A CN 112988411A CN 202110133176 A CN202110133176 A CN 202110133176A CN 112988411 A CN112988411 A CN 112988411A
- Authority
- CN
- China
- Prior art keywords
- node
- information
- communication link
- message middleware
- consumption
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000005540 biological transmission Effects 0.000 title claims abstract description 118
- 238000000034 method Methods 0.000 title claims abstract description 56
- 230000006854 communication Effects 0.000 claims abstract description 108
- 238000004891 communication Methods 0.000 claims abstract description 108
- 238000004519 manufacturing process Methods 0.000 claims abstract description 58
- 230000007613 environmental effect Effects 0.000 claims abstract description 12
- 238000005192 partition Methods 0.000 claims description 80
- 238000004590 computer program Methods 0.000 claims description 9
- 238000012545 processing Methods 0.000 description 12
- 238000011161 development Methods 0.000 description 8
- 238000003860 storage Methods 0.000 description 7
- 238000010586 diagram Methods 0.000 description 6
- 238000006243 chemical reaction Methods 0.000 description 5
- 230000003993 interaction Effects 0.000 description 5
- 238000004422 calculation algorithm Methods 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 3
- 238000009826 distribution Methods 0.000 description 3
- VPNMENGBDCZKOE-LNYNQXPHSA-N [(1R,5S)-8-methyl-8-azabicyclo[3.2.1]octan-3-yl] 3-hydroxy-2-phenylpropanoate [(1S,2S,4R,5R)-9-methyl-3-oxa-9-azatricyclo[3.3.1.02,4]nonan-7-yl] (2S)-3-hydroxy-2-phenylpropanoate (1R,3S)-1,2,2-trimethylcyclopentane-1,3-dicarboxylic acid Chemical compound CC1(C)[C@H](CC[C@@]1(C)C(O)=O)C(O)=O.CC1(C)[C@H](CC[C@@]1(C)C(O)=O)C(O)=O.CN1[C@H]2CC[C@@H]1CC(C2)OC(=O)C(CO)c1ccccc1.CN1[C@H]2CC(C[C@@H]1[C@H]1O[C@@H]21)OC(=O)[C@H](CO)c1ccccc1 VPNMENGBDCZKOE-LNYNQXPHSA-N 0.000 description 2
- 230000007175 bidirectional communication Effects 0.000 description 2
- 230000003139 buffering effect Effects 0.000 description 2
- 238000005034 decoration Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 239000013307 optical fiber Substances 0.000 description 2
- 238000012216 screening Methods 0.000 description 2
- 238000012546 transfer Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000001502 supplementing effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/54—Interprogram communication
- G06F9/545—Interprogram communication where tasks reside in different layers, e.g. user- and kernel-space
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/54—Interprogram communication
- G06F9/544—Buffers; Shared memory; Pipes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/54—Interprogram communication
- G06F9/546—Message passing systems or structures, e.g. queues
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2209/00—Indexing scheme relating to G06F9/00
- G06F2209/54—Indexing scheme relating to G06F9/54
- G06F2209/547—Messaging middleware
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2209/00—Indexing scheme relating to G06F9/00
- G06F2209/54—Indexing scheme relating to G06F9/54
- G06F2209/548—Queue
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computer And Data Communications (AREA)
Abstract
The application discloses an information transmission method, a device and a system, wherein the method comprises the following steps: acquiring current environmental parameters in real time; when the current environmental parameters meet first preset conditions, controlling each node in a node cluster to carry out end-to-end information transmission through a communication link between each node, wherein each node at least comprises a production node and a consumption node; and when the current environment parameters do not meet the first preset condition, switching each node to a communication link with a message middleware, and transmitting information among the nodes through the message middleware.
Description
Technical Field
The present application relates to the field of data processing technologies, and in particular, to an information transmission method, apparatus, and system.
Background
In the process of application development based on message driving, message production and consumption are generally required to be realized through a plurality of production nodes, a plurality of consumption nodes and a message queue so as to complete the application development. The production node is responsible for sending messages to the message queue, the consumption node is a node for acquiring messages from the message queue, and the message queue is message middleware for storing information. Because the message queue can be used for storing information, the normal operation of the background program can be ensured through the buffer memory of the middle message queue regardless of the rapid speed of the information generated by a producer. However, when the system performs application development based on message driving, since messages all need to be buffered through the intermediate message queue, the processing mode is not flexible enough, and the system is only suitable for services requiring integrated heavyweight centralized services and cannot meet service requirements under different environments.
Disclosure of Invention
The present application aims to solve at least one of the technical problems in the prior art, and provides an information transmission method, an information transmission device, and an information transmission system, which can improve the service processing efficiency while meeting the service requirements at any time.
The embodiment of the application provides an information transmission method, which comprises the following steps:
acquiring current environmental parameters in real time;
when the current environmental parameters meet first preset conditions, controlling each node in a node cluster to carry out end-to-end information transmission through a communication link between each node, wherein each node at least comprises a production node and a consumption node;
and when the current environment parameters do not meet the first preset condition, switching each node to a communication link with a message middleware, and transmitting information among the nodes through the message middleware.
Further, the end-to-end information transmission performed by each node in the control node cluster through a communication link between each node includes:
and controlling the production node to respectively provide corresponding information for each consumption node according to the partition to which the communication link between the production node and each consumption node belongs.
Further, the partition is determined by a hash partition policy or a Round-robin partition policy.
Further, the production node comprises a first node, and the consumption node comprises a second node and a third node;
each node in the control node cluster performs end-to-end information transmission through a communication link between each node, and the method comprises the following steps:
and controlling the second node to receive original information sent by the first node through a communication link with the first node so that the second node sends subscription information generated after consuming the original information to the third node, wherein the original information is generated by the first node according to a consumption request of the third node.
Further, the sending, by the second node, the subscription information generated after the consumption of the original information to the third node includes:
and controlling the second node to receive the consumption request of the third node through a communication link with the third node, so that the second node consumes the original information according to the matching result of the consumption request and the original information, generates the subscription information and sends the subscription information to the third node.
Further, the switching each node to a communication link with a message middleware, and performing information transmission between the nodes through the message middleware, includes:
after each node is switched to a communication link with a message middleware, receiving information provided by the production node, and marking the information received from the production node as cache information;
and detecting information existing in the message middleware, and sending the cache information to the message middleware when detecting that the message middleware does not have the same information as the cache information, so that the message middleware forwards the cache information to the corresponding consumption node.
Further, a communication protocol between the message middleware and each of the nodes is RSocket.
Further, the API used by each of the nodes is the same.
Further, in an embodiment of the present application, there is provided an information transmission apparatus including:
the data acquisition module is used for acquiring current environmental parameters in real time;
a first control module, configured to control, when the current environment parameter meets a first preset condition, each node in a node cluster to perform end-to-end information transmission through a communication link between the nodes, where each node at least includes a production node and a consumption node;
and the second control module is used for switching each node to a communication link with a message middleware when the current environment parameter does not meet the first preset condition, and transmitting information among the nodes through the message middleware.
Further, an embodiment of the present application provides an electronic device, including: a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the information transmission method as described in the above embodiments when executing the program.
Further, in an embodiment of the present application, there is provided an information transmission system, including a plurality of nodes, a message middleware, and the electronic device according to the above embodiment, where each of the nodes includes at least a production node and a consumption node;
each node is used for carrying out end-to-end information transmission through a communication link between the nodes when receiving a first control signal of the electronic equipment; and the number of the first and second groups,
and when receiving a second control signal of the electronic equipment, switching to a communication link with a message middleware, and transmitting information between the nodes through the message middleware.
Further, the present application provides a computer-readable storage medium, which stores computer-executable instructions for causing a computer to execute the information transmission method according to the above embodiment.
Compared with the prior art, the embodiment obtains the current environment parameters in real time, judges whether the current environment parameters meet the first preset condition, at least controls the generation node and the consumption node to transmit information through the link between the nodes when the current environment parameters meet the first preset condition, and controls each node to realize message transmission between the nodes through the communication link with the message middleware when the current environment parameters do not meet the first preset condition, so that the problem that all messages need to be cached through a message queue in the application development process to cause an inflexible processing mode can be avoided, the centralized transmission of the message middleware and the switching of the end-to-end transmission of the nodes are realized, and the service requirements under different environments can be met.
In the above embodiment, the control production node provides the information corresponding to the partition to the consumption node according to the partition to which the communication link between the production node and the consumption node belongs, so as to provide the information corresponding to the communication link as much as possible, which can meet the consumption requirement of the consumption node and improve the information transmission efficiency.
The above embodiment determines the partition to which the communication link belongs by using a hash partition policy or a Round-robin partition policy, which can ensure that data is evenly distributed in a predetermined number of partitions, or partition distribution is uniform when subscription information of the partitions is the same.
In the above embodiment, the second node is controlled to receive the original information sent by the first node and generated according to the consumption request of the third node, and to consume the received original information to generate the subscription information, and then to send the subscription information to the third node, so that the information received by the third node is filtered by the second node, and the information which is desired to be processed to a certain extent can be received, thereby satisfying the consumption requirement of the third node as the consumption node.
In the embodiment, the second node is controlled to receive the consumption request of the third node, so that redundant information which is not matched with the consumption request in the original information and information which is matched with the consumption request are determined, one of the information is consumed according to the consumption request of the third node, subscription information is generated, the redundant information which is filtered by the original information can be effectively consumed or the information which is generated according to the consumption request is accurately determined, and therefore the influence on message transmission of other nodes is reduced.
In the embodiment, the information provided by the production node is received and marked as the cache information, whether the information with the same cache information exists in the message middleware is detected, if not, the cache information is sent to the message middleware to be forwarded to the consumption node, and if so, the message middleware is controlled to be directly sent to the consumption node, so that the information load of the message middleware is reduced to the maximum extent, and the information transmission efficiency under different conditions is accelerated.
In the above embodiment, RSocket is used as a communication protocol between the message middleware and each node, so that the unified application programming interfaces in the message middleware and each node may use different underlying transport layers and have different interaction modes, thereby better avoiding occurrence of transmission failure due to differences between the message middleware and the node or differences between different nodes.
In the above embodiment, each node uses the same API, which can improve the message transmission efficiency between nodes and does not need multiple devices.
Drawings
The present application is further described with reference to the following figures and examples;
FIG. 1 is a diagram of an exemplary embodiment of a method for transferring information;
FIG. 2 is a flow chart illustrating a method of information transfer according to an embodiment;
FIG. 3 is a schematic diagram of an embodiment of an information transmission apparatus;
FIG. 4 is a block diagram of a computer device in one embodiment;
fig. 5 is a schematic structural diagram of an information transmission system in one embodiment.
Detailed Description
Reference will now be made in detail to the present embodiments of the present application, preferred embodiments of which are illustrated in the accompanying drawings, which are for the purpose of visually supplementing the description with figures and detailed description, so as to enable a person skilled in the art to visually and visually understand each and every feature and technical solution of the present application, but not to limit the scope of the present application.
In the process of application development based on message driving, message production and consumption are generally required to be realized through a plurality of production nodes, a plurality of consumption nodes and a message queue so as to complete the application development. The production node is responsible for sending messages to the message queue, the consumption node is a node for acquiring messages from the message queue, and the message queue is message middleware for storing information. Because the message queue can be used for storing information, the normal operation of the background program can be ensured through the buffer memory of the middle message queue regardless of the rapid speed of the information generated by a producer. However, when the system performs application development based on message driving, since messages all need to be buffered through the intermediate message queue, the processing mode is not flexible enough, and the system is only suitable for services requiring integrated heavyweight centralized services and cannot meet service requirements under different environments. The production nodes are computers or computer clusters for generating data or information, and the consumption nodes are computers or computer clusters for processing data or information. The message queue is a message middleware storing data or information, and the message middleware is a computer or a computer cluster. When the information provided by the producer is cached through the intermediate message queue, the service requirements under different environments have different bias degrees to the intermediate message queue, and thus the method can not be suitable for the service requirements under different environments.
To solve the above technical problem, as shown in fig. 1, it is an application environment diagram of an information transmission method in an embodiment. Referring to fig. 1, the information transmission system includes an information transmission apparatus 110, a node 120, and message middleware 130. The information transmission device 110 and the node 120 are connected through a network, the information transmission device 110 and the message middleware 130 are connected through a network, and the node 120 and the message middleware 130 are connected through a network. The information transmission device 110 may be implemented by a stand-alone computer or a computer cluster composed of a plurality of computers. Node 120 and message middleware 130 may each be implemented as separate servers or as a cluster of servers. Wherein the nodes 120 include production nodes 120 and consumption nodes 120. The information transmission device 110 obtains the current environmental parameter in real time, and determines whether the current environmental parameter meets a first preset condition. When the current environment parameter meets the first preset condition, the information transmission device 110 at least controls the generating node 120 and the consuming node 120 to perform information transmission through the link between the nodes. When the current environment parameter does not meet the first preset condition, the information transmission device 110 controls each node 120 to realize message transmission among the nodes through a communication link with the message middleware 130.
Hereinafter, the information transmission method provided by the embodiments of the present application will be described and explained in detail by several specific embodiments.
In one embodiment, as shown in fig. 2, an information transmission method is provided. The embodiment is mainly illustrated by applying the method to computer equipment. The computer device may specifically be the information transmission apparatus 110 in fig. 1 described above.
Referring to fig. 2, the information transmission method specifically includes the following steps:
and S11, acquiring the current environment parameters in real time.
In this embodiment, the information transmission apparatus obtains a current environment parameter in real time, where the current environment parameter is used to determine a service requirement in an environment to which the current information transmission system belongs, and the current environment parameter may be in the form of one or more parameter items to determine the service requirement in the current environment to which the current environment belongs, for example, a requirement of 0 indicates that the service requirement in the current environment to which the current environment belongs does not require an intermediate messaging element, and a requirement of 1 indicates that the service requirement in the current environment to which the current environment belongs requires an intermediate messaging element; or, the 2 parameter items environment and require indicate that the service requirement under the current environment must be the middle messaging piece when simultaneously being 1, indicate that the service requirement under the current environment does not need the middle messaging piece when simultaneously being 0, and indicate that the service requirement under the current environment may not need the middle messaging piece when one is 1 and the other is 0. The current environment parameter may also directly adopt an identifier form to determine the service requirement in the current environment, for example, when the information transmission system has an identifier, such as require-1, it indicates that the service requirement in the current environment needs an intermediate message. When the identifier is not identified, the service requirement under the current environment is represented without the need of the middleware. The current environment parameter may also directly adopt a data form to determine the service requirement in the current environment, for example, the current environment parameter is 3737, and whether the service requirement in the current environment of the information transmission system needs the message middleware is determined according to whether the current environment parameter is in a data range where the information transmission system needs the message middleware.
In this embodiment, the current environment parameter may be stored in the message middleware, may be stored in one or more nodes, and may be input into the information transmission apparatus, the message middleware, or the node by the user. When the current environment parameters are stored in the message middleware or the nodes and the service requirements of the environment to which the current information transmission system belongs are changed, the current user parameters can be changed in an automatic matching mode or a manual user modifying mode. In addition, the real-time acquisition mode of the current environment parameters can be acquired at intervals through the information transmission device, or the current environment parameters in each time unit in a time period are acquired, and the current environment parameters in the last time unit are used as the current environment parameters of the time period.
And S12, when the current environment parameter meets a first preset condition, controlling each node in the node cluster to perform end-to-end information transmission through a communication link between each node, wherein each node at least comprises a production node and a consumption node.
In this embodiment, the information transmission device determines whether the current environment parameter meets a first preset condition, where the first preset condition may be set by a user, or the information transmission device may automatically generate the first preset condition. When the current environment parameter accords with a first preset condition, the information transmission device controls each node to carry out end-to-end information transmission through a communication link between the nodes, wherein each node at least comprises a production node and a consumption node. The production nodes are computers or computer clusters that generate data or information, and the consumption nodes are computers or computer clusters that process data or information. A communication link is a physical channel between two nodes, and the transmission medium of the communication link may be twisted pair, optical fiber, or microwave. And the communication link may be a unidirectional communication link or a bidirectional communication link.
In this embodiment, the production node and the consumption node may be the same node, for example, both the node1 and the node 2 may be the production node and the consumption node. The node1 and the node 2 carry out end-to-end information transmission through a communication link between the two nodes, when the node1 is a production node, the node 2 is a consumption node, the node1 transmits information to the node 2 through the communication link between the node1 and the node 2, and the node 2 processes the received information; when the node1 is a consuming node, the node 2 is a producing node, the node 2 transmits information to the node1 through a communication link with the node1, and the node1 processes the received information. The node1 can receive the information transmitted by the node 2 through a communication link with the node 2 and process the information, simultaneously transmit the information to the node 2, and the node 2 transmits the information to the node1 and simultaneously receives the information transmitted by the node1 and processes the received information.
In one embodiment, controlling each node in the node cluster to perform end-to-end information transmission through a communication link between each node includes:
and controlling the production node to respectively provide corresponding information for each consumption node according to the partition to which the communication link between the production node and each consumption node belongs.
In this embodiment, the information transmission apparatus controls the production node to provide the information corresponding to the partition to which the communication link between the production node and each of the consumption nodes belongs, to each of the consumption nodes.
In the embodiment, in order to better store, manage or query the information provided by the production node, the information may be divided into different categories according to the information type, the information storage address, the information size or the information content, and each category corresponds to a different partition. The partitions to which the communication links correspond refer to different ranges of information that are allocated to the communication links to enable the communication links between the production nodes and the consumption nodes to efficiently handle or better meet the needs of the consumption nodes. For example, the information provided by the node1 (production node) includes the recharge information of any game product, wherein the recharge information is divided into category A, category B, category C and category D according to the information size, and the information sizes corresponding to the 4 categories are 0-1000, 2100-. The partition comprises a partition a, a partition b and a partition c, and the information ranges are 0-1000, 1001-2000 and 2001-3000 respectively. The information of the partition a can be efficiently processed for the communication link between the node1 (production node) and the node 2 (consumption node), so the node1 provides the node 2 with the information of the category A in the recharge information of any money game product according to the information range of the partition a. For the communication link between node1 (production node) and node 3 (consumption node), node 3 wants to process the information of partition c, so node1 provides node 3 with the information of class B in the top-up information of any money game product according to the information range of partition c.
In this embodiment, the production node is controlled to provide the information corresponding to the partition to the consumption node according to the partition to which the communication link between the production node and the consumption node belongs, so as to provide the information corresponding to the communication link as much as possible, thereby meeting the consumption requirement of the consumption node and improving the information transmission efficiency.
In one embodiment, the partitions are determined by a hash partition policy or a Round-robin partition policy.
In this embodiment, the information corresponding to the partition is determined by a hash partition policy, and the partition corresponding to the communication link may be set manually or matched automatically. And distributing each piece of information to the corresponding partition determined according to the hash value, and providing the information in the partition to the corresponding communication link by the first node according to the partition matched with the communication link.
In this embodiment, the hash value of the information may be calculated by a hash algorithm, where the hash algorithm may be MD4, MD5, SHA-1, or the like. The selection of the hash value calculation method and the hash algorithm is common, and is not described in detail in this embodiment.
After the hash value is obtained through calculation, a plurality of corresponding partitions can be determined in a hash mode, a consistency hash mode and the like. For example, the hash scheme is adopted, the first node has three partitions in total, namely area0, area1 and area2, the information has five, the hash values obtained by calculation are 247, 760, 903, 533 and 302, the number N of partitions is obtained by subtracting the calculated hash value from 3, and the remainder indicates the partition in which the information corresponding to the hash value is located. Therefore, it can be seen that there is information corresponding to the hash value 903 in area0, information corresponding to 247 and 760 in area1, and information corresponding to 533 and 302 in area 2.
In this embodiment, the information corresponding to the partition is determined by a Round-robin partition policy, and the partition corresponding to the communication link may be set manually or matched automatically. And distributing each piece of information to the corresponding partition in a polling mode, and providing the information in the partition to the corresponding communication link by the first node according to the partition matched with the communication link. If the information corresponding to the partitions is the same, the partition allocation of the Round-robin policy will be uniform. For example, there are 2 partitions C0 and C1, which correspond to topics t0 and t1, and each topic has 3 pieces of information, then all the corresponding information can be identified as: t0p0, t0p1, t0p2, t1p0, t1p1, t1p 2. The final distribution result is: partition C0: t0p0, t0p2, t1p 1; partition C1: t0p1, t1p0, t1p 2.
If the information corresponding to the partitions is different, the allocation of the partitions is not completely round-robin when the allocation of the partitions is performed, which may result in uneven allocation of the partitions. If a partition does not correspond to a topic, then the partition will not have any information for that topic assigned at the time the partition is assigned.
For example, there are 3 partitions, C0, C1, and C2, which subscribe to 3 topics in total: t0, t1 and t2, wherein the 3 themes respectively have 1, 2 and 3 information, that is, 3 partitions correspond to 6 partitions of t0p0, t1p0, t1p1, t2p0, t2p1 and t2p 2. Specifically, partition C0 corresponds to topic t0, partition C1 corresponds to topics t0 and t1, and partition C2 corresponds to topics t0, t1, and t2, so the final assignment result is: partition C0: t0p 0; partition C1: t1p 0; partition C2: t1p1, t2p0, t2p1, t2p 2.
In this embodiment, the hash partition policy or Round-robin partition policy is used to determine the partition to which the communication link belongs, so that it can be ensured that data is evenly distributed in a predetermined number of partitions, or partition distribution is uniform when the subscription information of the partitions is the same.
In one embodiment, the production node comprises a first node and the consumption node comprises a second node and a third node.
The control node cluster is characterized in that each node performs end-to-end information transmission through a communication link between each node, and the method comprises the following steps:
and controlling the second node to receive the original information sent by the first node through a communication link with the first node so that the second node sends subscription information generated after consuming the original information to the third node, wherein the original information is generated by the first node according to a consumption request of the third node.
In this embodiment, each node may be a production node or a consumption node, and therefore, when the first node is node1 in fig. 1, the second node may be node 2 or node 3, and correspondingly, the third node may be node 3 or node 2. When the first node is node 2 in fig. 1, the second node may be node1 or node 3, and correspondingly, the third node may be node 3 or node 1. When the first node is node 3 in fig. 1, the second node may be node1 or node 2, and correspondingly, the third node may be node 2 or node 1.
In this embodiment, the first node generates original information according to a consumption request of the third node, where the original information may include only the original information generated according to the consumption request of the third node, or further include information generated according to consumption requests of other nodes, or further include redundant information that is adulterated. The consumption request is information to be consumed by the consumption node, and may specifically define information content, information type, information size, and the like. The consumption request of the third node can be set manually or generated by matching the nodes. The original information generated according to the consumption request of the third node can be encrypted or subjected to format conversion or screening processing according to the information in the consumption request. For example, the first node has the login data and the recharge data of any money game product, and the consumption request of the third node is that the data content is the login data, at this time, the first node screens out the login data of any money game product as the original information to be provided to the communication link between the first node and the second node.
The first node transmits the original information to the second node through a communication link between the first node and the second node after generating the original information, the second node consumes the original information after receiving the original information to generate subscription information, wherein the original information in the information is consumed, namely the original information is processed to generate the subscription information, and the processing mode can be encryption, format conversion, content simplification and the like. For example, the first node provides original information to the second node through a communication link between the first node and the second node, the original information includes data 1-100, the original information generated according to the consumption request of the third node is also data 1-100, at this time, the second node performs format conversion on the data 1-100, converts the data 1-100 into a hexadecimal representation form, obtains subscription information, and sends the subscription information to the third node through the communication link between the second node and the third node.
In this embodiment, the second node is controlled to receive the original information sent by the first node and generated according to the consumption request of the third node, and to consume the received original information to generate subscription information, and then to send the subscription information to the third node, so that the information received by the third node is filtered by the second node, and the desired information after certain processing can be received, thereby satisfying the consumption requirement of the third node as a consumption node.
In one embodiment, the causing the second node to send subscription information generated after consuming the original information to the third node includes:
and controlling the second node to receive the consumption request of the third node through a communication link with the third node so that the second node consumes the original information according to the matching result of the consumption request and the original information, generates subscription information and sends the subscription information to the third node.
In this embodiment, the first node generates original information according to a consumption request of the third node, where the original information may include only the original information generated according to the consumption request of the third node, or further include information generated according to consumption requests of other nodes, or further include redundant information that is adulterated. The consumption request is information to be consumed by the node, and may specifically define information content, information type, information size, and the like. The consumption request of the third node can be set manually or generated by matching the nodes. The original information generated according to the consumption request of the third node can be encrypted or subjected to format conversion or screening processing according to the information in the consumption request. For example, the first node has the login data and the recharge data of any money game product, the consumption request of the third node is the data content of the login data, and at the moment, the first node screens out the login data of any money game product as original information to be provided to a communication link between the first node and the second node.
After the first node generates the original information, the information transmission device controls the first node to transmit the original information to the second node through a communication link between the first node and the second node, the second node matches the received original information sent by the first node according to the received consumption request of the third node, and the second node consumes the original information according to the matching result of the message request and the original information. The second node is matched with the original information according to the consumption request, the original information is consumed according to the matching node, information generated according to the third node consumption request in the original information sent by the first node can be specifically determined through matching, and then redundant information except the information generated according to the third node consumption request is determined, so that the second node consumes the redundant information. And determining information generated according to the consumption request of the third node in the original information sent by the first node through matching, so that the second node consumes the information generated according to the consumption request of the third node. The matching manner of the second node according to the consumption request and the information generated according to the third node consumption request may be that the information generated according to the third node consumption request includes an identifier of the third node consumption request, for example, the original information is login data 1-100, the information generated according to the third node consumption request is data 51-100, each data is 51-ysczg, …, 100-ysczg in format, and the consumption request is login data of a song of a city on the cloud, so that the second node can recognize the login data 51-100 of the song of the city on the cloud as the information generated according to the consumption request from the login data 1-100.
In this embodiment, the second node consumes the redundant information in the original information, that is, consumes other information in the original information except the information generated according to the third node consumption request, and may convert or not process the remaining information generated according to the third node consumption request after consumption to generate the subscription information, where the second node consumes the redundant information, that is, filters the original information, consumes the redundant information to obtain the information generated according to the third node consumption request, specifically processes the redundant information except the information generated according to the third node consumption request, and converts or does not process the information generated according to the third node consumption request, that is, generates the subscription information. For example, the first node provides original information to the second node through a communication link between the first node and the second node, the original information comprises data 1-100, and information generated according to a consumption request of the third node is data 51-100, so that redundant information of the data 1-50 is doped in the original information, at this time, the second node consumes 1-50 parts of the data 1-100 in the original information, generates the remaining information 51-100 as subscription information after consumption, and sends the subscription information to the third node through a third channel.
And the second node consumes the information generated according to the third node consumption request, specifically, after the second node determines the information generated according to the third node consumption request through the matching node, the second node consumes the information generated according to the third node consumption request and generates subscription information after consumption, wherein the mode of generating the subscription information by consumption can be the processing of encrypting or format converting the information. For example, the first node provides original information to the second node through a communication link between the first node and the second node, the original information comprises data 1-100, and information generated according to a consumption request of the third node is data 51-100, so that redundant information of the data 1-50 is doped in the original information, the second node determines that 51-100 parts in the original information 1-100 are information generated according to the consumption request according to a matching result, at the moment, the second node performs format conversion on the data 51-100, converts the data 51-100 into a hexadecimal representation form to obtain subscription information, and sends the subscription information to the third node through the communication link between the second node and the third node.
In this embodiment, the second node is controlled to receive the consumption request of the third node, so that redundant information which is not matched with the consumption request in the original information and information which is matched with the consumption request are determined, one of the information is consumed according to the consumption request of the third node, and subscription information is generated, so that redundant information which is filtered from the original information can be effectively consumed or information which is generated according to the consumption request is accurately determined, and the influence on message transmission of other nodes is reduced.
And S13, when the current environment parameters do not meet the first preset condition, switching each node to a communication link with the message middleware, and transmitting information among the nodes through the message middleware.
In this embodiment, the information transmission apparatus determines whether the current environmental parameter meets a first preset condition. When the current environment parameters do not meet the first preset condition, the information transmission device controls each node to disconnect a communication link between the nodes, and controls each node to transmit information with the message middleware through the communication link between the node and the message middleware. When one or more of the nodes and the message middleware do not have a communication link, if the current environment parameters do not meet the first preset condition, the information transmission device controls the corresponding one or more of the nodes to establish the communication link with the message middleware; when a communication link exists between each node and the message middleware, when the current environment parameter does not meet the first preset condition, the information transmission device controls each node to stop transmitting information between each node through the communication link between each node and controls each node to transmit information with the message middleware through the communication link between each node and the message middleware, so that the production node provides the information to the message middleware, and the message middleware forwards the received information to the consumption node.
The communication link between each node and the message middleware is a physical channel, and the transmission medium of the communication link can be a twisted pair, an optical fiber or a microwave. And the communication links between the nodes and the message middleware may be unidirectional communication links or bidirectional communication links.
In one embodiment, switching each node to a communication link with a message middleware, and performing information transmission between each node through the message middleware, includes:
and after each node is switched to a communication link with the message middleware, receiving the information provided by the production node, and marking the information received from the production node as cache information.
And detecting information existing in the message middleware, and sending the cache information to the message middleware when detecting that the message middleware does not have the same information as the cache information, so that the message middleware forwards the cache information to the corresponding consumption node.
In this embodiment, the information transmission apparatus receives information provided by the production node, and marks the information provided by the generation node as cache information, where the marking may be performed by adding a prefix or a suffix to the information, or adding a specific identifier to the information, or adding an identifier to an address or a folder where the information is located.
The information transmission device detects the information existing in the message middleware, wherein the information existing in the message middleware can be the information stored in the message middleware, or the information body is stored in another computer or computer cluster, and the message middleware stores the information of the identification or address of the information. The information transmission apparatus determines whether there is information identical to the buffering information in the message middleware by comparing the buffering information with information stored in the message middleware. In this embodiment, the comparison method for determining whether the information is the same is not particularly limited.
In this embodiment, when the information transmission apparatus detects that the message middleware has the same information as the cache information, the information transmission apparatus sends an instruction to the message middleware to make the message middleware transmit the same information as the cache information to the corresponding consumption node. Wherein the instruction may include the content of the cached information or the identifier of the same information as the cached information stored in the message middleware, for example, the cached information is login data of song of city on the cloud, and the identifier is login data-ysczg in the message middleware, then the information transmission apparatus sends the instruction "send-login data-ysczg-node 1" to the message middleware, that is, the message middleware transmits the stored information identified as login data-ysczg to the node 1. Or sending an instruction 'send-login data of the song of city on cloud', and enabling the message middleware to transmit the same information as the login data of the song of city on cloud in the message middleware to the corresponding consumption node.
When the information transmission device detects that the cache information does not exist in the message middleware, the information transmission device transmits the received cache information to the message middleware, wherein the information transmission device performs information transmission through a communication link between the information transmission device and the message middleware, and the communication link between the information transmission device and the message middleware can be a one-way communication link or a two-way communication link. And after receiving the cache information, the message middleware forwards the cache information to the corresponding consumption node through a communication link between the message middleware and the node. The corresponding relationship between the cache information and the consumption node may be that the cache information includes an identifier of the consumption node, or the cache information includes an identifier of a specific communication link.
In one embodiment, the communication protocol between the message middleware and each node is RSocket.
RSocket is 5/6-layer protocol in OSL seven-layer model, and is application layer protocol above TCP/IP. RSocket may use different underlying transport layers including TCP, WebSocket, and Aeron. TCP is suitable for interaction among all components of a distributed system, WebSocket is suitable for interaction between a browser and a server, and Aeron is a transmission mode based on a UDP protocol, so that RSocket can be suitable for different scenes. The application layer implementation using RSocket can be kept unchanged, and only a proper bottom layer transmission mode needs to be selected according to the system environment, the device capability and the performance requirement. RSocket, as an application layer protocol, can easily define its own protocol on its basis. In addition, the RSocket uses a binary format, so that the transmission efficiency is ensured, and the bandwidth is saved. Moreover, through flow control based on reactive flow semantics, RSocket ensures that both parties in message transmission do not crash due to excessive pressure on requests, and can model all interactions in an application as network primitives, i.e., can stream data or perform publish/subscribe, without setting up application queues.
In this embodiment, RSocket is used as the communication protocol between the message middleware and each node.
In this embodiment, the communication protocol between the message middleware and each node adopts RSocket, so that the unified application programming interfaces in the message middleware and each node can use different underlying transport layers and have different interaction modes, thereby better avoiding the occurrence of transmission failure caused by differences between the message middleware and the node or differences between different nodes.
In one embodiment, the API employed by each node is the same.
Among them, the API (Application Programming Interface) is some predefined interfaces (such as function and HTTP Interface), or refers to the convention for linking different components of the software system. In this embodiment, the first node, the second node, and the third node are all computers or computer clusters, so the specific form of the API is the prior art, and is not described herein again. And the API is not particularly limited in this embodiment.
In this embodiment, each node uses the same API, which can improve the message passing efficiency between nodes and does not need multiple devices.
In the above embodiment, the current environment parameter is obtained in real time, whether the current environment parameter meets a first preset condition is judged, when the current environment parameter meets the first preset condition, at least the generating node and the consuming node are controlled to transmit information through a link between the nodes, and when the current environment parameter does not meet the first preset condition, each node is controlled to transmit the information between the nodes through a communication link with the message middleware, so that the problem that all messages need to be cached through a message queue in an application development process, the processing mode is not flexible is avoided, the centralized transmission of the message middleware and the switching of the node end-to-end transmission are realized, and the service requirements under different environments can be met.
In one embodiment, as shown in fig. 3, there is provided an information transmission apparatus including:
and the data acquisition module 101 is configured to acquire the current environmental parameter in real time.
The first control module 102 is configured to control, when the current environment parameter meets a first preset condition, each node in the node cluster to perform end-to-end information transmission through a communication link between the nodes, where each node at least includes a production node and a consumption node.
And the second control module 103 is configured to switch each node to a communication link with the message middleware when the current environment parameter does not meet the first preset condition, and perform information transmission between the nodes through the message middleware.
In one embodiment, the first control module 102 is further configured to: and controlling the production node to respectively provide corresponding information for each consumption node according to the partition to which the communication link between the production node and each consumption node belongs.
In one embodiment, the partitions are determined by a hash partition policy or a Round-robin partition policy.
In one embodiment, the production node comprises a first node and the consumption node comprises a second node and a third node.
The first control module 102 is further configured to:
and controlling the second node to receive the original information sent by the first node through a communication link with the first node so that the second node sends subscription information generated after consuming the original information to the third node, wherein the original information is generated by the first node according to a consumption request of the third node.
In one embodiment, the first control module 102 is further configured to:
and controlling the second node to receive the consumption request of the third node through a communication link with the third node so that the second node consumes the original information according to the matching result of the consumption request and the original information, generates subscription information and sends the subscription information to the third node.
In one embodiment, the second control module 103 is further configured to:
and after each node is switched to a communication link with the message middleware, receiving the information provided by the production node, and marking the information received from the production node as cache information.
And detecting information existing in the message middleware, and sending the cache information to the message middleware when detecting that the message middleware does not have the same information as the cache information, so that the message middleware forwards the cache information to the corresponding consumption node.
In one embodiment, the communication protocol between the message middleware and each node is RSocket.
In one embodiment, the API employed by each node is the same.
In one embodiment, a computer apparatus is provided, as shown in fig. 4, comprising a processor, a memory, a network interface, an input device, and a display screen connected by a system bus. Wherein the memory includes a non-volatile storage medium and an internal memory. The non-volatile storage medium of the computer device stores an operating system and may also store a computer program that, when executed by the processor, causes the processor to implement the information transmission method. The internal memory may also have a computer program stored therein, which when executed by the processor, causes the processor to perform the information transfer method. Those skilled in the art will appreciate that the architecture shown in fig. 4 is merely a block diagram of some of the structures associated with the disclosed aspects and is not intended to limit the computing devices to which the disclosed aspects apply, as particular computing devices may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.
In one embodiment, the service scheduling apparatus provided in the present application may be implemented in the form of a computer program, and the computer program may be run on a computer device as shown in fig. 4. The memory of the computer device may store therein the individual program modules that make up the service scheduler. The computer program constituted by the respective program modules causes the processor to execute the steps in the information transmission method of the respective embodiments of the present application described in the present specification.
In one embodiment, there is provided an electronic device including: a memory, a processor and a computer program stored on the memory and executable on the processor, the processor executing the program to perform the steps of the information transmission method described above. The steps of the information transmission method herein may be steps in the information transmission methods of the respective embodiments described above.
In one embodiment, as shown in fig. 5, there is provided an information transmission system comprising a plurality of nodes, each node comprising at least a production node and a consumption node, message middleware, and an electronic device as described in the above embodiments.
Each node is used for carrying out end-to-end information transmission through a communication link between the nodes when receiving a first control signal of the electronic equipment; and the number of the first and second groups,
and when receiving a second control signal of the electronic equipment, switching to a communication link with the message middleware, and transmitting information between the nodes through the message middleware.
In one embodiment, a computer-readable storage medium is provided, which stores computer-executable instructions for causing a computer to perform the steps of the above-described information transmission method. The steps of the information transmission method herein may be steps in the information transmission methods of the respective embodiments described above.
The foregoing is a preferred embodiment of the present application, and it should be noted that, for those skilled in the art, various modifications and decorations can be made without departing from the principle of the present application, and these modifications and decorations are also regarded as the protection scope of the present application.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by a computer program, which can be stored in a computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. The storage medium may be a magnetic disk, an optical disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), or the like.
Claims (11)
1. An information transmission method, comprising:
acquiring current environmental parameters in real time;
when the current environmental parameters meet first preset conditions, controlling each node in a node cluster to carry out end-to-end information transmission through a communication link between each node, wherein each node at least comprises a production node and a consumption node;
and when the current environment parameters do not meet the first preset condition, switching each node to a communication link with a message middleware, and transmitting information among the nodes through the message middleware.
2. The information transmission method according to claim 1, wherein the end-to-end information transmission performed by each node in the control node cluster through a communication link between each node includes:
and controlling the production node to respectively provide corresponding information for each consumption node according to the partition to which the communication link between the production node and each consumption node belongs.
3. The information transmission method according to claim 2, wherein the partition is determined by a hash partition policy or a Round-robin partition policy.
4. The information transmission method according to claim 1, wherein the production node comprises a first node, and the consumption node comprises a second node and a third node;
each node in the control node cluster performs end-to-end information transmission through a communication link between each node, and the method comprises the following steps:
and controlling the second node to receive original information sent by the first node through a communication link with the first node so that the second node sends subscription information generated after consuming the original information to the third node, wherein the original information is generated by the first node according to a consumption request of the third node.
5. The information transmission method according to claim 4, wherein the step of enabling the second node to send subscription information generated after the second node consumes the original information to the third node includes:
and controlling the second node to receive the consumption request of the third node through a communication link with the third node, so that the second node consumes the original information according to the matching result of the consumption request and the original information, generates the subscription information and sends the subscription information to the third node.
6. The method of claim 1, wherein the switching the nodes to a communication link with a message middleware through which information is transmitted between the nodes comprises:
after each node is switched to a communication link with a message middleware, receiving information provided by the production node, and marking the information received from the production node as cache information;
and detecting information existing in the message middleware, and sending the cache information to the message middleware when detecting that the message middleware does not have the same information as the cache information, so that the message middleware forwards the cache information to the corresponding consumption node.
7. The information transmission method according to claim 1, wherein a communication protocol between the message middleware and each of the nodes is RSocket.
8. The information transmission method according to claim 1, wherein the API used by each of the nodes is the same.
9. An information transmission apparatus, comprising:
the data acquisition module is used for acquiring current environmental parameters in real time;
a first control module, configured to control, when the current environment parameter meets a first preset condition, each node in a node cluster to perform end-to-end information transmission through a communication link between the nodes, where each node at least includes a production node and a consumption node;
and the second control module is used for switching each node to a communication link with a message middleware when the current environment parameter does not meet the first preset condition, and transmitting information among the nodes through the message middleware.
10. An electronic device, comprising: memory, processor and computer program stored on the memory and executable on the processor, characterized in that the processor implements the information transmission method according to any one of claims 1 to 8 when executing the program.
11. An information transmission system comprising a plurality of nodes, message middleware, and an electronic device according to claim 10, each of said nodes comprising at least a production node and a consumption node;
each node is used for carrying out end-to-end information transmission through a communication link between the nodes when receiving a first control signal of the electronic equipment; and the number of the first and second groups,
and when receiving a second control signal of the electronic equipment, switching to a communication link with a message middleware, and transmitting information between the nodes through the message middleware.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110133176.4A CN112988411A (en) | 2021-01-29 | 2021-01-29 | Information transmission method, device and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110133176.4A CN112988411A (en) | 2021-01-29 | 2021-01-29 | Information transmission method, device and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112988411A true CN112988411A (en) | 2021-06-18 |
Family
ID=76345940
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110133176.4A Pending CN112988411A (en) | 2021-01-29 | 2021-01-29 | Information transmission method, device and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112988411A (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108566614A (en) * | 2017-11-06 | 2018-09-21 | 南京大学 | A kind of end-to-end communication method and system towards time delay tolerant network |
| CN111338814A (en) * | 2020-02-13 | 2020-06-26 | 北京奇艺世纪科技有限公司 | Message processing method and device, storage medium and electronic device |
| CN111381986A (en) * | 2020-03-10 | 2020-07-07 | 北京奇艺世纪科技有限公司 | Message transmission method, system, device, electronic equipment and readable storage medium |
-
2021
- 2021-01-29 CN CN202110133176.4A patent/CN112988411A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108566614A (en) * | 2017-11-06 | 2018-09-21 | 南京大学 | A kind of end-to-end communication method and system towards time delay tolerant network |
| CN111338814A (en) * | 2020-02-13 | 2020-06-26 | 北京奇艺世纪科技有限公司 | Message processing method and device, storage medium and electronic device |
| CN111381986A (en) * | 2020-03-10 | 2020-07-07 | 北京奇艺世纪科技有限公司 | Message transmission method, system, device, electronic equipment and readable storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102170396B (en) | QoS control method of cloud storage system based on differentiated service | |
| CN109451072A (en) | A kind of message caching system and method based on Kafka | |
| US10341792B1 (en) | System for distributing audio output using multiple devices | |
| US7594007B2 (en) | Distributed service management for distributed networks | |
| EP2321937B1 (en) | Load balancing for services | |
| CN110278161B (en) | Message distribution method, device and system based on user mode protocol stack | |
| Kliazovich et al. | CA-DAG: Communication-aware directed acyclic graphs for modeling cloud computing applications | |
| CN114064211B (en) | Video stream analysis system and method based on end-side-cloud computing architecture | |
| Banaie et al. | Performance analysis of multithreaded IoT gateway | |
| CN110535811B (en) | Remote memory management method and system, server, client and storage medium | |
| CN114710571B (en) | Data packet processing system | |
| CN105700940A (en) | Scheduler and dynamic multiplexing method thereof | |
| US20090132582A1 (en) | Processor-server hybrid system for processing data | |
| CN113254050A (en) | Micro front-end system | |
| CN116614517B (en) | Container mirror image preheating and distributing method for edge computing scene | |
| CN112988411A (en) | Information transmission method, device and system | |
| CN116107710A (en) | Method, apparatus, device and medium for processing offline rendering tasks | |
| CN111431955A (en) | Streaming data processing system and method | |
| CN114978998B (en) | Flow control method, device, terminal and storage medium | |
| Nakai et al. | Improving the QoS of web services via client-based load distribution | |
| US7224681B2 (en) | Processor with dynamic table-based scheduling using multi-entry table locations for handling transmission request collisions | |
| CN114499834A (en) | Internet of things quantum key distribution method and system, electronic equipment and storage medium | |
| CN107995264B (en) | CDN service verification code distribution method and system based on message queue | |
| CN110324367B (en) | Remote monitoring system, monitoring method and device thereof, storage medium and processor | |
| Prellwitz et al. | Adaptive information distribution for dynamic sets using multicast push and pull |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |