CN111131083B - Method, device and equipment for data transmission between nodes and computer readable storage medium - Google Patents
Method, device and equipment for data transmission between nodes and computer readable storage medium Download PDFInfo
- Publication number
- CN111131083B CN111131083B CN201911382416.3A CN201911382416A CN111131083B CN 111131083 B CN111131083 B CN 111131083B CN 201911382416 A CN201911382416 A CN 201911382416A CN 111131083 B CN111131083 B CN 111131083B
- Authority
- CN
- China
- Prior art keywords
- queue
- low
- bandwidth
- data
- delay
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000005540 biological transmission Effects 0.000 title claims abstract description 170
- 238000000034 method Methods 0.000 title claims abstract description 35
- 230000004044 response Effects 0.000 claims description 21
- 238000004590 computer program Methods 0.000 claims description 11
- 238000001514 detection method Methods 0.000 claims description 3
- 230000008569 process Effects 0.000 claims description 2
- 230000000694 effects Effects 0.000 abstract description 3
- 239000002699 waste material Substances 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 230000003993 interaction Effects 0.000 description 4
- 230000006872 improvement Effects 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000004891 communication Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L47/00—Traffic control in data switching networks
- H04L47/50—Queue scheduling
- H04L47/62—Queue scheduling characterised by scheduling criteria
- H04L47/625—Queue scheduling characterised by scheduling criteria for service slots or service orders
- H04L47/6275—Queue scheduling characterised by scheduling criteria for service slots or service orders based on priority
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L47/00—Traffic control in data switching networks
- H04L47/70—Admission control; Resource allocation
- H04L47/72—Admission control; Resource allocation using reservation actions during connection setup
- H04L47/722—Admission control; Resource allocation using reservation actions during connection setup at the destination endpoint, e.g. reservation of terminal resources or buffer space
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention discloses a method for transmitting data between nodes, which comprises the following steps: acquiring data to be transmitted between two target nodes; classifying the data to be transmitted to obtain a low-delay data group, a bandwidth guarantee data group and a data group to be transmitted; sending the low-delay data group to each low-delay queue for transmission, and sending the bandwidth guarantee data group to each bandwidth guarantee queue for transmission; wherein, each low-delay queue and each bandwidth guarantee queue are obtained by dividing the bandwidth of the non-transparent transmission bridge between two target nodes; judging whether idle bandwidth exists in each low-delay queue and/or each bandwidth guarantee queue; and if so, transmitting the data group to be transmitted by using the idle bandwidth. The invention ensures the normal transmission of data, realizes the full utilization of the bandwidth of the non-transparent transmission bridge and greatly improves the data transmission efficiency. The invention also discloses a device, equipment and a storage medium for data transmission between nodes, and the device, the equipment and the storage medium have corresponding technical effects.
Description
Technical Field
The present invention relates to the field of distributed storage technologies, and in particular, to a method, an apparatus, a device, and a computer-readable storage medium for data transmission between nodes.
Background
With the development of distributed storage technology, distributed storage is to store data on a plurality of independent devices in a distributed manner. The defects that the traditional network storage system adopts centralized computer nodes to store all data, the computer nodes become the bottleneck of system performance, and the requirement of large-scale storage application cannot be met are overcome. The distributed storage system adopts an expandable system structure, utilizes a plurality of computer nodes to share the storage load, and utilizes the position server to position the storage information, thereby not only improving the reliability, the availability and the access efficiency of the system, but also being easy to expand. Because the distributed storage system shares the storage load by using a plurality of computer nodes, the transmission of data among the computer nodes has a great influence on the system performance.
In the existing distributed storage system, the data transmission mode between any two nodes is to arbitrarily divide the bandwidth of a NON-Transparent transport Bridge (NTB) between the two nodes into a plurality of queues, and perform data transmission based on a Priority Queue (PQ) mode. The phenomenon that data transmission fails due to bandwidth contention among data is easy to occur, bandwidth of a non-transparent transmission bridge cannot be fully utilized, bandwidth waste is caused, and data transmission efficiency is low.
In summary, how to effectively solve the problems that data transmission is easy to fail, bandwidth is wasted, and data transmission efficiency is low is a problem that needs to be solved by those skilled in the art.
Disclosure of Invention
The invention aims to provide a method for transmitting data between nodes, which ensures the normal transmission of data, realizes the full utilization of the bandwidth of a non-transparent transmission bridge and greatly improves the data transmission efficiency; another object of the present invention is to provide an inter-node data transmission device, an apparatus and a computer readable storage medium.
In order to solve the technical problems, the invention provides the following technical scheme:
an inter-node data transmission method comprises the following steps:
acquiring data to be transmitted between two target nodes;
classifying the data to be transmitted to obtain a low-delay data group, a bandwidth guarantee data group and a data group to be transmitted;
sending the low-delay data group to each low-delay queue for transmission, and sending the bandwidth guarantee data group to each bandwidth guarantee queue for transmission; wherein each of the low-latency queues and each of the bandwidth guarantee queues are obtained by bandwidth division of a non-transparent transmission bridge between two of the target nodes;
judging whether idle bandwidth exists in each low-delay queue and/or each bandwidth guarantee queue;
and if so, transmitting the data group waiting for transmission by using the idle bandwidth.
In a specific embodiment of the present invention, before determining whether there is idle bandwidth in each of the low latency queues and each of the bandwidth guarantee queues, the method further includes:
detecting whether an idle low-delay queue and/or a bandwidth guarantee queue exist or not;
if yes, determining an idle low-delay queue and/or a bandwidth guarantee queue as a target queue, and dividing the bandwidth of the target queue into each low-delay queue and each bandwidth guarantee queue according to the bandwidth proportional relation of each queue except the target queue.
In a specific embodiment of the present invention, sending the low latency data group to each low latency queue for transmission includes:
acquiring the information response time between the two target nodes;
determining the target data transmission rate of the low-delay data group according to the information response duration;
and sending the low-delay data group to each low-delay queue for transmission according to the target data transmission rate.
In a specific embodiment of the present invention, sending the bandwidth guarantee data group to each bandwidth guarantee queue for transmission includes:
and sending the bandwidth guarantee data group to each bandwidth guarantee queue for transmission according to a first-in first-out queue scheduling algorithm.
In a specific embodiment of the present invention, a process of dividing a non-transparent transmission bridge between two target nodes to obtain each low-latency queue and each guaranteed bandwidth queue includes:
acquiring the data volume size ratio of a pre-stored low-delay data group and a bandwidth guarantee data group;
and according to the data volume size ratio, performing bandwidth division on a non-transparent transmission bridge between the two target nodes to obtain each low-delay queue and each bandwidth guarantee queue.
An inter-node data transmission apparatus comprising:
the data acquisition module is used for acquiring data to be transmitted between two target nodes;
the data classification module is used for classifying the data to be transmitted to obtain a low-delay data set, a bandwidth guarantee data set and a data set waiting for transmission;
the first data transmission module is used for sending the low-delay data group to each low-delay queue for transmission and sending the bandwidth guarantee data group to each bandwidth guarantee queue for transmission; wherein each of the low-latency queues and each of the bandwidth guarantee queues are obtained by bandwidth division of a non-transparent transmission bridge between two of the target nodes;
the judging module is used for judging whether idle bandwidth exists in each low-delay queue and/or each bandwidth guarantee queue;
and a second data transmission module, configured to transmit the data group to be transmitted by using the idle bandwidth when it is determined that idle bandwidth exists in each low-latency queue and/or each bandwidth guarantee queue.
In an embodiment of the present invention, the method further comprises:
a queue detection module, configured to detect whether there is an idle low-delay queue and/or an idle bandwidth guarantee queue before determining whether there is an idle bandwidth in each low-delay queue and each bandwidth guarantee queue;
and the bandwidth dividing module is used for determining the idle low-delay queues and/or the bandwidth guarantee queues as target queues when determining that the idle low-delay queues and/or the bandwidth guarantee queues exist, and dividing the bandwidth of the target queues into the low-delay queues and the bandwidth guarantee queues according to the bandwidth proportional relation of the queues except the target queues.
In a specific embodiment of the present invention, the first data transmission module includes a low latency data set transmission sub-module, and the low latency data set transmission sub-module includes:
a response time length obtaining unit, configured to obtain an information response time length between the two target nodes;
a transmission efficiency determining unit, configured to determine a target data transmission rate of the low latency data group according to the information response duration;
and the low-delay data group transmission unit is used for sending the low-delay data group to each low-delay queue to transmit according to the target data transmission rate.
An inter-node data transmission apparatus comprising:
a memory for storing a computer program;
a processor for implementing the steps of the inter-node data transmission method when executing the computer program.
A computer-readable storage medium, having stored thereon a computer program which, when being executed by a processor, carries out the steps of the inter-node data transmission method as set forth above.
By applying the method provided by the embodiment of the invention, the data to be transmitted between two target nodes is obtained; classifying the data to be transmitted to obtain a low-delay data group, a bandwidth guarantee data group and a data group to be transmitted; sending the low-delay data group to each low-delay queue for transmission, and sending the bandwidth guarantee data group to each bandwidth guarantee queue for transmission; wherein, each low-delay queue and each bandwidth guarantee queue are obtained by dividing the bandwidth of the non-transparent transmission bridge between the two target nodes; judging whether idle bandwidth exists in each low-delay queue and/or each bandwidth guarantee queue; and if so, transmitting the data group to be transmitted by using the idle bandwidth. The data to be transmitted are classified, and the bandwidth of the non-transparent transmission bridge between the two target nodes is divided into queues corresponding to the classified data groups according to needs, so that normal transmission of the data is guaranteed, waste of the bandwidth is avoided by fully utilizing the idle bandwidth, full utilization of the bandwidth of the non-transparent transmission bridge is realized, and the data transmission efficiency is greatly improved.
Accordingly, embodiments of the present invention further provide an inter-node data transmission apparatus, a device, and a computer readable storage medium corresponding to the inter-node data transmission method, which have the foregoing technical effects and are not described herein again.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
Fig. 1 is a flowchart illustrating an implementation of a method for data transmission between nodes according to an embodiment of the present invention;
FIG. 2 is a flowchart illustrating another method for transmitting data between nodes according to an embodiment of the present invention;
fig. 3 is a block diagram of an inter-node data transmission apparatus according to an embodiment of the present invention;
fig. 4 is a block diagram of an inter-node data transmission apparatus according to an embodiment of the present invention.
Detailed Description
In order that those skilled in the art will better understand the disclosure, the invention will be described in further detail with reference to the accompanying drawings and specific embodiments. It should be apparent that the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The first embodiment is as follows:
referring to fig. 1, fig. 1 is a flowchart of an implementation of a method for data transmission between nodes according to an embodiment of the present invention, where the method may include the following steps:
s101: and acquiring data to be transmitted between the two target nodes.
When two target nodes in the distributed storage system need to perform data interaction, the data to be transmitted between the two target nodes can be acquired. The data to be transmitted may include heartbeat data, IO data streams, inter-node control information streams, and other data.
The two target nodes can be any two nodes in the distributed storage system which are established with data interaction communication connection.
S102: and classifying the data to be transmitted to obtain a low-delay data group, a bandwidth guarantee data group and a data group waiting for transmission.
The division rule for dividing the data group of the data to be transmitted between the two target nodes may be preset, and for example, the data to be transmitted, which may be divided, may be preset to include a low-latency data group, a bandwidth guarantee data group, and a data group to be transmitted according to the transmission speed requirement and the transmission reliability requirement of the data to be transmitted. After the data to be transmitted between the two target nodes is acquired, the data to be transmitted can be classified to obtain a low-delay data group, a bandwidth guarantee data group and a data group to be transmitted. For example, volume mapping, volume copying, snapshot, etc. as data of the fast forwarding service may be divided into low-latency data groups, a node control information stream between two target nodes is divided into bandwidth guarantee data groups, and heartbeat data of two nodes is divided into data groups to be transmitted.
S103: and sending the low-delay data group to each low-delay queue for transmission, and sending the bandwidth guarantee data group to each bandwidth guarantee queue for transmission.
Wherein each low-latency queue and each bandwidth guarantee queue are obtained by performing bandwidth division on a non-transparent transmission bridge between two target nodes.
The bandwidth division can be performed on the non-transparent transmission bridge between the two target nodes in advance to obtain each low-delay queue and each bandwidth guarantee queue. After the low-delay data group, the bandwidth guarantee data group and the waiting transmission data group are obtained by classifying the data to be transmitted, the low-delay data group is sent to each low-delay queue for transmission, and the bandwidth guarantee data group is sent to each bandwidth guarantee queue for transmission. By transmitting data to be transmitted through class-based queues (CBQ), the data transmission rate is greatly improved.
S104: and judging whether idle bandwidth exists in each low-delay queue and/or each bandwidth guarantee queue, if so, executing the step S105, and if not, caching the peer-to-peer transmission data group.
Because the amount of data in the low-latency data group and the bandwidth-guaranteed data group at different time intervals is not constant, each low-latency queue and each bandwidth-guaranteed queue may store idle bandwidth, that is, remaining bandwidth other than the bandwidth used for transmitting the low-latency data group and the bandwidth-guaranteed data group, for example, only idle bandwidth in the low-latency queue may exist, only idle bandwidth in the bandwidth-guaranteed queue may exist, or both idle bandwidth in the low-latency queue and idle bandwidth in the bandwidth-guaranteed queue may exist. After sending the low-delay data group to each low-delay queue for transmission and sending the bandwidth guarantee data group to each bandwidth guarantee queue for transmission, judging whether idle bandwidth exists in each low-delay queue and/or each bandwidth guarantee queue, if so, indicating that the idle bandwidth really exists, in this case, executing step S105, and if not, indicating that each low-delay queue and each bandwidth guarantee queue are occupied, in this case, performing a caching operation on the data group to be transmitted, and after detecting that the idle bandwidth exists subsequently, sequentially transmitting each cached data group to be transmitted.
S105: and transmitting the data group to be transmitted by utilizing the idle bandwidth.
After determining that idle bandwidth exists in each low-delay queue and/or each bandwidth guarantee queue, that is, after determining that only idle bandwidth exists in the low-delay queue, or only idle bandwidth exists in the bandwidth guarantee queue, or both idle bandwidth exists in the low-delay queue and idle bandwidth exists in the bandwidth guarantee queue, the idle bandwidth can be used for transmitting peer-to-peer data groups, so that the bandwidth of a non-transparent transmission bridge between two target nodes is utilized to the maximum, and bandwidth waste is avoided.
By applying the method provided by the embodiment of the invention, the data to be transmitted between two target nodes is obtained; classifying the data to be transmitted to obtain a low-delay data group, a bandwidth guarantee data group and a data group to be transmitted; sending the low-delay data group to each low-delay queue for transmission, and sending the bandwidth guarantee data group to each bandwidth guarantee queue for transmission; wherein, each low-delay queue and each bandwidth guarantee queue are obtained by dividing the bandwidth of the non-transparent transmission bridge between the two target nodes; judging whether idle bandwidth exists in each low-delay queue and/or each bandwidth guarantee queue; and if so, transmitting the data group to be transmitted by utilizing the idle bandwidth. The data to be transmitted are classified, and the bandwidth of the non-transparent transmission bridge between the two target nodes is divided into queues corresponding to the classified data groups according to needs, so that normal transmission of the data is guaranteed, waste of the bandwidth is avoided by fully utilizing the idle bandwidth, full utilization of the bandwidth of the non-transparent transmission bridge is realized, and the data transmission efficiency is greatly improved.
It should be noted that, based on the first embodiment, the embodiment of the present invention further provides a corresponding improvement scheme. In the following embodiments, steps that are the same as or correspond to those in the first embodiment may be referred to each other, and corresponding advantageous effects may also be referred to each other, which are not described in detail in the following modified embodiments.
Example two:
referring to fig. 2, fig. 2 is a flowchart of another implementation of a method for data transmission between nodes in the embodiment of the present invention, where the method may include the following steps:
s201: and acquiring data to be transmitted between the two target nodes.
S202: and classifying the data to be transmitted to obtain a low-delay data group, a bandwidth guarantee data group and a data group waiting for transmission.
S203: and acquiring the information response time length between the two target nodes.
Because the bandwidth of the non-transparent transmission bridge between the two target nodes is limited, when the data interaction amount between the two target nodes is large, the response speed of the data interaction is reduced, so that the information response time between the two target nodes is increased, and therefore after the data to be transmitted are classified to obtain a low-delay data group, a bandwidth guarantee data group and a data group to be transmitted, the information response time between the two target nodes can be obtained, for example, the average time can be calculated by testing the time of sending a request and responding to information for many times, and the average time is determined as the information response time between the two target nodes.
S204: and determining the target data transmission rate of the low-delay data group according to the information response time.
After the information response time length between the two target nodes is obtained, the target data transmission rate of the low-delay data group can be determined according to the information response time length.
S205: and sending the low-delay data group to each low-delay queue for transmission according to the target data transmission rate.
And each low-delay queue and each bandwidth guarantee queue are obtained by dividing the bandwidth of the non-transparent transmission bridge between the two target nodes according to the data volume proportion of the pre-stored low-delay data group and the pre-stored bandwidth guarantee data group.
The data quantity of the low-delay data group and the bandwidth guarantee data group which need to be interacted between the two target nodes can be summarized in advance to obtain the data quantity size ratio of the low-delay data group and the bandwidth guarantee data group, and therefore according to the data quantity size ratio of the pre-stored low-delay data group and the pre-stored bandwidth guarantee data group, the bandwidth of the non-transparent transmission bridge between the two target nodes is divided to obtain each low-delay queue and each bandwidth guarantee queue.
After the target data transmission rate of the low-delay data group is determined, the low-delay data group is sent to each low-delay queue to be transmitted according to the target data transmission rate, so that the low-delay data group is limited in speed, the congestion of a non-transparent transmission bridge is avoided, and the transmission reliability of the low-delay data group is improved.
S206: and sending the bandwidth guarantee data group to each bandwidth guarantee queue for transmission according to a first-in first-out queue scheduling algorithm.
After the bandwidth guarantee data groups are obtained by classifying the data to be transmitted and the bandwidth guarantee data groups are obtained by bandwidth division of the non-transparent transmission bridge between the two target nodes according to the data volume proportion of the pre-stored low-delay data groups and the pre-stored bandwidth guarantee data groups, the bandwidth guarantee data groups can be sent to the bandwidth guarantee queues to be transmitted according to a first-in first-out queue scheduling algorithm.
S207: and detecting whether an idle low-delay queue and/or a bandwidth guarantee queue exist or not, if so, executing the step S208, otherwise, skipping the step S208 and directly executing the step S209.
After the low-latency data group is sent to each low-latency queue for transmission and the bandwidth guarantee data group is sent to each bandwidth guarantee queue for transmission, the low-latency queue may be idle due to a small data amount of the low-latency data group, or the bandwidth guarantee queue may be idle due to a small data amount of the bandwidth guarantee data group, or both the low-latency queue and the bandwidth guarantee queue may be idle due to small data amounts of the low-latency data group and the bandwidth guarantee data group.
S208: determining the idle low-delay queues and/or bandwidth guarantee queues as target queues, and dividing the bandwidth of the target queues into the low-delay queues and the bandwidth guarantee queues according to the bandwidth proportional relation of the queues except the target queues.
After determining that there is an idle low-latency queue and/or bandwidth guaranteed queue, determining the idle low-latency queue and/or bandwidth guaranteed queue as a target queue. After only the low-delay queue is determined to be idle, determining the idle low-delay queue as a target queue; after determining that only the bandwidth guarantee queue is free, determining the free bandwidth guarantee queue as a target queue; after determining that both the low-delay queue is idle and the bandwidth guarantee queue is idle, determining both the idle low-delay queue and the idle bandwidth guarantee queue as target queues. After the target queue is determined, dividing the bandwidth of the target queue into each low-delay queue and each bandwidth guarantee queue according to the bandwidth proportional relation of each queue except the target queue.
S209: and judging whether idle bandwidth exists in each low-delay queue and/or each bandwidth guarantee queue, if so, executing the step S210, and otherwise, caching the peer-to-peer data group to be transmitted.
S210: and transmitting the data group to be transmitted by utilizing the idle bandwidth.
Corresponding to the above method embodiment, an embodiment of the present invention further provides an inter-node data transmission apparatus, and the inter-node data transmission apparatus described below and the inter-node data transmission method described above may be referred to in a corresponding manner.
Referring to fig. 3, fig. 3 is a block diagram of an inter-node data transmission apparatus according to an embodiment of the present invention, where the apparatus may include:
a data obtaining module 31, configured to obtain data to be transmitted between two target nodes;
a data classification module 32, configured to classify each to-be-transmitted data to obtain a low-latency data group, a bandwidth guarantee data group, and a data group to be transmitted;
a first data transmission module 33, configured to send the low-latency data group to each low-latency queue for transmission, and send the bandwidth guarantee data group to each bandwidth guarantee queue for transmission; wherein, each low-delay queue and each bandwidth guarantee queue are obtained by dividing the bandwidth of the non-transparent transmission bridge between two target nodes;
a determining module 34, configured to determine whether there is an idle bandwidth in each low-delay queue and/or each bandwidth guarantee queue;
and a second data transmission module 35, configured to transmit the data group to be transmitted by using the idle bandwidth when it is determined that idle bandwidths exist in each low-delay queue and/or each bandwidth guarantee queue.
By applying the device provided by the embodiment of the invention, the data to be transmitted between two target nodes is acquired; classifying the data to be transmitted to obtain a low-delay data group, a bandwidth guarantee data group and a data group to be transmitted; sending the low-delay data group to each low-delay queue for transmission, and sending the bandwidth guarantee data group to each bandwidth guarantee queue for transmission; wherein, each low-delay queue and each bandwidth guarantee queue are obtained by dividing the bandwidth of the non-transparent transmission bridge between two target nodes; judging whether idle bandwidth exists in each low-delay queue and/or each bandwidth guarantee queue; and if so, transmitting the data group to be transmitted by using the idle bandwidth. The data to be transmitted are classified, and the bandwidth of the non-transparent transmission bridge between the two target nodes is divided into queues corresponding to the classified data groups according to needs, so that normal transmission of the data is guaranteed, waste of the bandwidth is avoided by fully utilizing the idle bandwidth, full utilization of the bandwidth of the non-transparent transmission bridge is realized, and the data transmission efficiency is greatly improved.
In one embodiment of the present invention, the apparatus may further include:
the queue detection module is used for detecting whether idle low-delay queues and/or bandwidth guarantee queues exist before judging whether idle bandwidths exist in the low-delay queues and the bandwidth guarantee queues or not;
and the bandwidth dividing module is used for determining the idle low-delay queue and/or bandwidth guarantee queue as a target queue when determining that the idle low-delay queue and/or bandwidth guarantee queue is detected to exist, and dividing the bandwidth of the target queue into the low-delay queues and the bandwidth guarantee queues according to the bandwidth proportional relation of the queues except the target queue.
In a specific embodiment of the present invention, the first data transmission module 33 includes a low latency data set transmission sub-module, and the low latency data set transmission sub-module includes:
a response time length obtaining unit, configured to obtain an information response time length between two target nodes;
a transmission efficiency determining unit, configured to determine a target data transmission rate of the low latency data group according to the information response duration;
and the low-delay data group transmission unit is used for sending the low-delay data group to each low-delay queue to transmit according to the target data transmission rate.
In one embodiment of the present invention, the first data transmission module 33 includes a bandwidth-guaranteed data set transmission sub-module,
the bandwidth guarantee data group transmission submodule is a module which sends the bandwidth guarantee data group to each bandwidth guarantee queue for transmission according to a first-in first-out queue scheduling algorithm.
In a specific embodiment of the present invention, the apparatus includes a queue obtaining module, where the queue obtaining module includes:
the data volume proportion obtaining submodule is used for obtaining the data volume size proportion of a pre-stored low-delay data set and a bandwidth guarantee data set;
and the queue obtaining submodule is used for carrying out bandwidth division on the non-transparent transmission bridge between the two target nodes according to the size proportion of the data volume to obtain each low-delay queue and each bandwidth guarantee queue.
Corresponding to the above method embodiment, referring to fig. 4, fig. 4 is a schematic diagram of an inter-node data transmission apparatus provided in the present invention, where the apparatus may include:
a memory 41 for storing a computer program;
the processor 42, when executing the computer program stored in the memory 41, may implement the following steps:
acquiring data to be transmitted between two target nodes; classifying the data to be transmitted to obtain a low-delay data group, a bandwidth guarantee data group and a data group to be transmitted; sending the low-delay data group to each low-delay queue for transmission, and sending the bandwidth guarantee data group to each bandwidth guarantee queue for transmission; wherein, each low-delay queue and each bandwidth guarantee queue are obtained by dividing the bandwidth of the non-transparent transmission bridge between two target nodes; judging whether idle bandwidth exists in each low-delay queue and/or each bandwidth guarantee queue; and if so, transmitting the data group to be transmitted by using the idle bandwidth.
For the introduction of the device provided by the present invention, please refer to the above method embodiment, which is not described herein again.
Corresponding to the above method embodiment, the present invention further provides a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, can implement the following steps:
acquiring data to be transmitted between two target nodes; classifying the data to be transmitted to obtain a low-delay data group, a bandwidth guarantee data group and a data group to be transmitted; sending the low-delay data group to each low-delay queue for transmission, and sending the bandwidth guarantee data group to each bandwidth guarantee queue for transmission; wherein, each low-delay queue and each bandwidth guarantee queue are obtained by dividing the bandwidth of the non-transparent transmission bridge between two target nodes; judging whether idle bandwidth exists in each low-delay queue and/or each bandwidth guarantee queue; and if so, transmitting the data group to be transmitted by using the idle bandwidth.
The computer-readable storage medium may include: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, an optical disk, or other various media capable of storing program codes.
For the introduction of the computer-readable storage medium provided by the present invention, please refer to the above method embodiments, which are not described herein again.
The embodiments are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same or similar parts among the embodiments are referred to each other. The device, the apparatus and the computer-readable storage medium disclosed in the embodiments correspond to the method disclosed in the embodiments, so that the description is simple, and the relevant points can be referred to the description of the method.
The principle and the embodiment of the present invention are explained by applying specific examples, and the above description of the embodiments is only used to help understanding the technical solution and the core idea of the present invention. It should be noted that, for those skilled in the art, without departing from the principle of the present invention, it is possible to make various improvements and modifications to the present invention, and those improvements and modifications also fall within the scope of the claims of the present invention.
Claims (7)
1. An inter-node data transmission method, comprising:
acquiring data to be transmitted between two target nodes;
classifying the data to be transmitted to obtain a low-delay data set, a bandwidth guarantee data set and a data set waiting to be transmitted;
sending the low-delay data group to each low-delay queue for transmission, and sending the bandwidth guarantee data group to each bandwidth guarantee queue for transmission; wherein each of the low-latency queues and each of the bandwidth guarantee queues are obtained by bandwidth division of a non-transparent transmission bridge between two of the target nodes;
judging whether idle bandwidth exists in each low-delay queue and/or each bandwidth guarantee queue;
if so, transmitting the data group waiting for transmission by using the idle bandwidth;
wherein, sending the low-latency data group to each low-latency queue for transmission comprises:
acquiring the information response time between the two target nodes;
determining the target data transmission rate of the low-delay data group according to the information response duration;
sending the low-delay data group to each low-delay queue for transmission according to the target data transmission rate;
the dividing process of dividing the bandwidth of the non-transparent transmission bridge between the two target nodes to obtain each low-delay queue and each bandwidth guarantee queue includes:
acquiring the data volume size ratio of a pre-stored low-delay data group and a bandwidth guarantee data group;
and according to the data volume size ratio, performing bandwidth division on a non-transparent transmission bridge between the two target nodes to obtain each low-delay queue and each bandwidth guarantee queue.
2. The method of claim 1, wherein before determining whether there is free bandwidth in each of the low latency queues and each of the guaranteed bandwidth queues, the method further comprises:
detecting whether an idle low-delay queue and/or a bandwidth guarantee queue exist or not;
if yes, determining an idle low-delay queue and/or a bandwidth guarantee queue as a target queue, and dividing the bandwidth of the target queue into the low-delay queues and the bandwidth guarantee queues according to the bandwidth proportional relation of the queues except the target queue.
3. The method according to claim 1 or 2, wherein the sending the bandwidth guarantee data group to each bandwidth guarantee queue for transmission comprises:
and sending the bandwidth guarantee data group to each bandwidth guarantee queue for transmission according to a first-in first-out queue scheduling algorithm.
4. An apparatus for transmitting data between nodes, comprising:
the data acquisition module is used for acquiring data to be transmitted between two target nodes;
the data classification module is used for classifying the data to be transmitted to obtain a low-delay data set, a bandwidth guarantee data set and a data set waiting for transmission;
the first data transmission module is used for sending the low-delay data group to each low-delay queue for transmission and sending the bandwidth guarantee data group to each bandwidth guarantee queue for transmission; wherein each of the low-latency queues and each of the bandwidth guarantee queues are obtained by performing bandwidth division on a non-transparent transmission bridge between two of the target nodes;
the judging module is used for judging whether idle bandwidth exists in each low-delay queue and/or each bandwidth guarantee queue;
a second data transmission module, configured to transmit the data group to be transmitted by using an idle bandwidth when it is determined that the idle bandwidth exists in each low-latency queue and/or each bandwidth guarantee queue;
a queue acquisition module comprising:
the data volume proportion obtaining submodule is used for obtaining the data volume size proportion of a pre-stored low-delay data set and a bandwidth guarantee data set;
and the queue obtaining submodule is used for carrying out bandwidth division on the non-transparent transmission bridge between the two target nodes according to the data volume size ratio to obtain each low-delay queue and each bandwidth guarantee queue.
5. The inter-node data transmission apparatus according to claim 4, further comprising:
a queue detection module, configured to detect whether there is an idle low-delay queue and/or a bandwidth guarantee queue before determining whether there is an idle bandwidth in each low-delay queue and each bandwidth guarantee queue;
a bandwidth dividing module, configured to determine an idle low-latency queue and/or a bandwidth guarantee queue as a target queue when it is determined that the idle low-latency queue and/or the bandwidth guarantee queue exists, and divide a bandwidth of the target queue into each low-latency queue and each bandwidth guarantee queue according to a bandwidth proportional relationship of each queue except the target queue;
wherein the first data transmission module comprises a low-latency data group transmission submodule, and the low-latency data group transmission submodule comprises:
a response time length obtaining unit, configured to obtain an information response time length between two target nodes;
a transmission efficiency determining unit, configured to determine a target data transmission rate of the low latency data group according to the information response duration;
and the low-delay data group transmission unit is used for sending the low-delay data group to each low-delay queue to transmit according to the target data transmission rate.
6. An inter-node data transmission apparatus, comprising:
a memory for storing a computer program;
a processor for implementing the steps of the method for data transmission between nodes according to any one of claims 1 to 3 when executing said computer program.
7. A computer-readable storage medium, characterized in that a computer program is stored on the computer-readable storage medium, which computer program, when being executed by a processor, carries out the steps of the method for data transmission between nodes according to any one of claims 1 to 3.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911382416.3A CN111131083B (en) | 2019-12-27 | 2019-12-27 | Method, device and equipment for data transmission between nodes and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911382416.3A CN111131083B (en) | 2019-12-27 | 2019-12-27 | Method, device and equipment for data transmission between nodes and computer readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111131083A CN111131083A (en) | 2020-05-08 |
| CN111131083B true CN111131083B (en) | 2023-02-28 |
Family
ID=70505004
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911382416.3A Active CN111131083B (en) | 2019-12-27 | 2019-12-27 | Method, device and equipment for data transmission between nodes and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111131083B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112311695B (en) * | 2020-10-21 | 2022-09-30 | 中国科学院计算技术研究所 | On-chip bandwidth dynamic allocation method and system |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102098217A (en) * | 2011-01-14 | 2011-06-15 | 中国科学技术大学 | Probability-based multipriority queue scheduling method |
| CN102231697A (en) * | 2011-06-17 | 2011-11-02 | 瑞斯康达科技发展股份有限公司 | Bandwidth dispatching method of message queues as well as message reporting method and device |
| CN106899514A (en) * | 2017-02-24 | 2017-06-27 | 西安电子科技大学 | Ensure the array dispatching method of QoS in Multimedia Service |
| CN108259383A (en) * | 2016-12-29 | 2018-07-06 | 北京华为数字技术有限公司 | The transmission method and the network equipment of a kind of data |
| CN110062425A (en) * | 2019-04-08 | 2019-07-26 | 京信通信系统（中国）有限公司 | Control method, device, computer equipment and the storage medium of network transmission speed |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106851782B (en) * | 2012-05-07 | 2020-11-17 | 华为终端有限公司 | Data transmission method and device |
| CN109548052B (en) * | 2019-01-21 | 2020-11-03 | 北京邮电大学 | Data transmission method and device, electronic equipment and readable storage medium |
-
2019
- 2019-12-27 CN CN201911382416.3A patent/CN111131083B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102098217A (en) * | 2011-01-14 | 2011-06-15 | 中国科学技术大学 | Probability-based multipriority queue scheduling method |
| CN102231697A (en) * | 2011-06-17 | 2011-11-02 | 瑞斯康达科技发展股份有限公司 | Bandwidth dispatching method of message queues as well as message reporting method and device |
| CN108259383A (en) * | 2016-12-29 | 2018-07-06 | 北京华为数字技术有限公司 | The transmission method and the network equipment of a kind of data |
| CN106899514A (en) * | 2017-02-24 | 2017-06-27 | 西安电子科技大学 | Ensure the array dispatching method of QoS in Multimedia Service |
| CN110062425A (en) * | 2019-04-08 | 2019-07-26 | 京信通信系统（中国）有限公司 | Control method, device, computer equipment and the storage medium of network transmission speed |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111131083A (en) | 2020-05-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111966289B (en) | Partition optimization method and system based on Kafka cluster | |
| CN104092719B (en) | Document transmission method, device and distributed cluster file system | |
| US9014156B2 (en) | Traffic forwarding in mesh networks | |
| CN112511325B (en) | Network congestion control method, node, system and storage medium | |
| CN103493449A (en) | Effective circuits in packet-switched networks | |
| US10484730B1 (en) | Chunked transfer mode bandwidth estimation | |
| CN106603692B (en) | Data storage method and device in distributed storage system | |
| EP2755363A1 (en) | Data-fast-distribution method and device | |
| US20220137876A1 (en) | Method and device for distributed data storage | |
| CN113992594B (en) | Flow control method, flow control device, electronic equipment and computer readable storage medium | |
| US10581708B2 (en) | Website monitoring method and device | |
| CN115174490B (en) | Data transmission method and network application terminal | |
| CN113296976A (en) | Message processing method, message processing device, electronic equipment, storage medium and program product | |
| CN111131083B (en) | Method, device and equipment for data transmission between nodes and computer readable storage medium | |
| US10951707B2 (en) | Selection device, device selection method, and program | |
| CN101867580A (en) | Method for allocating network flow and device | |
| CN106790354B (en) | Communication method and device for preventing data congestion | |
| CN110659132B (en) | Request processing optimization method and computer-readable storage medium | |
| CN110247847B (en) | Method and device for back source routing between nodes | |
| CN101958934A (en) | Electronic program guide incremental content synchronization method, device and system | |
| CN109284257A (en) | A kind of log write-in method, apparatus, electronic equipment and storage medium | |
| CN111857992B (en) | Method and device for allocating linear resources in Radosgw module | |
| US10250515B2 (en) | Method and device for forwarding data messages | |
| CN115391051A (en) | Video computing task scheduling method, device and computer readable medium | |
| CN114500544A (en) | Method, system, equipment and medium for load balancing among nodes |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |