WO2019148716A1

WO2019148716A1 - Data transmission method, server, and storage medium

Info

Publication number: WO2019148716A1
Application number: PCT/CN2018/089133
Authority: WO
Inventors: 刘浩月
Original assignee: 平安科技（深圳）有限公司
Priority date: 2018-01-30
Filing date: 2018-05-31
Publication date: 2019-08-08
Also published as: CN108306717A; CN108306717B

Abstract

Disclosed are a data transmission method, a server, and a storage medium. The method comprises: monitoring system resources of the server and current states of storage nodes; determining whether the system resources and the current states of the storage nodes are normal; and if the system resources and the current states of the storage nodes are normal, selecting one routing algorithm from preset routing algorithms, and distributing, according to the selected routing algorithm, data packets to corresponding networks to be transmitted to the corresponding storage nodes. According to the present application, by constructing two groups of network systems, the data transmission efficiency is improved, and the transmission performance of a storage system is improved.

Description

Data transmission method, server and storage medium

Priority claim

The present application claims priority to Chinese Patent Application No. 201810089659.7, entitled "Data Transmission Method, Server and Storage Medium", which is incorporated by reference. The method is combined with this application.

Technical field

The present application relates to the field of data transmission technologies, and in particular, to a data transmission method, a server, and a computer readable storage medium.

Background technique

At present, in the existing distributed storage system, there is only one set of networks between storage nodes for data transmission, and there is a risk of single point network failure. A single transmission network is prone to data transmission delays or system congestion or even crashes when transmitting large amounts of data. At the same time, when a single network used for data transmission experiences system instability or failure, it not only causes data loss, but also degrades the performance of the entire distributed storage system.

Summary of the invention

In view of the above, the present application provides a data transmission method, a server, and a computer readable storage medium, the main purpose of which is to share data transmission tasks, improve network transmission efficiency, increase redundant backup, and improve system stability and reliability.

To achieve the above objective, the present application provides a data transmission method, including:

Monitoring step: monitoring the system resources of the server and the current state of the storage node;

Judgment step: determining whether the current state of the system resource and the storage node is normal;

Selection step: when the system resources are normal and the current state of the storage node is normal, a routing algorithm is selected from the preset routing algorithm, and the data packet is allocated to the corresponding network according to the selected routing algorithm to be transmitted to the corresponding network. The storage node, the preset routing algorithm includes:

Polling algorithm: use two sets of network transmission data packets in turn;

Optimal Path Algorithm: Statistics preset number of network packet transmission is completed each time, denoted as N _1, N _2, calculates the absolute value of N ₁ -N _2, when the absolute value is less than N ₁ -N ₂ When the preset value is used, the two groups of networks are assigned the same priority, and the polling algorithm is used to allocate data packets to the two groups of networks. When the absolute value of N ₁ -N ₂ is greater than the preset value, the transmission is completed within the preset time. The network with a large number of data packets is allocated the first priority, and the remaining network is assigned the second priority, and the first proportion of the data packet is allocated to the network transmission of the first priority, and the second proportion of the data packet is allocated to the first a second priority network transmission, wherein the first priority is higher than the second priority, and the first ratio is greater than the second ratio;

Shortest Queue Algorithm: counting the number of packets in each group of network transmission has not completed, are referred to as N _1, N _2, calculates the absolute value of N ₁ -N _2, when an absolute value of N ₁ -N ₂ is less than a preset threshold value, Assigning the same priority to the two groups of networks, using the polling algorithm to allocate data packets to the two groups of networks. When the absolute value of N ₁ -N ₂ is greater than the preset threshold, the number of packets that are not completed for the current transmission is small. The network allocates the first priority, and the remaining network allocates the second priority, and the third proportion of the data packet is allocated to the network transmission of the first priority, and the fourth proportion of the data packet is allocated to another network transmission, where the first The priority is higher than the second priority, and the third ratio is greater than the fourth ratio.

In addition, the application further provides a server, the server includes: a memory and a processor, wherein the memory stores a data transmission program, and the data transmission program is executed by the processor, and the following steps can be implemented:

Polling algorithm: use two sets of network transmission data packets in turn;

In addition, in order to achieve the above object, the present application further provides a computer readable storage medium including a data transmission program, where the data transmission program is executed by a processor, and the data transmission as described above can be implemented. Any step in the method.

The data transmission method, the server and the computer readable storage medium proposed by the present application, by constructing two groups of networks, select an optimal routing algorithm from a preset routing algorithm by using a preset selection rule according to the state of the system resource. The transmission of the two groups of networks is allocated to improve the stability and transmission efficiency of the system.

DRAWINGS

1 is a schematic diagram of an application environment of a data transmission method according to the present application;

2 is a schematic diagram of a preferred embodiment of a server of the present application;

3 is a block diagram showing a preferred embodiment of the data transmission program of FIG. 2;

4 is a flowchart of a first embodiment of a data transmission method according to the present application;

FIG. 5 is a flowchart of a second embodiment of a data transmission method according to the present application.

The implementation, functional features and advantages of the present application will be further described with reference to the accompanying drawings.

Detailed ways

It is understood that the specific embodiments described herein are merely illustrative of the application and are not intended to be limiting.

FIG. 1 is a schematic diagram of an application environment of a data transmission method of the present application.

In this embodiment, the present application provides a distributed storage system, which includes a server 1, a storage node 2, two sets of networks - Network A and Network B, and one or more clients 3. The client 3 issues a request to the server to read and write data to the storage node 2. After receiving the request of the client 3 to read and write data, the server 1 selects a group of networks from the network A and the network B to read and write data from the storage node according to a preset selection rule.

The server 1 is also used to monitor the working state and capacity of the storage node 2. The network includes transmission lines such as communication lines and network switches.

2 is a schematic diagram of a preferred embodiment of the server 1 of the present application.

In this embodiment, the server 1 may be a server, a smart phone, a tablet computer, a personal computer, a portable computer, and other electronic devices having computing functions.

The server 1 includes a memory 11, a processor 12, a network interface 13, and a communication bus 14. The network interface 13 can optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). Communication bus 14 is used to implement connection communication between these components.

The memory 11 includes at least one type of readable storage medium. The at least one type of readable storage medium may be a non-volatile storage medium such as a flash memory, a hard disk, a multimedia card, a card type memory, or the like. In some embodiments, the memory 11 may be an internal storage unit of the server 1, such as a hard disk of the server 1. In other embodiments, the memory 11 may also be an external storage unit of the server 1, such as a plug-in hard disk equipped on the server 1, a smart memory card (SMC), and a secure digital ( Secure Digital, SD) cards, flash cards, etc.

In the present embodiment, the memory 11 can be used not only for storing application software installed on the server 1 and various types of data, such as the data transmission program 10.

The processor 12, in some embodiments, may be a Central Processing Unit (CPU), microprocessor or other data processing chip for running program code or processing data stored in the memory 11, such as executing a data transfer program. 10 computer program code, monitoring of the current state of the storage node and system resources, and the like.

Figure 2 shows only server 1 with components 11-14 and data transfer program 10, but it should be understood that not all illustrated components may be implemented, and more or fewer components may be implemented instead.

Optionally, the server 1 may also include a display, which may be referred to as a display screen or a display unit. In some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, and an Organic Light-Emitting Diode (OLED) touch sensor. The display is used to display information processed in the server 1 and a work interface for displaying visualizations.

Optionally, the server 1 may further include a user interface, and the user interface may include an input unit such as a keyboard, a voice output device such as an audio, a headset, etc., optionally, the user interface may further include a standard wired interface and a wireless interface.

In the server embodiment shown in FIG. 2, the program code of the data transfer program 10 is stored in the memory 11 as a computer storage medium. When the processor 12 executes the program code of the data transfer program 10, the following steps are implemented:

Polling algorithm: use two sets of network transmission data packets in turn;

For specific principles, please refer to the following FIG. 3 for a block diagram of a preferred embodiment of the data transmission program 10, FIG. 4 for a flowchart of the first embodiment of the data transmission method, and FIG. 5 for an introduction of the flowchart of the second embodiment of the data transmission method. .

3 is a block diagram of a preferred embodiment of the data transfer program 10 of FIG. 2. A module as referred to in this application refers to a series of computer program instructions that are capable of performing a particular function.

In this embodiment, the data transmission program 10 includes: a monitoring module 110, a determining module 120, a selecting module 130, a calculating module 140, and a notification module 150. The functions or operating steps implemented by the modules 110-150 are similar to the above. , not detailed here, exemplarily, for example:

The monitoring module 110 is configured to monitor a system resource of the server and a current state of the storage node. The monitoring module 110 monitors the current state of the storage node and system resources in real time. The current state of the monitoring storage node refers to monitoring whether the storage node is faulty. The system resources include: local disk utilization, memory utilization, CPU utilization, and the like.

The determining module 120 is configured to determine whether the current state of the storage node is normal or determine the current state of the system resource and determine whether the difference between the delay time of the current transmission data and the average delay time of the network exceeds a first preset value. The determining module 120 determines whether the storage node is faulty according to the system resource of the monitored server and the current state of the storage node, and whether the system resource is normal. When a storage node fails, a group of networks is automatically designated to perform data repair on the failed storage node, and another group of networks is used to transmit data to other storage nodes. The data repair refers to a method for repairing data in the storage node when the storage node fails to cause data loss, wherein the data repair includes: copy data repair, regeneration code data repair, multi-node collaborative data repair, and the like. When the storage node is normal, the determination module 120 determines the current state of the system resource. The pre-determined determining method includes: determining whether the current state of the system resource is normal according to the preset usage of the local disk, the memory utilization, the first preset threshold of the CPU utilization, and the second preset threshold. When it is monitored that the utilization of one of the system resources is higher than the corresponding second preset threshold, the system resources are judged to be tight, and when the utilization rate of all system resources is monitored to be lower than the corresponding first preset threshold, the system resources are determined. Sufficiently, when it is monitored that the utilization of all system resources is lower than the second preset threshold, it is determined that the system resources are currently in a normal state. It can be understood that the normal state of the system resources includes sufficient system resources, and sufficient system resources are only a special case of normal system resources. For example, the first preset threshold of the local disk is 50%, the second preset threshold is 80%, the first preset threshold of the memory utilization is 40%, and the second preset threshold is 70%. The first preset threshold of the utilization is 40%, and the second preset threshold is 60%. When the local disk utilization is 82%, the memory utilization is 68%, and the CPU utilization is 30%, the system resources are tight. When the local disk utilization is 40%, the memory utilization is 30%, and the CPU utilization is 30%, the system resources are sufficient. When the local disk utilization is 60%, the memory utilization is 68%, and the CPU utilization is 30%, indicating that the system resources are currently in a normal state. For the specific determination method of the difference between the delay time of the current transmission data and the average delay time, refer to the following description of the flowchart of the second embodiment of the data transmission method.

The selecting module 130 is configured to select a routing algorithm from the preset routing algorithm when the system resources are normal and the current state of the storage node is normal, and allocate the data packet to the corresponding network transmission according to the selected routing algorithm. Go to the corresponding storage node. In this embodiment, when the current state of the system resource is normal, the selection module 130 selects a routing algorithm from the preset routing algorithm. The preset routing algorithm includes:

Polling algorithm: Two sets of network transmission packets are used in turn. For example, the server 1 divides the data packets that need to be transmitted into several parts, the first part of the data packets are transmitted using the network A, the second part of the data packets are transmitted using the network B, and the third part of the data packets are transmitted using the network A, so that the loop is repeated until all the needs are needed. The uploaded data packet is transferred.

Optimal Path Algorithm: Statistics preset number of network packet transmission is completed each time, denoted as N _1, N _2, calculates the absolute value of N ₁ -N _2, when the absolute value is less than N ₁ -N ₂ When the preset value is used, the two groups of networks are assigned the same priority, and the polling algorithm is used to allocate data packets to the two groups of networks. When the absolute value of N ₁ -N ₂ is greater than the preset value, the transmission is completed within the preset time. The network with a large number of data packets is assigned a first priority, and the remaining network is assigned a second priority, and the first proportion of data packets are allocated to the network transmission of the first priority, and the second proportion of data packets are allocated to The second priority network transmission, wherein the first priority is higher than the second priority, and the first ratio is greater than the second ratio. Assume that the number of data packets completed by network A and network B in a unit time is 30 and 20 respectively. If the preset value is set to 8, the first priority is assigned to network A, and the data packets to be uploaded are to be uploaded. 60% is allocated to Network A, the second priority is assigned to Network B, and 40% of the packets that need to be uploaded are assigned to Network B.

Shortest Queue Algorithm: counting the number of packets in each group of network transmission has not completed, are referred to as N _1, N _2, calculates the absolute value of N ₁ -N _2, when an absolute value of N ₁ -N ₂ is less than a preset threshold value, Assigning the same priority to the two groups of networks, using the polling algorithm to allocate data packets to the two groups of networks; when the absolute value of N ₁ -N ₂ is greater than the preset threshold, the number of packets that are not completed for the current transmission is small. The network allocates the first priority, the remaining network allocates the second priority, allocates the third proportion of the data packet to the network transmission of the first priority, and allocates the fourth proportion of the data packet to another network transmission, where the One priority is higher than the second priority, and the third ratio is greater than the fourth ratio. Assume that the number of packets that have not been transmitted by Network A and Network B is 18 and 25 respectively. If the preset threshold is set to 6, the first priority will be assigned to Network A, and the packets to be uploaded will be used. 60% is assigned to Network A, the second priority is assigned to Network B, and 40% of the packets that need to be uploaded are assigned to Network B.

Specifically, in another embodiment, a handover step should be further included: when monitoring system resources of the server is tight, switching to the optimal path algorithm allocates the data packet to the corresponding network and transmits to the corresponding storage node, when When monitoring the system resources of the server is sufficient, switching to the polling algorithm allocates the data packet to the corresponding network and transmits to the corresponding storage node, and when the system resource is in a normal state, switching to the shortest queue algorithm allocates the data packet to the corresponding The network is transferred to the corresponding storage node. For example, when the memory utilization is higher than 70%, it indicates that the current system resources are tight, and the optimal path algorithm is automatically selected to allocate the data packets to the network A and the network B respectively.

The calculating module 140 is configured to calculate an average delay time of each group of network data transmissions in real time, and calculate a difference between a delay time of the current transmission data and an average delay time of the network. The delay time refers to a portion where the current transmission time exceeds the transmission time in the normal state. For example, in the normal state, it takes 10 seconds to transmit a data packet from the client 3 to the storage node 2, and if the transmission time is 16 seconds, the delay time is 6 seconds. Statistics and calculations The average of the delay times of all packets transmitted by each group of networks at the current time, which is the average delay time. Assuming an average delay time of 2 seconds, the difference between the current transmission data delay time and the network average delay time is 4 seconds.

The notification module 150 is configured to notify the storage node that receives the data that the current data transmission is invalid when the delay time of the current data transmission of the group of networks exceeds the second preset value, and request another group of networks to retransmit the data to the storage node. . Assuming that the second preset value is set to 20 seconds, when the delay time of the network A transmitting certain data to the storage node 2 exceeds 20 seconds, the notification module 150 sends abort information to notify the storage node 2 that the current data transmission is invalid, and requests the network. B retransmits the data to storage node 2.

As shown in FIG. 4, it is a flowchart of the first embodiment of the data transmission method of the present application.

In this embodiment, when the processor 12 executes the computer program of the data transmission program 10 stored in the memory 11, the method for implementing data transmission includes: Step S10 - Step S30:

In step S10, the monitoring module 110 monitors the current state of the storage node and system resources in real time. The current state of the monitoring storage node refers to monitoring whether the storage node is faulty. The system resources include: local disk utilization, memory utilization, CPU utilization, and the like.

In step S20, the determining module 120 determines the current state of the system resource. The pre-determined determining method includes: determining whether the current state of the system resource is normal according to the preset usage of the local disk, the memory utilization, the first preset threshold of the CPU utilization, and the second preset threshold. When it is monitored that the utilization of one of the system resources is higher than the corresponding second preset threshold, the system resources are judged to be tight, and when the utilization rate of all system resources is monitored to be lower than the corresponding first preset threshold, the system resources are determined. Sufficiently, when it is monitored that the utilization of all system resources is lower than the second preset threshold, it is determined that the system resources are currently in a normal state. It can be understood that the normal state of the system resources includes sufficient system resources, and sufficient system resources are only a special case of normal system resources. For example, the first preset threshold of the local disk is 50%, the second preset threshold is 80%, the first preset threshold of the memory utilization is 40%, and the second preset threshold is 70%. The first preset threshold of the utilization is 40%, and the second preset threshold is 60%. When the local disk utilization is 82%, the memory utilization is 68%, and the CPU utilization is 30%, the system resources are tight. When the local disk utilization is 40%, the memory utilization is 30%, and the CPU utilization is 30%, the system resources are sufficient. When the local disk utilization is 60%, the memory utilization is 68%, and the CPU utilization is 30%, indicating that the system resources are currently in a normal state.

In step S30, when the system resources are normal and the current state of the storage node is normal, the selection module 130 selects a routing algorithm from the preset routing algorithm, and allocates the data packet to the corresponding network transmission according to the selected routing algorithm. Go to the corresponding storage node. The preset routing algorithm includes:

The data transmission method according to the foregoing embodiment, according to monitoring the current state of the storage node and the system resource of the server, selects a routing algorithm from the preset routing algorithm by using a preset selection rule, according to the selected routing algorithm, The data packets are allocated to the corresponding network and transmitted to the corresponding storage nodes, thereby improving data transmission efficiency, reducing data loss during transmission, and improving data transmission stability and reliability.

As shown in FIG. 5, it is a flowchart of the second embodiment of the data transmission method of the present application.

In this embodiment, the data transmission method includes: Step S10 - Step S80. Step S10, step S40, and step S50 are substantially the same as those in the first embodiment, and are not described herein again.

In step S20, the determining module 120 determines whether the current state of the storage node is normal. When the storage node fails, step S30 is performed, and the process ends. When the current state of the storage node is normal, step S40 is performed. When a storage node fails, the data stored in the storage node is often lost or damaged. Therefore, when a storage node is damaged, the failed storage node is automatically repaired.

In step S30, when the storage node fails, the selection module 130 specifies a set of networks for data repair of the failed storage node, and another set of networks is used for data transmission of other storage nodes. For example, when it is monitored that a storage node fails, the selection module 130 automatically allocates a set of networks to perform data repair on the failed storage node, and another set of networks is used to transmit data to other storage nodes. The data repair refers to a method for repairing data in the storage node when the storage node fails to cause data loss, wherein the data repair includes: copy data repair, regeneration code data repair, multi-node collaborative data repair, and the like.

In step S60, the calculation module 140 calculates the average delay time of each group of network data transmissions in real time, and calculates the difference between the delay time of the current transmission data and the average delay time of the network. The delay time refers to a difference between a time when a data packet is currently transmitted to the storage node and a transmission time of a data packet in a normal state. For example, in the normal state, it takes 7 seconds to transmit a data packet from the client 3 to the storage node 2, and if the transmission time is 16 seconds, the delay time is 9 seconds. Statistics and calculations The average of the delay times of all packets transmitted by each group of networks at the current time, which is the average delay time. Assuming an average delay time of 4 seconds, the difference between the current transmission data delay time and the network average delay time is 5 seconds.

In step S70, the determining module 120 determines whether the difference exceeds the first preset value. When the difference exceeds the first preset value, the priority of the network is lowered, and the data transmission task is re-allocated according to the current priorities of the two groups of networks. . Assume that the first preset value is set to 4 seconds and the average delay time is 4 seconds. If the delay time of network A currently transmitting one data packet is 9 seconds, network A is reduced from the first priority to the second priority.

Step S80, when the delay time of the current data transmission of the group of networks exceeds the second preset value, indicating that the network congestion or the data packet has been lost, the notification module 150 notifies the storage node that receives the data that the current data transmission is invalid, and requests another A set of networks retransmits the data to the storage node. Assuming that the second preset value is set to 20 seconds, when the delay time of the network A transmitting certain data to the storage node 2 exceeds 20 seconds, the notification module 150 sends abort information to notify the storage node 2 that the current data transmission is invalid, and requests the network. B retransmits the data to storage node 2.

The data transmission method proposed in this embodiment is constructed by constructing two sets of transmission networks for data transmission, and according to the state of the storage node and system resources, selecting different routing algorithms to allocate the data packets to be transmitted to the two groups of networks, increasing redundancy Backup, prevent data loss, improve the effectiveness and reliability of data transmission.

In addition, the embodiment of the present application further provides a computer readable storage medium, where the computer readable storage medium includes a data transmission program 10, and when the data transmission program 10 is executed by the processor, the following operations are implemented:

Polling algorithm: use two sets of network transmission data packets in turn;

Preferably, the system resources include: local disk utilization, memory utilization, and central processor utilization.

Preferably, the preset determination rule is:

Determining whether the current state of the system resource is normal according to the preset usage of the local disk, the memory utilization, the first preset threshold of the CPU utilization, and the second preset threshold;

When it is monitored that the utilization of one of the system resources is higher than the corresponding second preset threshold, the system resources are judged to be tight, and when the utilization rate of all system resources is monitored to be lower than the corresponding first preset threshold, the system resources are determined. Sufficiently, when it is monitored that the utilization of all system resources is lower than the second preset threshold, it is determined that the system resources are currently in a normal state.

Preferably, the method further comprises:

a switching step: when monitoring system resources of the server is tight, switching to the optimal path algorithm allocates the data packet to the corresponding network and transmits to the corresponding storage node, and when the system resources of the monitored server are sufficient, switching to the The polling algorithm allocates the data packet to the corresponding network. When the system resource of the monitoring server is in a normal state, the shortest queue algorithm is selected to allocate the data packet to the corresponding network.

Preferably, the method further comprises:

Calculation step: calculating the average delay time of each group of network data transmissions in real time, and calculating the difference between the delay time of the current transmission data and the average delay time of the network;

The adjusting step is: determining whether the difference exceeds a first preset value, and when the difference exceeds the first preset value, lowering the priority of the network, and reallocating the data transmission task according to the current priority of the two groups of networks.

Preferably, the method further comprises:

Notification step: when the delay time of the current data transmission of a group of networks exceeds the second preset value, the current data transmission of the storage node that notifies the received data is invalid, and another group of networks is requested to retransmit the data to the storage node.

Preferably, the method further comprises:

Designation step: When monitoring the failure of a storage node, a set of networks is designated for data repair of the failed storage node, and another set of networks is used for data transfer of other storage nodes.

The specific implementation manner of the computer readable storage medium of the present application is substantially the same as the specific implementation manner of the foregoing data transmission method, and details are not described herein again.

The serial numbers of the embodiments of the present application are merely for the description, and do not represent the advantages and disadvantages of the embodiments.

Through the description of the above embodiments, those skilled in the art can clearly understand that the foregoing embodiment method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be through hardware, but in many cases, the former is better. Implementation. Based on such understanding, the technical solution of the present application, which is essential or contributes to the prior art, may be embodied in the form of a software product stored in a storage medium (such as ROM/RAM as described above). , a disk, an optical disk, including a number of instructions for causing a terminal device (which may be a mobile phone, a computer, a server, or a network device, etc.) to perform the methods described in the various embodiments of the present application.

The above is only a preferred embodiment of the present application, and is not intended to limit the scope of the patent application, and the equivalent structure or equivalent process transformations made by the specification and the drawings of the present application, or directly or indirectly applied to other related technical fields. The same is included in the scope of patent protection of this application.

Claims

A data transmission method is applied to a server, and the server is connected to the client through two groups of networks, wherein the method includes:

Monitoring step: monitoring the system resources of the server and the current state of the storage node;

Judgment step: determining whether the current state of the system resource and the storage node is normal;

Selection step: when the system resources are normal and the current state of the storage node is normal, a routing algorithm is selected from the preset routing algorithm, and the data packet is allocated to the corresponding network according to the selected routing algorithm to be transmitted to the corresponding network. The storage node, the preset routing algorithm includes:

Polling algorithm: use two sets of network transmission data packets in turn;

Optimal Path Algorithm: Statistics preset number of network packet transmission is completed each time, denoted as N 1, N 2, calculates the absolute value of N 1 -N 2, when the absolute value is less than N 1 -N 2 When the preset value is used, the two groups of networks are assigned the same priority, and the polling algorithm is used to allocate data packets to the two groups of networks. When the absolute value of N 1 -N 2 is greater than the preset value, the transmission is completed within the preset time. The network with a large number of data packets is allocated the first priority, and the remaining network is assigned the second priority, and the first proportion of the data packet is allocated to the network transmission of the first priority, and the second proportion of the data packet is allocated to the first a second priority network transmission, wherein the first priority is higher than the second priority, and the first ratio is greater than the second ratio;

Shortest Queue Algorithm: counting the number of packets in each group of network transmission has not completed, are referred to as N 1, N 2, calculates the absolute value of N 1 -N 2, when an absolute value of N 1 -N 2 is less than a preset threshold value, Assigning the same priority to the two groups of networks, using the polling algorithm to allocate data packets to the two groups of networks. When the absolute value of N 1 -N 2 is greater than the preset threshold, the number of packets that are not completed for the current transmission is small. The network allocates the first priority, and the remaining network allocates the second priority, and the third proportion of the data packet is allocated to the network transmission of the first priority, and the fourth proportion of the data packet is allocated to another network transmission, where the first The priority is higher than the second priority, and the third ratio is greater than the fourth ratio.
The data transmission method according to claim 1, wherein the system resources include: local disk utilization, memory utilization, and CPU utilization.
The data transmission method according to claim 2, wherein the determining step further comprises:

Determining whether the current state of the system resource is normal according to the preset usage of the local disk, the memory utilization, the first preset threshold of the CPU utilization, and the second preset threshold;

When it is monitored that the utilization of one of the system resources is higher than the corresponding second preset threshold, the system resources are judged to be tight, and when the utilization rate of all system resources is monitored to be lower than the corresponding first preset threshold, the system resources are determined. Sufficiently, when it is monitored that the utilization of all system resources is lower than the second preset threshold, it is determined that the system resources are currently in a normal state.
The data transmission method according to claim 1, wherein the method further comprises:

a switching step: when monitoring system resources of the server is tight, switching to the optimal path algorithm allocates the data packet to the corresponding network and transmits to the corresponding storage node, and when the system resources of the monitored server are sufficient, switching to the The polling algorithm allocates the data packet to the corresponding network and transmits it to the corresponding storage node. When the system resource of the monitoring server is in a normal state, the shortest queue algorithm is selected to allocate the data packet to the corresponding network for transmission to the corresponding storage node.
The data transmission method according to claim 4, wherein the method further comprises:

Calculation step: calculating the average delay time of each group of network data transmissions in real time, and calculating the difference between the delay time of the current transmission data and the average delay time of the network;

The adjusting step is: determining whether the difference exceeds a first preset value, and when the difference exceeds the first preset value, lowering the priority of the network, and reallocating the data transmission task according to the current priority of the two groups of networks.
The data transmission method according to claim 5, wherein the method further comprises:

Notification step: when the delay time of the current data transmission of a group of networks exceeds the second preset value, the current data transmission of the storage node that notifies the received data is invalid, and another group of networks is requested to retransmit the data to the storage node.
The data transmission method according to claim 1, wherein the method further comprises:

Designation step: When monitoring the failure of a storage node, a set of networks is designated for data repair of the failed storage node, and another set of networks is used for data transfer of other storage nodes.
A server, comprising: a memory and a processor, wherein the memory stores a data transmission program, and the data transmission program is executed by the processor, and the following steps can be implemented:

Monitoring step: monitoring the system resources of the server and the current state of the storage node;

Judgment step: determining whether the current state of the system resource and the storage node is normal;

Selection step: when the system resources are normal and the current state of the storage node is normal, a routing algorithm is selected from the preset routing algorithm, and the data packet is allocated to the corresponding network according to the selected routing algorithm to be transmitted to the corresponding network. The storage node, the preset routing algorithm includes:

Polling algorithm: use two sets of network transmission data packets in turn;

Optimal Path Algorithm: Statistics preset number of network packet transmission is completed each time, denoted as N 1, N 2, calculates the absolute value of N 1 -N 2, when the absolute value is less than N 1 -N 2 When the preset value is used, the two groups of networks are assigned the same priority, and the polling algorithm is used to allocate data packets to the two groups of networks. When the absolute value of N 1 -N 2 is greater than the preset value, the transmission is completed within the preset time. The network with a large number of data packets is allocated the first priority, and the remaining network is assigned the second priority, and the first proportion of the data packet is allocated to the network transmission of the first priority, and the second proportion of the data packet is allocated to the first a second priority network transmission, wherein the first priority is higher than the second priority, and the first ratio is greater than the second ratio;

Shortest Queue Algorithm: counting the number of packets in each group of network transmission has not completed, are referred to as N 1, N 2, calculates the absolute value of N 1 -N 2, when an absolute value of N 1 -N 2 is less than a preset threshold value, Assigning the same priority to the two groups of networks, using the polling algorithm to allocate data packets to the two groups of networks. When the absolute value of N 1 -N 2 is greater than the preset threshold, the number of packets that are not completed for the current transmission is small. The network allocates the first priority, and the remaining network allocates the second priority, and the third proportion of the data packet is allocated to the network transmission of the first priority, and the fourth proportion of the data packet is allocated to another network transmission, where the first The priority is higher than the second priority, and the third ratio is greater than the fourth ratio.
The server according to claim 8, wherein the system resources include: local disk utilization, memory utilization, and CPU utilization.
The server according to claim 9, wherein the determining step further comprises:

Determining whether the current state of the system resource is normal according to the preset usage of the local disk, the memory utilization, the first preset threshold of the CPU utilization, and the second preset threshold;

When it is monitored that the utilization of one of the system resources is higher than the corresponding second preset threshold, the system resources are judged to be tight, and when the utilization rate of all system resources is monitored to be lower than the corresponding first preset threshold, the system resources are determined. Sufficiently, when it is monitored that the utilization of all system resources is lower than the second preset threshold, it is determined that the system resources are currently in a normal state.
The server according to claim 8, wherein when said data transmission program is executed by said processor, the following steps are further implemented:

a switching step: when monitoring system resources of the server is tight, switching to the optimal path algorithm allocates the data packet to the corresponding network and transmits to the corresponding storage node, and when the system resources of the monitored server are sufficient, switching to the The polling algorithm allocates the data packet to the corresponding network and transmits it to the corresponding storage node. When the system resource of the monitoring server is in a normal state, the shortest queue algorithm is selected to allocate the data packet to the corresponding network for transmission to the corresponding storage node.
The server according to claim 11, wherein when said data transmission program is executed by said processor, the following steps are further implemented:

Calculation step: calculating the average delay time of each group of network data transmissions in real time, and calculating the difference between the delay time of the current transmission data and the average delay time of the network;

The adjusting step is: determining whether the difference exceeds a first preset value, and when the difference exceeds the first preset value, lowering the priority of the network, and reallocating the data transmission task according to the current priority of the two groups of networks.
The server according to claim 12, wherein when said data transmission program is executed by said processor, the following steps are further implemented:

Notification step: when the delay time of the current data transmission of a group of networks exceeds the second preset value, the current data transmission of the storage node that notifies the received data is invalid, and another group of networks is requested to retransmit the data to the storage node.
The server according to claim 8, wherein when said data transmission program is executed by said processor, the following steps are further implemented:

Designation step: When monitoring the failure of a storage node, a set of networks is designated for data repair of the failed storage node, and another set of networks is used for data transfer of other storage nodes.
A computer readable storage medium, comprising: a data transfer program, wherein when the data transfer program is executed by a processor, the following steps can be implemented:

Monitoring step: monitoring the system resources of the server and the current state of the storage node;

Judgment step: determining whether the current state of the system resource and the storage node is normal;

Selection step: when the system resources are normal and the current state of the storage node is normal, a routing algorithm is selected from the preset routing algorithm, and the data packet is allocated to the corresponding network according to the selected routing algorithm to be transmitted to the corresponding network. The storage node, the preset routing algorithm includes:

Polling algorithm: use two sets of network transmission data packets in turn;

Optimal Path Algorithm: Statistics preset number of network packet transmission is completed each time, denoted as N 1, N 2, calculates the absolute value of N 1 -N 2, when the absolute value is less than N 1 -N 2 When the preset value is used, the two groups of networks are assigned the same priority, and the polling algorithm is used to allocate data packets to the two groups of networks. When the absolute value of N 1 -N 2 is greater than the preset value, the transmission is completed within the preset time. The network with a large number of data packets is allocated the first priority, and the remaining network is assigned the second priority, and the first proportion of the data packet is allocated to the network transmission of the first priority, and the second proportion of the data packet is allocated to the first a second priority network transmission, wherein the first priority is higher than the second priority, and the first ratio is greater than the second ratio;

Shortest Queue Algorithm: counting the number of packets in each group of network transmission has not completed, are referred to as N 1, N 2, calculates the absolute value of N 1 -N 2, when an absolute value of N 1 -N 2 is less than a preset threshold value, Assigning the same priority to the two groups of networks, using the polling algorithm to allocate data packets to the two groups of networks. When the absolute value of N 1 -N 2 is greater than the preset threshold, the number of packets that are not completed for the current transmission is small. The network allocates the first priority, and the remaining network allocates the second priority, and the third proportion of the data packet is allocated to the network transmission of the first priority, and the fourth proportion of the data packet is allocated to another network transmission, where the first The priority is higher than the second priority, and the third ratio is greater than the fourth ratio.
The computer readable storage medium according to claim 15, wherein the system resources comprise: local disk utilization, memory utilization, and CPU utilization.
The computer readable storage medium according to claim 16, wherein the determining step further comprises:

Determining whether the current state of the system resource is normal according to the preset usage of the local disk, the memory utilization, the first preset threshold of the CPU utilization, and the second preset threshold;

When it is monitored that the utilization of one of the system resources is higher than the corresponding second preset threshold, the system resources are judged to be tight, and when the utilization rate of all system resources is monitored to be lower than the corresponding first preset threshold, the system resources are determined. Sufficiently, when it is monitored that the utilization of all system resources is lower than the second preset threshold, it is determined that the system resources are currently in a normal state.
The computer readable storage medium according to claim 15, wherein when said data transfer program is executed by said processor, the following steps are further implemented:

a switching step: when monitoring system resources of the server is tight, switching to the optimal path algorithm allocates the data packet to the corresponding network and transmits to the corresponding storage node, and when the system resources of the monitored server are sufficient, switching to the The polling algorithm allocates the data packet to the corresponding network and transmits it to the corresponding storage node. When the system resource of the monitoring server is in a normal state, the shortest queue algorithm is selected to allocate the data packet to the corresponding network for transmission to the corresponding storage node.
The computer readable storage medium according to claim 18, wherein when said data transfer program is executed by said processor, the following steps are further implemented:

Calculation step: calculating the average delay time of each group of network data transmissions in real time, and calculating the difference between the delay time of the current transmission data and the average delay time of the network;

The adjusting step is: determining whether the difference exceeds a first preset value, and when the difference exceeds the first preset value, lowering the priority of the network, and reallocating the data transmission task according to the current priority of the two groups of networks.
The computer readable storage medium according to claim 15, wherein when said data transfer program is executed by said processor, the following steps are further implemented:

Designation step: When monitoring the failure of a storage node, a set of networks is designated for data repair of the failed storage node, and another set of networks is used for data transfer of other storage nodes.