CN110022268B - Data transmission control method, device and storage medium - Google Patents

Data transmission control method, device and storage medium Download PDF

Info

Publication number
CN110022268B
CN110022268B CN201810017412.4A CN201810017412A CN110022268B CN 110022268 B CN110022268 B CN 110022268B CN 201810017412 A CN201810017412 A CN 201810017412A CN 110022268 B CN110022268 B CN 110022268B
Authority
CN
China
Prior art keywords
data
cache
preset
storage space
input
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201810017412.4A
Other languages
Chinese (zh)
Other versions
CN110022268A (en
Inventor
蔡舒
孙彬
李�杰
王建
位爱伶
吴先斌
屈宇鹏
王利虎
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tencent Technology Shenzhen Co Ltd
Original Assignee
Tencent Technology Shenzhen Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tencent Technology Shenzhen Co Ltd filed Critical Tencent Technology Shenzhen Co Ltd
Priority to CN201810017412.4A priority Critical patent/CN110022268B/en
Publication of CN110022268A publication Critical patent/CN110022268A/en
Application granted granted Critical
Publication of CN110022268B publication Critical patent/CN110022268B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L47/00Traffic control in data switching networks
    • H04L47/10Flow control; Congestion control
    • H04L47/12Avoiding congestion; Recovering from congestion
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L49/00Packet switching elements
    • H04L49/90Buffering arrangements
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/50Network services
    • H04L67/56Provisioning of proxy services
    • H04L67/568Storing data temporarily at an intermediate stage, e.g. caching

Abstract

The embodiment of the invention discloses a data transmission control method, a data transmission control device and a storage medium, wherein the embodiment of the invention obtains the used storage space of a cache; when the used storage space of the cache is larger than a preset threshold value, acquiring transmission parameters of data transmission; acquiring a congestion control threshold according to the transmission parameters; and when the input time delay in the transmission parameters is larger than the congestion control threshold, storing the transmitted data into the cache according to a first preset input speed. Therefore, the input speed of data storage to the cache is automatically adjusted according to the transmission parameters, and the aim of avoiding network congestion is fulfilled; the scheme can ensure the reliability of data transmission and improve the efficiency of data transmission.

Description

Data transmission control method, device and storage medium
Technical Field
The present invention relates to the field of communications technologies, and in particular, to a data transmission control method, apparatus, and storage medium.
Background
In a communication system, in the process of data transmission, data generally needs to be cached and forwarded through an intermediate bridge, for example, when data is transmitted from a client to a server, a storage gateway needs to be used as a bridge for data transmission, the storage gateway receives data sent by the client, stores the received data in the cache, and then uploads the data to the server.
In the prior art, when a storage gateway serving as an intermediate bridge receives data sent by a client for a certain time and the speed of receiving the data sent by the client is higher than the uploading speed of uploading the data to a server, after a period of time, the cache of the data stored by the storage gateway is easily filled, the phenomenon of network congestion rapidly occurs, and the data cannot be stored again at the moment.
In the research and practice process of the prior art, the inventor of the present invention finds that, because the cache of the storage gateway storing data is easy to generate network congestion phenomenon and cannot store data, the data transmission is affected, and the efficiency and reliability of the data transmission are reduced.
Disclosure of Invention
The embodiment of the invention provides a data transmission control method, a data transmission control device and a storage medium, and aims to improve the efficiency and reliability of data transmission.
In order to solve the above technical problems, embodiments of the present invention provide the following technical solutions:
a data transmission control method, comprising:
obtaining the used storage space of the cache;
when the used storage space of the cache is larger than a preset threshold value, acquiring transmission parameters of data transmission;
acquiring a congestion control threshold according to the transmission parameters;
and when the input time delay in the transmission parameters is larger than the congestion control threshold, storing the transmitted data into the cache according to a first preset input speed.
A data transmission control apparatus comprising:
a storage space acquiring unit for acquiring a used storage space of the cache;
the parameter acquisition unit is used for acquiring transmission parameters of data transmission when the used storage space of the cache is larger than a preset threshold value;
a threshold obtaining unit, configured to obtain a congestion control threshold according to the transmission parameter;
and the first storage unit is used for storing the transmitted data to the cache according to a first preset input speed when the input time delay in the transmission parameters is greater than the congestion control threshold.
A storage medium storing a plurality of instructions, the instructions being suitable for being loaded by a processor to execute the steps of the above data transmission control method.
The embodiment of the invention can judge whether the network is congested or not by obtaining the used storage space of the cache, and when the network congestion is determined, for example, when the used storage space of the cache is larger than a preset threshold (the storage space of the cache is about to be occupied, and the network is relatively congested), obtain the transmission parameter of data transmission, and obtain the congestion control threshold according to the transmission parameter, then, when the input delay in the transmission parameter is larger than the congestion control threshold, control the input speed to slowly increase, and store the transmitted data into the cache according to the first preset input speed, thereby realizing the purpose of automatically adjusting the input speed of data storage into the cache according to the transmission parameter and avoiding the network congestion; the scheme can ensure the reliability of data transmission and improve the efficiency of data transmission.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
Fig. 1 is a schematic view of a data transmission control system according to an embodiment of the present invention;
fig. 2 is a schematic flow chart of a data transmission control method according to an embodiment of the present invention;
fig. 3 is a system architecture diagram of a storage gateway for data transmission control according to an embodiment of the present invention;
fig. 4 is another flow chart of a data transmission control method according to an embodiment of the present invention;
fig. 5 is a schematic diagram of a storage gateway performing data transmission control through a congestion control algorithm according to an embodiment of the present invention;
fig. 6 is a schematic structural diagram of a data transmission control apparatus according to an embodiment of the present invention;
fig. 7 is another schematic structural diagram of a data transmission control device according to an embodiment of the present invention;
fig. 8 is another schematic structural diagram of a data transmission control device according to an embodiment of the present invention;
fig. 9 is another schematic structural diagram of a data transmission control device according to an embodiment of the present invention;
fig. 10 is a schematic structural diagram of a server according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
In the description that follows, specific embodiments of the present invention are described with reference to steps and symbols executed by one or more computers, unless otherwise indicated. Accordingly, these steps and operations will be referred to, several times, as being performed by a computer, the computer performing operations involving a processing unit of the computer in electronic signals representing data in a structured form. This operation transforms the data or maintains it at locations in the computer's memory system, which may be reconfigured or otherwise altered in a manner well known to those skilled in the art. The data maintains a data structure that is a physical location of the memory that has particular characteristics defined by the data format. However, while the principles of the invention have been described in language specific to above, it is not intended to be limited to the specific form set forth herein, but on the contrary, it is to be understood that various steps and operations described hereinafter may be implemented in hardware.
The embodiment of the invention provides a data transmission control method, a data transmission control device and a storage medium.
Referring to fig. 1, fig. 1 is a schematic view of a scenario of a data transmission control system according to an embodiment of the present invention, the data transmission control system may comprise data transmission control means, which may in particular be integrated in a server, for example, the data transmission control device can be integrated in a storage gateway in the server, is mainly used for acquiring the used storage space of the cache, wherein, the obtaining time of the used storage space of the cache can be real-time or obtained at preset time intervals, or, when receiving the input request sent by the client, for example, the storage gateway may receive input request 1 sent by client a, input request 2 sent by client B, input request 3 sent by client C, and the like, it can be understood that the specific number of the clients and the input requests can be flexibly set according to actual situations. When the used storage space of the cache is larger than the preset threshold, the storage space of the cache is about to be occupied, the network is congested, the transmission parameters of data transmission are acquired at the moment, and the congestion control threshold is acquired according to the transmission parameters. And when the input time delay in the transmission parameters is larger than the congestion control threshold, storing the transmitted data into a buffer according to a first preset input speed, and the like.
It should be noted that the scenario diagram of the data transmission control system shown in fig. 1 is merely an example, and the data transmission control system and the scenario described in the embodiment of the present invention are for more clearly illustrating the technical solution of the embodiment of the present invention, and do not form a limitation on the technical solution provided in the embodiment of the present invention.
The following are detailed below.
In this embodiment, a description will be made from the perspective of a data transmission control apparatus, which may be specifically integrated in a network device such as a server or a gateway.
A data transmission control method, comprising: obtaining the used storage space of the cache; when the used storage space of the cache is larger than a preset threshold value, acquiring transmission parameters of data transmission; acquiring a congestion control threshold according to the transmission parameters; and when the input time delay in the transmission parameters is greater than the congestion control threshold, storing the transmitted data into a cache according to a first preset input speed.
Referring to fig. 2, fig. 2 is a flowchart illustrating a data transmission control method according to a first embodiment of the present invention. The data transmission control method comprises the following steps:
in step S101, the used storage space of the cache is acquired.
In this embodiment, the data transmission control device dynamically controls congestion mainly according to the current data transmission situation. For example, information such as a used memory space and a transmission parameter of the buffer is acquired, and congestion control is dynamically performed. The congestion control can be used to adjust the data transmission amount in the network, so that the data transmission amount approaches the load capacity of the current network, thereby avoiding the network performance degradation, and the like.
Optionally, in an embodiment, the step of the data transmission control device obtaining the used storage space of the cache may include: receiving an input request carrying data sent by a client; and acquiring the used storage space of the cache according to the input request.
Specifically, the data transmission control device may receive an input request sent by a client, where the input request may carry data to be transmitted, request information, and the like, the data may be data to be written into a cache, and the input request may include a write request or a snapshot request, and the like. When the data transmission control device receives the input request, the used storage space of the cache is obtained, that is, the data transmission control device detects the used storage space of the cache, wherein the cache is a cache preset by the data transmission control device itself, for example, a write cache or a snapshot cache, and the cache may only include one storage space or may include a plurality of storage spaces, and the sizes of the plurality of storage spaces may be consistent or inconsistent.
Optionally, in another embodiment, the step of the data transmission control device obtaining the used storage space of the cache may include: and the data transmission control device acquires the used storage space of the cache in real time or at preset time intervals. The preset time can be flexibly set according to the actual situation, and the specific value is not limited here. It is understood that the cache may be obtained in other manners, such as in a manner of obtaining the used storage space.
In step S102, when the used storage space of the buffer is greater than the preset threshold, the transmission parameter of the data transmission is obtained.
After the used storage space of the cache is obtained, the data transmission control device judges whether the used storage space of the cache is larger than a preset threshold value, wherein the preset threshold value can be flexibly set according to actual needs, and specific values are not limited here. For example, when the cache includes one storage space, the preset threshold may be set to a space size of 90% of the storage space of the cache; when the cache includes the first storage space and the second storage space, the preset threshold may be set to a space size of the first storage space of the cache, and so on.
When the used storage space of the buffer is less than or equal to the preset threshold, which indicates that the current network is not congested, the data transmission control device may maintain the current transmission parameter to transmit the data, for example, may maintain the current input speed to store the data in the buffer. When the used storage space of the buffer is larger than the preset threshold, which indicates that the current network is congested, the data transmission control device may perform the step of acquiring the transmission parameter of the data transmission.
The transmission parameter for acquiring data transmission may be a transmission parameter for acquiring data to be written into the buffer (i.e. stored into the buffer), that is, the data may be data to be written into the buffer. The transmission parameters may include an input delay when data is written into the buffer, an input data time when data is written into the buffer, the number of all input requests received, a speed increase factor, an average delay when data is written at a high speed, and the like.
In step S103, a congestion control threshold is obtained according to the transmission parameter.
After obtaining the transmission parameter, the data transmission control device may obtain the congestion control threshold according to the transmission parameter, and in some embodiments, the step of obtaining, by the data transmission control device, the congestion control threshold according to the transmission parameter may include:
extracting input data time and the number of all received input requests from the transmission parameters; calculating the average time delay of the input data according to the input data time and the number of all input requests; and calculating the congestion control threshold according to the average time delay.
And when the used storage space of the cache is larger than a preset threshold, triggering a congestion control algorithm by the data transmission control device, wherein the congestion control algorithm is mainly used for acquiring a congestion control threshold according to the transmission parameters, adjusting the input speed according to the congestion control threshold, and storing the data into the cache according to the adjusted input speed. Specifically, since the acquired transmission parameters include the time of inputting data when data is written into the buffer, the number of all received input requests, and the like, the data transmission control device may extract the time of inputting data from the transmission parameters, and extract the number of all received input requests from the transmission parameters, and then calculate the average time delay of the input data according to the time of inputting data and the number of all input requests, and the calculation formula is as follows:
avg_delay=duration/write_total;
where avg _ delay represents the average delay, duration represents the input data time, and write _ total represents the number of all input requests received.
Secondly, calculating the congestion control threshold according to the average time delay, and optionally, the step of calculating the congestion control threshold according to the average time delay by the data transmission control device may include:
acquiring the calculation times of a congestion control threshold; calculating a delay control factor according to the number of times of calculation; and calculating the congestion control threshold according to the delay control factor and the average time delay. The calculation times of the congestion control threshold are the times of triggering the congestion control algorithm. The calculation formula is as follows:
write_thres=avg_delay*(1+α)
α=exp(n,retreat_cnt)
where write _ thres represents a congestion control threshold, avg _ delay represents an average delay, α represents a delay control factor, retry _ cnt represents the number of times of calculation of the congestion control threshold (i.e., the number of times of triggering of a congestion control algorithm statistically obtained from the start of the data transmission control apparatus), exp represents an exponential function with a natural constant e as the base, and n may take a value of 1/4, or other values.
At this time, the obtained congestion control threshold write _ thres may be an approximate value of an average speed of data transmission, and as the number of times retreat _ cnt triggered by the congestion control algorithm approaches infinity, the delay control factor α approaches zero, which means that the larger the number of times retreat _ cnt triggered by the congestion control algorithm is, the more accurate statistics of transmission parameters of historical data transmission are, so that the congestion control threshold write _ thres approaches an average value of the statistics.
In step S104, when the input delay in the transmission parameter is greater than the congestion control threshold, the transmitted data is stored in the buffer according to a first preset input speed.
When the used memory space of the cache is larger than the preset threshold, the congestion control algorithm is triggered, and the input speed of the current client may be reduced to a very low speed value, for example, only 3 or 4 input requests are received per second, i.e. the input delay write _ delay may be a relatively large value.
The data transmission control device can extract the input delay from the transmission parameters, after the congestion control threshold is obtained, the data transmission control device can judge whether the input delay is greater than the congestion control threshold, when the input delay write _ thres is greater than the congestion control threshold write _ thres, the current network is congested, a slow increase algorithm is started, namely, the transmitted data is stored into a cache according to a first preset input speed, and the congestion is avoided better under the condition of solving the congestion or low speed of the network.
The input delay write _ delay is an additional delay for each input request, and the larger the input delay write _ delay is, the smaller the number of input requests received by the data transmission control device per second is. The data transmission control device can adjust the input speed by adjusting the value of the input time delay, thereby realizing congestion avoidance. In addition, the larger the speed increase factor write _ inc, the faster the input delay write _ delay decreases, and the faster the input speed increases.
It should be noted that the speed increase factor write _ inc may be set to a relatively small value, so that the input delay write _ delay is slowly reduced and the input speed of storing data into the buffer is slowly increased. Optionally, the first preset input speed increases more gradually to achieve a smooth effect, the speed curve may tend to a straight line, and the first preset input speed may be set to a speed at which the input speed can be written into the cache after a preset time (for example, 20 minutes or 30 minutes); or setting the first predetermined input speed as a predetermined multiple (e.g., 2 times or 3 times, etc.) at a predetermined time interval before the adjustment of the input speed is triggered, and then writing the corresponding speed of the buffer. Of course, the first preset input speed may also be flexibly set according to actual needs, and the specific value is not limited here.
In some embodiments, the step of the data transmission control device storing the transmitted data into the buffer memory at the first preset input speed may include:
acquiring data corresponding to the input request; calculating the data transmission speed required by the used cache space reaching the size of the preset space within a preset time period to obtain a first preset input speed; and storing the data into a cache according to the first preset input speed.
Specifically, the data transmission control device may obtain, according to a received input request sent by the client, data corresponding to the input request, and the cached used storage space reaches a data transmission speed required by a preset space size within a preset time period, where the preset time period and the preset space size may be flexibly set according to actual needs, and a specific value is not limited here. For example, it may be the data transfer rate required to write the buffer full of the used memory space of the buffer after 20 minutes. Then, the data transmission speed is set as a first preset input speed, and the data is stored in the cache according to the first preset input speed, so that the utilization rate of the cache can be kept stable, for example, the utilization rate of the cache is kept at 30% -80% all the time, the concurrency number of the network is effectively increased, and the storage space of the cache is fully utilized.
In some embodiments, after the step of acquiring, by the data transmission control apparatus, the congestion control threshold according to the transmission parameter, the data transmission control method may further include:
and when the input time delay in the transmission parameters is less than or equal to the congestion control threshold, storing the data into a cache according to a second preset input speed, wherein the second preset input speed is greater than the first preset input speed.
Specifically, in the process of comparing the input delay with the congestion control threshold, when the input delay write _ delay is less than or equal to the congestion control threshold write _ thres, it indicates that the current network is not congested, and at this time, the fast growth algorithm is started to enter the fast growth stage. In the fast increase stage, the input speed of the data stored in the buffer may increase exponentially, that is, the second preset input speed may be an exponential increase speed, and at this time, the data may be stored in the buffer at the second preset input speed, so that the input delay Write _ delay may quickly reach or be less than the congestion control threshold Write _ thres.
In some embodiments, the cache may include a first storage space and a second storage space, and after the step of obtaining the used storage space of the cache, the data transmission control method may further include:
and when the used storage space of the cache is full of the first storage space of the cache, storing the data into the second storage space according to a third preset input speed, wherein the third preset input speed is less than the first preset input speed.
Specifically, the cache may include one or more storage spaces, and for example, the cache includes a first storage space and a second storage space, where the first storage space is a normal storage space, the normal storage space may provide write data at any time, and the second storage space is an emergency storage space, and the emergency storage space does not write data when the network is not congested, and is only used for providing write data when the network is disconnected when the network is congested.
The data transmission control device may determine whether the used storage space of the cache is full of the first storage space of the cache in the process of detecting the used storage space of the cache, for example, may assign vm _ buf _ full to 1 when the first storage space of the cache is full, and assign vm _ buf _ full to 0 when the first storage space of the cache is not full; or vm _ buff _ full may be assigned to 0 when the first storage space of the cache is fully occupied, vm _ buff _ full may be assigned to 1 when the first storage space of the cache is not fully occupied, and so on.
At this time, the data transmission control device may determine that the first storage space of the cache is occupied by the value of vm _ buff _ full, and when the value of vm _ buff _ full is 1, it indicates that the first storage space of the cache is occupied by the used storage space of the cache, at this time, a slow service algorithm of network disconnection is started, the second storage space is started, and the input speed is adjusted, that is, the data is stored in the second storage space according to the third preset input speed. The third preset input speed may be set to be input for 1 time in 1 second, or the third preset input speed may be adjusted according to the size of the second storage space and information such as a network port, so as to ensure that the second storage space can provide storage service for several hours or several days. Of course, the third preset input speed may be flexibly set according to actual needs, and the specific value is not limited here.
As can be seen from the above, the embodiment of the present invention may determine whether the network is congested by obtaining the used storage space of the cache, and when determining that the network is congested, for example, when the used storage space of the cache is greater than a preset threshold (indicating that the storage space of the cache is about to be occupied and the network is relatively congested), start a congestion control algorithm, obtain a transmission parameter of data transmission, and obtain a congestion control threshold according to the transmission parameter; then, when the input time delay in the transmission parameters is larger than the congestion control threshold, controlling the input speed to slowly increase, and storing the transmitted data into a cache according to a first preset input speed; therefore, the input speed of data storage to the cache is automatically adjusted according to the transmission parameters, and the aim of avoiding network congestion is fulfilled; the scheme can ensure the reliability of data transmission and improve the efficiency of data transmission.
The method described in the above embodiments is further illustrated in detail by way of example.
Taking a storage gateway as an example, please refer to fig. 3, and fig. 3 is a system architecture diagram for controlling data transmission of the storage gateway, where the storage gateway may include a storage gateway module, a storage gateway policy module, and the like, the storage gateway module may include a storage gateway write cache, a storage gateway read cache, and the like, and the system architecture diagram may further include a storage gateway client module, a storage server command module, a storage server data module, and a final storage device, and the like.
The storage gateway client module can be connected to the storage gateway module and is mainly interacted with a storage gateway write cache and a storage gateway read cache of the storage gateway module.
The storage gateway module may be deployed locally to the storage device or in a virtual storage device in a displaced network environment.
The storage gateway write cache is a local storage medium that can be accessed by the storage gateway module and is used for storing data that is written into the storage gateway module but is not uploaded to the final storage device module.
The storage gateway read cache is a local storage medium which can be accessed by the storage gateway module and is used for storing data which are read by the storage gateway module or storing data which are read in advance by the storage gateway module.
The storage server side command module is mainly used for issuing a signaling to the storage gateway module, wherein the signaling can be used for controlling the behavior of the storage gateway module and receiving monitoring data reported by the storage gateway module.
The storage server data module can be deployed in the local of the final storage device and is used for receiving the data transmitted by the storage gateway module and landing the received data to the final storage device module.
The storage gateway policy module may be a module that intelligently determines the real-time transmission rate of the storage gateway module based on a set basic policy, a local network condition, a network historical transmission condition, and the like.
The final storage device module may comprise one or more heterogeneous storage media, primarily for storing data.
In the embodiment, the input speed of the storage gateway client module and the uploading speed of the storage gateway module can be balanced, so that the storage gateway module can automatically reduce the input speed of the storage gateway client module under the condition that the bandwidth between the write cache of the storage gateway and the data module of the storage server is low; when the bandwidth between the storage gateway module and the data module of the storage server is increased, the storage gateway module can automatically increase the input speed of the storage gateway client module; when no network exists between the storage gateway module and the data module of the storage server, the low-speed service can be continuously provided for the storage gateway client module until the network is recovered, and when the network is recovered, the input speed of the storage gateway client module can be rapidly increased, so that the efficiency and the reliability of data transmission are improved.
The data transmission flow of the above system architecture will be described as follows:
firstly, a storage gateway client module can initiate a read request or a write request to a storage gateway module, the storage gateway module receives the read request or the write request, and when the received read request is received, the storage gateway module determines whether to initiate the read request or a pre-read request to a storage server data module according to whether a storage gateway read cache is hit, namely when data to be read is stored in the storage gateway read cache, the storage gateway module can read the data from the storage gateway read cache; when the data to be read is not stored in the storage gateway read cache, the storage gateway module may initiate a read request or a read-ahead request to the storage server data module, and read the data from the storage server data module according to the read request or the read-ahead request.
When a write request or a snapshot request is received, the storage gateway module writes data to be written into a storage gateway write cache, and adds the data into an asynchronous write request queue of the storage gateway module, and the storage gateway module allocates a unique incremental version number to each write request.
The storage gateway module sends the received read request, asynchronous write request or snapshot request to the storage server data module, when the storage server data module receives the read request, asynchronous write request or snapshot request, the storage server data module triggers a snapshot generating algorithm first, records the network transmission condition (including but not limited to transmission parameters such as transmission data amount and transmission delay) of the request, reads data from the final storage device module or writes data into the final storage device module, and after the data is read or written, the final storage device module sends a response corresponding to the read request, asynchronous write request or snapshot request to the storage gateway module through the storage server data module.
And after receiving the response, the storage gateway module records the network transmission condition of the request and the response. And when the response to the read request is received, the storage gateway module returns the read data to the storage gateway client module, and writes the data into the storage gateway read cache according to the cache strategy. When the response to the asynchronous write request is received, the storage gateway module processes the write request in the asynchronous write request queue according to the cache policy.
The signaling transmission flow of the above system architecture will be described as follows:
when the storage gateway strategy module sends a control signaling to the storage server command module, the storage server command module sends the control signaling to the storage gateway module, and after receiving the signaling, the storage gateway module executes corresponding response operation according to the signaling.
The following will explain the data reporting flow of the cache condition and the historical transmission condition of the system architecture:
the storage gateway module can send data such as local cache conditions, historical read request network transmission conditions, historical write request network transmission conditions and the like to the storage server command module, the storage server command module stores the received data, and the data can be used for dynamic policy judgment by the storage gateway policy module, namely the storage gateway policy module can acquire the data from the storage server command module and determine a dynamic policy according to the data.
It should be noted that the system architecture for controlling data transmission of the storage gateway shown in fig. 3 is only an example, and the system architecture, the scenario, and the like described in the embodiment of the present invention are for more clearly illustrating the technical solution of the embodiment of the present invention, and do not form a limitation on the technical solution provided in the embodiment of the present invention.
Referring to fig. 4, fig. 4 is a flowchart illustrating a data transmission control method according to an embodiment of the present invention. The method flow can comprise the following steps:
s201, detecting used storage space of a cache by a storage gateway; if the used storage space of the cache is greater than the first preset threshold, executing step S202; if the used storage space of the cache is larger than a second preset threshold value; step S207 is performed.
The storage gateway module may detect a used storage space of the cache through the storage gateway module, where the storage gateway module may detect the used storage space of the cache when receiving an input request sent by the client; alternatively, the storage gateway module may detect the used storage space of the cache in real time or at preset time intervals.
The input request may carry data to be transmitted, request information, and the like, where the data may be data that needs to be written into a cache, the input request may include a write request or a snapshot request, and the cache may only include one or more storage spaces, and the cache may be a storage gateway write cache.
After obtaining the used storage space of the cache, the storage gateway determines the used storage space of the cache through the storage gateway module, for example, the used storage space of the cache may be compared with a first preset threshold or a second preset threshold, where the first preset threshold may be smaller than the second preset threshold, and values of the first preset threshold and the second preset threshold may be flexibly set according to actual needs, and specific values are not limited here.
S202, storing the transmission parameters of the data acquired by the gateway.
When the used storage space of the cache is smaller than or equal to the first preset threshold, it indicates that the current network is not congested, and at this time, the storage gateway does not need to adjust the input speed, and can maintain the current data transmission.
When the used storage space of the cache is larger than a first preset threshold value, it is indicated that the current network is congested due to more input requests sent by the current storage gateway client module, at this time, the storage gateway can start a congestion control algorithm through the storage gateway module, and the transmission parameters of data needing to be written into the cache are acquired through the congestion control algorithm to adjust the input speed. For example, when the used storage space of the cache is larger than 90% of the space size of the storage space, parameters such as input delay of the client, input data time of the client, the number of all input requests of the client, a speed increase factor, and average delay when the gateway writes at a high speed are obtained.
And S203, the storage gateway acquires the congestion control threshold according to the transmission parameters.
After the transmission parameters are obtained, the storage gateway can extract the input data time and the number of all received input requests from the transmission parameters through a storage gateway module; calculating the average time delay of the input data according to the input data time and the number of all input requests; the calculation formula is as follows: average latency is the input data time/number of all input requests. Then, calculating the congestion control threshold according to the average time delay, specifically, obtaining the number of times of triggering the congestion control algorithm, and then calculating the delay control factor according to the number of times, wherein the calculation formula is as follows: delay control factor exp (1/4, number of congestion control algorithm triggers); and finally, calculating the congestion control threshold according to the delay control factor and the average time delay, wherein the calculation formula is as follows: congestion control threshold is the average time delay (1+ delay control factor).
S204, the storage gateway judges whether the input time delay in the transmission parameters is greater than a congestion control threshold; if yes, go to step S205; if not, go to step S206.
And after the storage gateway obtains the congestion control threshold, comparing the input time delay with the congestion control threshold, and judging whether the input time delay is greater than the congestion control threshold.
S205, the storage gateway stores the data into a cache according to a first preset input speed.
When the input delay is greater than the congestion control threshold, it indicates that the current network is congested, that is, the bandwidth between the storage gateway write cache and the storage server data module is low, the storage gateway starts a slow increase algorithm, that is, the transmitted data is stored into the cache according to a first preset input speed, optionally, the first preset input speed increases more slowly, a speed curve thereof may tend to a straight line, the first preset input speed may be set as a used storage space cached after a preset time period, and an increase speed corresponding to a preset space size is reached, for example, the first preset input speed may be set as a speed at which the input speed can be written to the cache after 20 minutes. It can be understood that the first preset input speed can be flexibly set according to actual needs, and specific values are not limited herein. Through adjusting the input speed to the first preset input speed, the utilization rate of the cache can be kept stable, the storage space of the cache is fully utilized, and the concurrency number of the network is effectively increased.
For example, as shown in fig. 5, fig. 5 is a schematic diagram of the storage gateway performing data transmission control through a congestion control algorithm, that is, the storage gateway detects input performance data of a client in real time through a storage gateway module, where a horizontal axis is time, and a vertical axis is the number of times that the client (i.e., the storage gateway client module) performs input operations in unit time. As can be seen from fig. 5, the storage gateway initially maintains a relatively high input performance (e.g., the storage gateway maintains a relatively high input performance from 0 to 6000 seconds), i.e., the bandwidth between the storage gateway module and the storage server data module is relatively high, at this time, the input speed of the storage gateway client module is raised until the upload speed of the storage gateway does not meet the input requirement of the storage gateway client module (e.g., network congestion occurs from 6001 to 10001 seconds), i.e., the bandwidth between the storage gateway write cache and the storage server data module is relatively low, e.g., the input delay is greater than the congestion control threshold, at this time, the congestion control algorithm is triggered, the input speed to the storage gateway client module is reduced, and after several network jitters, the congestion control algorithm adjusts the input speed of the storage gateway client module to a suitable value, e.g., adjusts the current input speed to the first preset input speed, the input speed of the client and the uploading speed of the storage gateway are dynamically balanced, even if the input speed of the client module of the storage gateway and the uploading speed of the storage gateway module are dynamically balanced (for example, the input speed and the uploading speed are dynamically balanced after 10001 seconds), so that the input of the client can keep higher performance. It should be noted that, from 6001 to 10001 seconds, when there is no network between the storage gateway module and the storage server data module, the storage gateway client module can be continuously provided with slow service for a period of time, and when the network is restored, the input speed of the storage gateway client module can be increased, thereby improving the reliability of data transmission.
And S206, the storage gateway stores the data into the cache according to the second preset input speed.
When the input delay is less than or equal to the congestion control threshold, it indicates that the current network is not congested, and at this time, the storage gateway starts a fast increase algorithm, and stores the data into the cache according to a second preset input speed, for example, the second preset input speed may be a speed that increases exponentially. It can be understood that the second preset input speed can be flexibly set according to actual needs, and specific values are not limited herein.
For example, in fig. 5, after from 0 to 6000 seconds and 10001 seconds, the network congestion phenomenon does not occur, and the bandwidth between the storage gateway module and the storage server data module is higher, at this time, the input speed may be increased, for example, the current input speed is adjusted to the second preset input speed, so as to improve the utilization rate of the cache of the storage gateway.
And S207, the storage gateway stores the data into a cache according to a third preset input speed.
The cache of the storage gateway (i.e. the storage gateway write cache) may include a first storage space and a second storage space, and the like, where the first storage space may provide write data at any time, the second storage space does not write data when the network is not congested, and is only used for providing write data when the network is disconnected when the network is congested. When the used storage space of the cache is larger than a second preset threshold, the phenomenon of network disconnection is indicated, and the second preset threshold can be set to be the size of the first storage space, that is, when the used storage space of the cache occupies the first storage space of the cache, the phenomenon of network disconnection is indicated. At this time, the storage gateway starts a slow service algorithm of network disconnection, starts the second storage space, and simultaneously adjusts the input speed, that is, stores the data into the cached second storage space according to a third preset input speed. The third preset input speed may be set to be input 1 time in 1 second, or the third preset input speed may ensure that the second storage space can provide storage service for several hours or several days, and the like. It can be understood that the third preset input speed can be flexibly set according to actual needs, and specific values are not limited herein.
Due to the arrangement of the second storage space, after the storage gateway is congested, the storage gateway can still provide high-speed service for the client for a long time; when the network is completely disconnected, the local cache can be switched to low-speed service without causing the unavailability of storage service, so that the congestion control cannot be performed if transient network congestion can be solved through fast retransmission; and for a severe network environment, the stability of the storage gateway service can be ensured.
In the embodiment of the invention, based on the congestion control algorithm of the storage gateway, when the network performance is good, high-performance storage service is provided for the storage gateway; when the network is congested or at low speed, the input glitch of the client is smoothed. Under the condition of bad network condition or even network disconnection, the storage gateway can still continuously provide storage service for a long time. When the network performance of the storage gateway is improved, the input performance of the client can be rapidly improved to adapt to bandwidth improvement, the utilization rate of the local cache of the storage gateway is kept stable, and the uploading performance of the storage gateway is improved. Namely, the storage gateway can automatically adjust the input speed under the conditions of network disconnection, network low speed, network congestion and the like of the network for a long time, so that the client side can obtain smoother and more stable storage service. In addition, the congestion control algorithm of the storage gateway is automatically adjusted based on the network state, manual intervention of a user is not needed, and the method is very convenient.
In order to better implement the data transmission control method provided by the embodiment of the present invention, an embodiment of the present invention further provides a device based on the data transmission control method. The terms are the same as those in the above-mentioned data transmission control method, and details of implementation may refer to the description in the method embodiment.
Referring to fig. 6, fig. 6 is a schematic structural diagram of a data transmission control apparatus according to an embodiment of the present invention, where the data transmission control apparatus may include a storage space obtaining unit 301, a parameter obtaining unit 302, a threshold obtaining unit 303, a first storage unit 304, and the like.
The storage space obtaining unit 301 is configured to obtain a used storage space of the cache.
In this embodiment, congestion control is performed dynamically mainly according to the current data transmission situation. For example, information such as a used memory space and a transmission parameter of the buffer is acquired, and congestion control is dynamically performed. The congestion control can be used to adjust the data transmission amount in the network, so that the data transmission amount approaches the load capacity of the current network, thereby avoiding the network performance degradation, and the like.
Optionally, in an embodiment, the storage space obtaining unit 301 is specifically configured to: receiving an input request carrying data sent by a client; and acquiring the used storage space of the cache according to the input request.
Specifically, the storage space obtaining unit 301 may receive an input request sent by a client, where the input request may carry data that needs to be transmitted, request information, and the like, the data may be data that needs to be written into a cache, and the input request may include a write request or a snapshot request, and the like. When receiving the input request, the storage space obtaining unit 301 obtains the used storage space of the cache, that is, the storage space obtaining unit 301 detects the used storage space of the cache, where the cache is a cache preset by the data transmission control apparatus itself, for example, a write cache or a snapshot cache, and the cache may only include one storage space or may include a plurality of storage spaces, and the sizes of the plurality of storage spaces may be consistent or inconsistent.
Optionally, in another embodiment, the storage space obtaining unit 301 obtains the used storage space of the cache specifically for: and acquiring the used storage space of the cache in real time or at preset time intervals. The preset time can be flexibly set according to the actual situation, and the specific value is not limited here. It is understood that the cache may be obtained in other manners, such as in a manner of obtaining the used storage space.
A parameter obtaining unit 302, configured to obtain a transmission parameter for data transmission when the used storage space of the buffer is greater than a preset threshold.
After the used cache storage space is obtained, the parameter obtaining unit 302 determines whether the used cache storage space is greater than a preset threshold, where the preset threshold may be flexibly set according to actual needs, and a specific value is not limited here. For example, when the cache includes one storage space, the preset threshold may be set to a space size of 90% of the storage space of the cache; when the cache includes the first storage space and the second storage space, the preset threshold may be set to a space size of the first storage space of the cache, and so on.
When the used storage space of the buffer is less than or equal to the preset threshold, which indicates that the current network is not congested, the first storage unit 304 may maintain the current transmission parameter to transmit the data, for example, may maintain the current input speed to store the data into the buffer. When the used storage space of the buffer is larger than the preset threshold, which indicates that the current network is congested, the parameter obtaining unit 302 may perform the step of obtaining the transmission parameter of the data transmission.
The transmission parameter for acquiring data transmission may be a transmission parameter for acquiring data to be written into the buffer (i.e. stored into the buffer), that is, the data may be data to be written into the buffer. The transmission parameters may include an input delay when data is written into the buffer, an input data time when data is written into the buffer, the number of all received input requests, a speed increase factor, an average delay when data is written at a high speed, and the like.
A threshold obtaining unit 303, configured to obtain a congestion control threshold according to the transmission parameter.
After obtaining the transmission parameter, the threshold obtaining unit 303 may obtain the congestion control threshold according to the transmission parameter, and in some embodiments, as shown in fig. 7, the threshold obtaining unit 303 may include:
an extracting subunit 3031, configured to extract the input data time and the number of all received input requests from the transmission parameters;
a delay calculating subunit 3032, configured to calculate an average delay of the input data according to the input data time and the number of all input requests;
and a threshold calculating subunit 3033, configured to calculate the congestion control threshold according to the average time delay.
When the used storage space of the cache is larger than a preset threshold, the threshold obtaining unit 303 triggers a congestion control algorithm, where the congestion control algorithm is mainly used to obtain a congestion control threshold according to the transmission parameter, adjust the input speed according to the congestion control threshold, and store the data into the cache according to the adjusted input speed. Specifically, since the acquired transmission parameters include the time of inputting data when data is written into the buffer, the number of all received input requests, and the like, the extracting subunit 3031 may extract the time of inputting data from the transmission parameters, and extract the number of all received input requests from the transmission parameters, and then the delay calculating subunit 3032 calculates the average delay of the input data according to the time of inputting data and the number of all input requests, and the calculation formula is as follows:
avg_delay=duration/write_total;
where avg _ delay represents the average delay, duration represents the input data time, and write _ total represents the number of all input requests.
Secondly, the threshold calculation subunit 3033 calculates the congestion control threshold according to the average time delay, and optionally, the threshold calculation subunit 3033 is specifically configured to: acquiring the calculation times of a congestion control threshold; calculating a delay control factor according to the number of times of calculation; and calculating the congestion control threshold according to the delay control factor and the average time delay.
The calculation times of the congestion control threshold are the times of triggering the congestion control algorithm. The calculation formula is as follows:
write_thres=avg_delay*(1+α)
α=exp(n,retreat_cnt)
where write _ thres represents a congestion control threshold, avg _ delay represents an average delay, α represents a delay control factor, retry _ cnt represents the number of times of calculation of the congestion control threshold (i.e., the number of times of triggering of a congestion control algorithm statistically obtained from the start of the data transmission control apparatus), exp represents an exponential function with a natural constant e as the base, and n may take a value of 1/4, or other values.
At this time, the obtained congestion control threshold write _ thres may be an approximate value of an average speed of data transmission, and as the number of times retreat _ cnt triggered by the congestion control algorithm approaches infinity, the delay control factor α approaches zero, which means that the larger the number of times retreat _ cnt triggered by the congestion control algorithm is, the more accurate statistics of transmission parameters of historical data transmission are, so that the congestion control threshold write _ thres approaches an average value of the statistics.
A first storing unit 304, configured to store the transmitted data into a buffer according to a first preset input speed when the input delay in the transmission parameter is greater than the congestion control threshold.
When the congestion control algorithm is triggered, the input speed of the current client may be reduced to a very low speed value, for example, only 3 or 4 input requests are received per second, i.e., the input delay write _ delay may be a relatively large value.
Because the transmission parameter includes the input delay, the first storage unit 304 may extract the input delay from the transmission parameter, after obtaining the congestion control threshold, the first storage unit 304 may determine whether the input delay is greater than the congestion control threshold, and when the input delay write _ thres is greater than the congestion control threshold write _ thres, it indicates that the current network is congested, and starts a slow-growth algorithm, that is, the transmitted data is stored in the cache according to the first preset input speed, so that the congestion is better avoided under the condition of solving the congestion or low speed of the network.
The input delay write _ delay is an additional delay for each input request, and the larger the input delay write _ delay is, the smaller the number of input requests received by the data transmission control device per second is. The first storage unit 304 may adjust the input speed by adjusting the value of the input delay, thereby implementing congestion avoidance. In addition, the larger the speed increase factor write _ inc, the faster the input delay write _ delay decreases, and the faster the input speed increases.
It should be noted that the speed increase factor write _ inc may be set to a relatively small value, so that the input delay write _ delay is slowly reduced and the input speed of storing data into the buffer is slowly increased. Optionally, the first preset input speed increases more gradually to achieve a smooth effect, the speed curve may tend to a straight line, and the first preset input speed may be set to a speed at which the input speed can be written into the cache after a preset time (for example, 20 minutes or 30 minutes); or setting the first predetermined input speed as a predetermined multiple (e.g., 2 times or 3 times, etc.) at a predetermined time interval before the adjustment of the input speed is triggered, and then writing the corresponding speed of the buffer. Of course, the first preset input speed may also be flexibly set according to actual needs, and the specific value is not limited here.
In some embodiments, the first storage unit 304 is specifically configured to:
acquiring data corresponding to the input request; calculating the data transmission speed required by the used cache space reaching the size of the preset space within a preset time period to obtain a first preset input speed; and storing the data into a cache according to the first preset input speed.
Specifically, the first storage unit 304 may obtain, according to a received input request sent by the client, data corresponding to the input request, and a data transmission speed required for a cached used storage space to reach a preset space size within a preset time period, where the preset time period and the preset space size may be flexibly set according to actual needs, and a specific value is not limited here. For example, it may be a data transfer rate required to write the buffer full of the used memory space of the buffer after 20 minutes. Then, the first storage unit 304 sets the data transmission speed to the first preset input speed, and stores the data into the cache according to the first preset input speed, so that the utilization rate of the cache can be kept stable, for example, the utilization rate of the cache is always kept at 30% to 80%, the concurrency number of the network is effectively increased, and the storage space of the cache is fully utilized.
In some embodiments, as shown in fig. 8, the data transmission control device further includes:
a second storage unit 305, configured to store the data into the buffer at a second preset input speed when the input delay in the transmission parameter is less than or equal to the congestion control threshold, where the second preset input speed is greater than the first preset input speed.
Specifically, in the process of comparing the input delay with the congestion control threshold, when the input delay write _ delay is less than or equal to the congestion control threshold write _ thres, it indicates that the current network is not congested, and at this time, the fast growth algorithm is started to enter the fast growth stage. In the fast increase stage, the input speed of the data to be stored into the buffer memory may increase exponentially, that is, the second preset input speed may be an exponential increase speed, and at this time, the second storage unit 305 may store the data into the buffer memory at the second preset input speed, so that the input delay Write _ delay may quickly reach or be less than the congestion control threshold Write _ thres.
In some embodiments, as shown in fig. 9, the buffer may include a first storage space, a second storage space, and the like, and the data transmission control device further includes:
a third storage unit 306, configured to store data into the second storage space according to a third preset input speed when the used storage space of the cache fills up the first storage space of the cache, where the third preset input speed is smaller than the first preset input speed.
Specifically, the cache may include one or more storage spaces, and for example, the cache includes a first storage space and a second storage space, where the first storage space is a normal storage space, the normal storage space may provide write data at any time, and the second storage space is an emergency storage space, and the emergency storage space does not write data when the network is not congested, and is only used for providing write data when the network is disconnected when the network is congested.
The third storage unit 306 may determine whether the used storage space of the cache is full of the first storage space of the cache in the process of detecting the used storage space of the cache, for example, vm _ buf _ full may be assigned to 1 when the first storage space of the cache is full, and vm _ buf _ full may be assigned to 0 when the first storage space of the cache is not full; or vm _ buff _ full may be assigned to 0 when the first storage space of the cache is fully occupied, vm _ buff _ full may be assigned to 1 when the first storage space of the cache is not fully occupied, and so on.
At this time, the third storage unit 306 may determine a value of vm _ buff _ full to determine whether the first storage space of the cache is fully occupied, and when the value of vm _ buff _ full is 1, it indicates that the first storage space of the cache is fully occupied by the used storage space of the cache, at this time, a slow service algorithm of network disconnection is started, the second storage space is enabled, and the input speed is adjusted, that is, data is stored in the second storage space according to a third preset input speed. The third preset input speed may be set to be input for 1 time in 1 second, or the third preset input speed may be adjusted according to the size of the second storage space and information such as a network port, so as to ensure that the second storage space can provide storage service for several hours or several days. Of course, the third preset input speed may be flexibly set according to actual needs, and the specific value is not limited here.
As can be seen from the above, in the embodiment of the present invention, the storage space obtaining unit 301 may obtain the used storage space of the cache to determine whether the network is congested, and when determining that the network is congested, for example, when the used storage space of the cache is greater than a preset threshold (indicating that the storage space of the cache is to be occupied and the network is congested), start a congestion control algorithm, obtain the transmission parameter of data transmission through the parameter obtaining unit 302, and obtain the congestion control threshold according to the transmission parameter by the threshold obtaining unit 303; then, when the input delay in the transmission parameter is greater than the congestion control threshold, the first storage unit 304 controls the input speed to increase slowly, and stores the transmitted data into the cache according to a first preset input speed; therefore, the input speed of data storage to the cache is automatically adjusted according to the transmission parameters, and the aim of avoiding network congestion is fulfilled; the scheme can ensure the reliability of data transmission and improve the efficiency of data transmission.
An embodiment of the present invention further provides a server, as shown in fig. 10, which shows a schematic structural diagram of the server according to the embodiment of the present invention, specifically:
the server may include components such as a processor 401 of one or more processing cores, memory 402 of one or more computer-readable storage media, a power supply 403, and an input unit 404. Those skilled in the art will appreciate that the server architecture shown in FIG. 10 is not meant to be limiting, and may include more or fewer components than those shown, or some components may be combined, or a different arrangement of components. Wherein:
the processor 401 is a control center of the server, connects various parts of the entire server using various interfaces and lines, and performs various functions of the server and processes data by running or executing software programs and/or modules stored in the memory 402 and calling data stored in the memory 402, thereby performing overall monitoring of the server. Optionally, processor 401 may include one or more processing cores; preferably, the processor 401 may integrate an application processor, which mainly handles operating systems, user interfaces, application programs, etc., and a modem processor, which mainly handles wireless communications. It will be appreciated that the modem processor described above may not be integrated into the processor 401.
The memory 402 may be used to store software programs and modules, and the processor 401 executes various functional applications and data processing by operating the software programs and modules stored in the memory 402. The memory 402 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application program required by at least one function (such as a sound playing function, an image playing function, etc.), and the like; the storage data area may store data created according to the use of the server, and the like. Further, the memory 402 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid state storage device. Accordingly, the memory 402 may also include a memory controller to provide the processor 401 access to the memory 402.
The server further includes a power supply 403 for supplying power to each component, and preferably, the power supply 403 may be logically connected to the processor 401 through a power management system, so as to implement functions of managing charging, discharging, and power consumption through the power management system. The power supply 403 may also include any component of one or more dc or ac power sources, recharging systems, power failure detection circuitry, power converters or inverters, power status indicators, and the like.
The server may also include an input unit 404, the input unit 404 being operable to receive input numeric or character information and to generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.
Although not shown, the server may further include a display unit and the like, which will not be described in detail herein. Specifically, in this embodiment, the processor 401 in the server loads the executable file corresponding to the process of one or more application programs into the memory 402 according to the following instructions, and the processor 401 runs the application program stored in the memory 402, thereby implementing various functions as follows:
obtaining the used storage space of the cache; when the used storage space of the cache is larger than a preset threshold value, acquiring transmission parameters of data transmission; acquiring a congestion control threshold according to the transmission parameters; and when the input time delay in the transmission parameters is greater than the congestion control threshold, storing the transmitted data into a cache according to a first preset input speed.
Optionally, after the step of obtaining the congestion control threshold according to the transmission parameter, the data transmission control method further includes: and when the input time delay in the transmission parameters is less than or equal to the congestion control threshold, storing the data into a cache according to a second preset input speed, wherein the second preset input speed is greater than the first preset input speed.
Optionally, the cache may include a first storage space and a second storage space, and after the step of obtaining the used storage space of the cache, the data transmission control method further includes: and when the used storage space of the cache is full of the first storage space of the cache, storing the data into the second storage space according to a third preset input speed, wherein the third preset input speed is less than the first preset input speed.
In the above embodiments, the descriptions of the embodiments have respective emphasis, and parts that are not described in detail in a certain embodiment may refer to the above detailed description of the data transmission control method, and are not described herein again.
It will be understood by those skilled in the art that all or part of the steps of the methods of the above embodiments may be performed by instructions or by associated hardware controlled by the instructions, which may be stored in a computer readable storage medium and loaded and executed by a processor.
To this end, the present invention provides a storage medium, in which a plurality of instructions are stored, where the instructions can be loaded by a processor to execute the steps in any one of the data transmission control methods provided by the embodiments of the present invention. For example, the instructions may perform the steps of:
obtaining the used storage space of the cache; when the used storage space of the cache is larger than a preset threshold value, acquiring transmission parameters of data transmission; acquiring a congestion control threshold according to the transmission parameters; and when the input time delay in the transmission parameters is greater than the congestion control threshold, storing the transmitted data into a cache according to a first preset input speed.
The above operations can be implemented in the foregoing embodiments, and are not described in detail herein.
Wherein the storage medium may include: read Only Memory (ROM), Random Access Memory (RAM), magnetic or optical disks, and the like.
Since the instructions stored in the storage medium can execute the steps in any data transmission control method provided in the embodiments of the present invention, the beneficial effects that can be achieved by any data transmission control method provided in the embodiments of the present invention can be achieved, which are detailed in the foregoing embodiments and will not be described again here.
The data transmission control method, the data transmission control device and the storage medium provided by the embodiments of the present invention are described in detail above, and a specific example is applied in the description to explain the principle and the implementation of the present invention, and the description of the above embodiments is only used to help understanding the method and the core idea of the present invention; meanwhile, for those skilled in the art, according to the idea of the present invention, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present invention.

Claims (15)

1. A data transmission control method, comprising:
acquiring a used storage space of a cache of a storage gateway;
when the used storage space of the cache is larger than a preset threshold value, acquiring transmission parameters of data transmission;
acquiring a congestion control threshold according to the transmission parameters;
and when the input time delay in the transmission parameters is greater than the congestion control threshold, storing the transmitted data into the cache according to a first preset input speed, wherein the first preset input speed is a data transmission speed required by the used storage space of the cache to reach a preset space size within a preset time period.
2. The data transmission control method according to claim 1, wherein the step of obtaining the congestion control threshold according to the transmission parameter comprises:
extracting input data time and the number of all received input requests from the transmission parameters;
calculating the average time delay of the input data according to the input data time and the number of all input requests;
and calculating the congestion control threshold according to the average time delay.
3. The data transmission control method of claim 2, wherein the step of calculating the congestion control threshold according to the average delay comprises:
acquiring the calculation times of a congestion control threshold;
calculating a delay control factor according to the calculation times;
and calculating a congestion control threshold according to the delay control factor and the average time delay.
4. The data transmission control method according to claim 1, wherein the step of obtaining the used memory space of the buffer memory comprises:
receiving an input request carrying data sent by a client;
and acquiring the used storage space of the cache according to the input request.
5. The data transmission control method of claim 4, wherein the step of storing the transmitted data in the buffer at the first preset input speed comprises:
acquiring data corresponding to the input request;
calculating the data transmission speed required by the used cache space reaching the size of the preset space within a preset time period to obtain a first preset input speed;
and storing the data to the cache according to the first preset input speed.
6. The data transmission control method according to any one of claims 1 to 5, wherein after the step of obtaining the congestion control threshold according to the transmission parameter, the method further comprises:
and when the input time delay in the transmission parameters is less than or equal to the congestion control threshold, storing the data into the cache according to a second preset input speed, wherein the second preset input speed is greater than the first preset input speed.
7. The data transmission control method according to any one of claims 1 to 5, wherein the buffer includes a first storage space and a second storage space, and after the step of obtaining the used storage space of the buffer, the method further comprises:
and when the used storage space of the cache is full of the first storage space of the cache, storing the data into a second storage space according to a third preset input speed, wherein the third preset input speed is less than the first preset input speed.
8. A data transmission control apparatus, comprising:
a storage space acquiring unit, configured to acquire a used storage space of a cache of a storage gateway;
the parameter acquisition unit is used for acquiring transmission parameters of data transmission when the used storage space of the cache is larger than a preset threshold value;
a threshold obtaining unit, configured to obtain a congestion control threshold according to the transmission parameter;
and the first storage unit is used for storing the transmitted data into the cache according to a first preset input speed when the input time delay in the transmission parameters is greater than the congestion control threshold, wherein the first preset input speed is a data transmission speed required by the used storage space of the cache reaching a preset space size within a preset time period.
9. The data transmission control device according to claim 8, wherein the threshold acquisition unit includes:
the extraction subunit is used for extracting the input data time and the number of all the received input requests from the transmission parameters;
the time delay calculating subunit is used for calculating the average time delay of the input data according to the input data time and the number of all input requests;
and the threshold calculation subunit is used for calculating the congestion control threshold according to the average time delay.
10. The data transmission control device according to claim 9, wherein the threshold calculation subunit is specifically configured to:
acquiring the calculation times of a congestion control threshold;
calculating a delay control factor according to the calculation times;
and calculating a congestion control threshold according to the delay control factor and the average time delay.
11. The data transmission control device according to claim 8, wherein the storage space obtaining unit is specifically configured to:
receiving an input request carrying data sent by a client;
and acquiring the used storage space of the cache according to the input request.
12. The data transmission control device according to claim 11, wherein the first storage unit is specifically configured to:
acquiring data corresponding to the input request;
calculating the data transmission speed required by the used cache space reaching the size of the preset space within a preset time period to obtain a first preset input speed;
and storing the data to the cache according to the first preset input speed.
13. The data transmission control device according to any one of claims 8 to 12, characterized by further comprising:
and the second storage unit is used for storing the data into the cache according to a second preset input speed when the input time delay in the transmission parameters is less than or equal to the congestion control threshold, wherein the second preset input speed is greater than the first preset input speed.
14. The data transmission control device according to any one of claims 8 to 12, wherein the buffer includes a first storage space and a second storage space, the data transmission control device further comprising:
and the third storage unit is used for storing the data into a second storage space according to a third preset input speed when the used storage space of the cache is full of the first storage space of the cache, wherein the third preset input speed is less than the first preset input speed.
15. A storage medium storing a plurality of instructions adapted to be loaded by a processor to perform the steps of the data transmission control method according to any one of claims 1 to 7.
CN201810017412.4A 2018-01-09 2018-01-09 Data transmission control method, device and storage medium Active CN110022268B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201810017412.4A CN110022268B (en) 2018-01-09 2018-01-09 Data transmission control method, device and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201810017412.4A CN110022268B (en) 2018-01-09 2018-01-09 Data transmission control method, device and storage medium

Publications (2)

Publication Number Publication Date
CN110022268A CN110022268A (en) 2019-07-16
CN110022268B true CN110022268B (en) 2022-05-03

Family

ID=67187654

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201810017412.4A Active CN110022268B (en) 2018-01-09 2018-01-09 Data transmission control method, device and storage medium

Country Status (1)

Country Link
CN (1) CN110022268B (en)

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110391956B (en) * 2019-07-23 2021-08-13 中国工商银行股份有限公司 Method and device for identifying and monitoring state of network service process
CN112737971B (en) * 2019-10-28 2023-06-23 腾讯科技(深圳)有限公司 Data processing method, device, storage medium and network equipment
CN110784414B (en) * 2019-10-30 2023-06-23 北京达佳互联信息技术有限公司 Data interaction method, device, electronic equipment and storage medium
CN112751778A (en) * 2019-10-30 2021-05-04 阿里巴巴集团控股有限公司 Data transmission control method and device, congestion detection and device and server system
CN111324309A (en) * 2020-02-21 2020-06-23 北京天融信网络安全技术有限公司 Data storage method and electronic equipment
CN113840272A (en) * 2021-10-12 2021-12-24 北京奕斯伟计算技术有限公司 Data transmission method, data transmission device and electronic device
CN114826374B (en) * 2022-03-28 2023-12-29 西安空间无线电技术研究所 Ka frequency band satellite high-speed data transmission system and method

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103782555A (en) * 2012-09-06 2014-05-07 华为技术有限公司 Network transmission time delay control method, service quality control entity and communication device
CN104301066A (en) * 2013-07-19 2015-01-21 华为技术有限公司 Data packet transmission method and device
CN105337891A (en) * 2015-11-02 2016-02-17 北京百度网讯科技有限公司 Traffic control method and traffic control device for distributed cache system
CN105912639A (en) * 2016-04-08 2016-08-31 浪潮(北京)电子信息产业有限公司 Automatic test method and apparatus for data write-in file system
WO2016206043A1 (en) * 2015-06-25 2016-12-29 Thomson Licensing Method and device for data transmission
CN106489136A (en) * 2014-06-02 2017-03-08 美光科技公司 For adjusting the system and method for bag transmission in expansible accumulator system agreement
CN106549876A (en) * 2015-09-22 2017-03-29 中兴通讯股份有限公司 Based on the method for controlling network congestion of ICAP agreements, device and client

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103782555A (en) * 2012-09-06 2014-05-07 华为技术有限公司 Network transmission time delay control method, service quality control entity and communication device
CN104301066A (en) * 2013-07-19 2015-01-21 华为技术有限公司 Data packet transmission method and device
CN106489136A (en) * 2014-06-02 2017-03-08 美光科技公司 For adjusting the system and method for bag transmission in expansible accumulator system agreement
WO2016206043A1 (en) * 2015-06-25 2016-12-29 Thomson Licensing Method and device for data transmission
CN106549876A (en) * 2015-09-22 2017-03-29 中兴通讯股份有限公司 Based on the method for controlling network congestion of ICAP agreements, device and client
CN105337891A (en) * 2015-11-02 2016-02-17 北京百度网讯科技有限公司 Traffic control method and traffic control device for distributed cache system
CN105912639A (en) * 2016-04-08 2016-08-31 浪潮(北京)电子信息产业有限公司 Automatic test method and apparatus for data write-in file system

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
"A ME-based approximation model for the performance evaluation of congestion control mechanism using queue thresholds";L.Guan 等;《The Fourth International Conference onComputer and Information Technology, 2004. CIT "04.》;20041130;全文 *
"机会网络路由策略及拥塞控制研究";王斌;《中国优秀硕士学位论文全文数据库·信息科技辑》;20170215;全文 *

Also Published As

Publication number Publication date
CN110022268A (en) 2019-07-16

Similar Documents

Publication Publication Date Title
CN110022268B (en) Data transmission control method, device and storage medium
CN110048968B (en) Domain name bandwidth adjusting method and device
US10768823B2 (en) Flow control for unaligned writes in network storage device
US10838626B2 (en) Methods, apparatuses, and computer program products for controlling write requests in storage system
WO2017031958A1 (en) Write strategy control method and device
CN105302497B (en) A kind of buffer memory management method and system
US8914501B2 (en) Method, apparatus, and system for scheduling distributed buffer resources
US10958592B2 (en) Domain name bandwidth adjustment method and apparatus
US10862992B2 (en) Resource cache management method and system and apparatus
WO2014000300A1 (en) Data buffer device, data storage system and method
CN106648456B (en) Dynamic copies file access method based on user's amount of access and forecasting mechanism
CN103559072A (en) Method and system for implementing bidirectional auto scaling service of virtual machines
CN105446653B (en) A kind of data merging method and equipment
WO2021103596A1 (en) Data migration method, device, and computer-readable storage medium
CN105656810B (en) Method and device for updating application program
CN101772061A (en) Sleeping timer control method and device and communication system
CN111212114B (en) Method and device for downloading resource file
US9734048B2 (en) Storage management device, performance adjustment method, and computer-readable recording medium
WO2019011262A1 (en) Method and apparatus for resource allocation
CN112783807A (en) Model calculation method and system
US20070033372A1 (en) Windowing external block translations
WO2015018003A1 (en) Ltr/obff design scheme for ethernet adapter application
WO2016206463A1 (en) Write operation control method, device and system
US20070106774A1 (en) Computer system controlling bandwidth according to priority state
CN108459821A (en) A kind of method and device of data buffer storage

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant