CN110022268A - A kind of data transfer control method, device and storage medium - Google Patents

A kind of data transfer control method, device and storage medium Download PDF

Info

Publication number
CN110022268A
CN110022268A CN201810017412.4A CN201810017412A CN110022268A CN 110022268 A CN110022268 A CN 110022268A CN 201810017412 A CN201810017412 A CN 201810017412A CN 110022268 A CN110022268 A CN 110022268A
Authority
CN
China
Prior art keywords
data
caching
memory space
input
input speed
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201810017412.4A
Other languages
Chinese (zh)
Other versions
CN110022268B (en
Inventor
蔡舒
孙彬
李�杰
王建
位爱伶
吴先斌
屈宇鹏
王利虎
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tencent Technology Shenzhen Co Ltd
Original Assignee
Tencent Technology Shenzhen Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tencent Technology Shenzhen Co Ltd filed Critical Tencent Technology Shenzhen Co Ltd
Priority to CN201810017412.4A priority Critical patent/CN110022268B/en
Publication of CN110022268A publication Critical patent/CN110022268A/en
Application granted granted Critical
Publication of CN110022268B publication Critical patent/CN110022268B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L47/00Traffic control in data switching networks
    • H04L47/10Flow control; Congestion control
    • H04L47/12Avoiding congestion; Recovering from congestion
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L49/00Packet switching elements
    • H04L49/90Buffering arrangements
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/50Network services
    • H04L67/56Provisioning of proxy services
    • H04L67/568Storing data temporarily at an intermediate stage, e.g. caching

Abstract

The embodiment of the invention discloses a kind of data transfer control method, device and storage medium, the embodiment of the present invention has used memory space by acquisition caching;When the configured transmission that the caching is when having used memory space to be greater than preset threshold, and acquisition data are transmitted;Congestion control threshold is obtained according to the configured transmission;When the input delay in the configured transmission is greater than the congestion control threshold, the data of transmission are stored to the caching according to the first default input speed.To realize the purpose for storing according to configured transmission adjust automatically data and avoiding network congestion to the input speed of caching;The program can guarantee the reliability of data transmission, and improve the efficiency of data transmission.

Description

A kind of data transfer control method, device and storage medium
Technical field
The present invention relates to fields of communication technology, and in particular to a kind of data transfer control method, device and storage medium.
Background technique
In a communications system, it during data are transmitted, generally requires and data is cached simultaneously by intermediate bridge Forwarding stores gateway for example, needing to store the bridge that gateway is transmitted as data when data are transmitted from client to server-side The data that client is sent are received, and the data received are stored to caching, then data are uploaded to server-side.
In the prior art, when the storage gateway certain time as intermediate axle beam receives the number of client transmission According to, and receive client send speed be greater than be uploaded to server-side uploading speed when, after a period of time, store gateway The caching of storing data will be easy to be occupied full, and occur network congestion phenomenon rapidly, can not be restored again into data at this time.
In the research and practice process to the prior art, it was found by the inventors of the present invention that since storage gateway stores number According to caching be easy to appear network congestion phenomenon, data can not be restored again into, therefore, cause data transmission be affected, reduce The efficiency and reliability of data transmission.
Summary of the invention
The embodiment of the present invention provides a kind of data transfer control method, device and storage medium, it is intended to improve data transfer Efficiency and reliability.
In order to solve the above technical problems, the embodiment of the present invention the following technical schemes are provided:
A kind of data transfer control method, comprising:
Obtain caching has used memory space;
When the configured transmission that the caching is when having used memory space to be greater than preset threshold, and acquisition data are transmitted;
Congestion control threshold is obtained according to the configured transmission;
When the input delay in the configured transmission is greater than the congestion control threshold, according to the first default input speed The data of transmission are stored to the caching.
A kind of data transfer controller, comprising:
Memory space acquiring unit, for obtain caching used memory space;
Parameter acquiring unit, for when the caching is when having used memory space to be greater than preset threshold, acquisition data to be passed Defeated configured transmission;
Thresholding acquiring unit, for obtaining congestion control threshold according to the configured transmission;
First storage unit, for pressing when the input delay in the configured transmission is greater than the congestion control threshold The data of transmission are stored to the caching according to the first default input speed.
A kind of storage medium, the storage medium are stored with a plurality of instruction, and described instruction is suitable for processor and is loaded, with Execute the step in above-mentioned data transfer control method.
The embodiment of the present invention can by obtain caching use memory space, come judge network whether congestion, and When determining network congestion, for example, when caching used memory space be greater than preset threshold when (illustrate caching memory space i.e. To be occupied full, network compares congestion), the configured transmission of data transmission is obtained, and congestion control door is obtained according to configured transmission Limit, then, when the input delay in configured transmission is greater than congestion control threshold, control input speed is increased slowly, according to First default input speed stores the data of transmission to caching, is stored to realize according to configured transmission adjust automatically data To the input speed of caching, the purpose of network congestion is avoided;The program can guarantee the reliability of data transmission, and improve The efficiency of data transmission.
Detailed description of the invention
To describe the technical solutions in the embodiments of the present invention more clearly, make required in being described below to embodiment Attached drawing is briefly described, it should be apparent that, drawings in the following description are only some embodiments of the invention, for For those skilled in the art, without creative efforts, it can also be obtained according to these attached drawings other attached Figure.
Fig. 1 is the schematic diagram of a scenario of data transfer control system provided in an embodiment of the present invention;
Fig. 2 is the flow diagram of data transfer control method provided in an embodiment of the present invention;
Fig. 3 is the system architecture diagram that storage gateway provided in an embodiment of the present invention carries out data transmission control;
Fig. 4 is another flow diagram of data transfer control method provided in an embodiment of the present invention;
Fig. 5 is the signal that storage gateway provided in an embodiment of the present invention carries out data transmission control by congestion avoidance algorithm Figure;
Fig. 6 is the structural schematic diagram of data transfer controller provided in an embodiment of the present invention;
Fig. 7 is another structural schematic diagram of data transfer controller provided in an embodiment of the present invention;
Fig. 8 is another structural schematic diagram of data transfer controller provided in an embodiment of the present invention;
Fig. 9 is another structural schematic diagram of data transfer controller provided in an embodiment of the present invention;
Figure 10 is the structural schematic diagram of server provided in an embodiment of the present invention.
Specific embodiment
Following will be combined with the drawings in the embodiments of the present invention, and technical solution in the embodiment of the present invention carries out clear, complete Site preparation description, it is clear that described embodiments are only a part of the embodiments of the present invention, instead of all the embodiments.It is based on Embodiment in the present invention, those skilled in the art's every other implementation obtained without creative efforts Example, shall fall within the protection scope of the present invention.
In the following description, specific embodiments of the present invention will refer to the step as performed by one or multi-section computer And symbol illustrates, unless otherwise stating clearly.Therefore, these steps and operation will have to mention for several times is executed by computer, this paper institute The computer execution of finger includes by representing with the computer processing unit of the electronic signal of the data in a structuring pattern Operation.This operation is converted at the data or the position being maintained in the memory system of the computer, reconfigurable Or in addition change the running of the computer in mode known to the tester of this field.The maintained data structure of the data For the provider location of the memory, there is the specific feature as defined in the data format.But the principle of the invention is with above-mentioned text Word illustrates that be not represented as a kind of limitation, this field tester will appreciate that plurality of step and behaviour as described below Also it may be implemented in hardware.
The embodiment of the present invention provides a kind of data transfer control method, device and storage medium.
Referring to Fig. 1, Fig. 1 is the schematic diagram of a scenario of data transfer control system provided by the embodiment of the present invention, the number It may include data transfer controller according to transmission control system, which specifically can integrate in server In, for example, the data transfer controller can integrate in storage gateway in the server, it is mainly used for having obtained caching Using memory space, wherein caching can be in real time or be spaced preset time acquisition using the acquisition opportunity of memory space, Either, the acquisition when receiving the input request of client transmission, for example, storage gateway can receive customer end A transmission The input request 2 that input request 1, reception customer end B are sent and the input request 3 for receiving client C transmission etc., Ke Yili Solution be client and input request particular number can also carry out flexible setting according to the actual situation.When the use of caching When memory space is greater than preset threshold, illustrate that the memory space of caching will be occupied full, network compares congestion, obtains data at this time The configured transmission of transmission, and congestion control threshold is obtained according to configured transmission.It is gathered around when the input delay in configured transmission is greater than Plug control thresholding when, according to the first default input speed by the data of transmission store to caching, etc..
It should be noted that the schematic diagram of a scenario of data transfer control system shown in FIG. 1 is only an example, this hair The data transfer control system and scene of bright embodiment description are the technologies in order to more clearly illustrate the embodiment of the present invention Scheme does not constitute the restriction for technical solution provided in an embodiment of the present invention, those of ordinary skill in the art it is found that with The differentiation of communication system and the appearance of new business scene, technical solution provided in an embodiment of the present invention ask similar technology Topic, it is equally applicable.
It is described in detail separately below.
In the present embodiment, it will be described from the angle of data transfer controller, data transfer controller tool Body can integrate in the network equipments such as server or gateway.
A kind of data transfer control method, comprising: obtain caching has used memory space;Storage has been used when caching When space is greater than preset threshold, the configured transmission of data transmission is obtained;Congestion control threshold is obtained according to configured transmission;Work as transmission When input delay in parameter is greater than congestion control threshold, the data of transmission are stored to slow according to the first default input speed It deposits.
Referring to Fig. 2, Fig. 2 is the flow diagram for the data transfer control method that first embodiment of the invention provides.It should Data transfer control method includes:
In step s101, obtain caching has used memory space.
In the present embodiment, data transfer controller mainly according to present data transmission the case where, dynamic carry out congestion control System.For example, acquisition caching has used the information such as memory space and configured transmission, dynamic carries out congestion control.Wherein, congestion Control can be used for adjusting data transmission in network amount, so that volume of transmitted data is approached the bearing capacity of current network, avoids internetworking Can decline, etc..
Optionally, in one embodiment, what data transfer controller acquisition cached can the step of having used memory space To include: the input request for receiving the carrying data that client is sent;Use storage empty according to input request caching Between.
Specifically, data transfer controller, which can be, receives the input request that client is sent, wherein input request In can carry data and solicited message of transmission in need etc., the data can be need be written caching data, the input Request may include write request or snapshot request etc..When data transfer controller receives input request, caching is obtained Used memory space, i.e. data transfer controller detecting caching use memory space, wherein the caching is data The preset caching of transmission control unit itself, for example, write buffer or snapshot caching etc., it only includes a storage which, which can be, Space also may include multiple memory spaces, and the size of multiple memory space can be consistent, can also be inconsistent.
Optionally, in another embodiment, data transfer controller obtains the step of having used memory space of caching May include: data transfer controller in real time or interval preset time obtain caching used memory space.Wherein, this is pre- If the time can carry out flexible setting according to the actual situation, specific value is not construed as limiting here.It is understood that caching The acquisition modes for having used memory space can also be acquisition otherwise.
In step s 102, when the transmission that caching is when having used memory space to be greater than preset threshold, and acquisition data are transmitted Parameter.
Get caching used memory space after, data transfer controller judgement caching use storage sky Between whether be greater than preset threshold, wherein preset threshold can carry out flexible setting according to actual needs, and specific value is here not It limits.For example, preset threshold can be set to the sky of the memory space 90% of caching when caching includes a memory space Between size;When caching includes the first memory space and the second memory space, preset threshold can be set to first depositing for caching Store up the space size, etc. in space.
When caching used memory space be less than or equal to preset threshold when, illustrate the non-congestion of current network, at this time The configured transmission that data transfer controller can remain current transmits data, for example, current input can be maintained Speed stores data to caching.When caching used memory space be greater than preset threshold when, illustrate that current network is relatively gathered around The step of plug, then data transfer controller can execute the configured transmission for obtaining data transmission.
Wherein, the configured transmission for obtaining data transmission can be, and obtain the number for needing that caching (being stored into caching) is written According to configured transmission, i.e., the data can be need be written caching data.The configured transmission may include write data into it is slow Input delay when depositing, input data time when writing data into caching, all input number of requests, the speed received increase Average delay etc. when the long factor and high speed writein, it is to be understood that the configured transmission can also include other parameters, Particular content is not construed as limiting here.
In step s 103, congestion control threshold is obtained according to configured transmission.
After obtaining configured transmission, data transfer controller can obtain congestion control threshold according to configured transmission, In certain embodiments, data transfer controller may include: according to the step of configured transmission acquisition congestion control threshold
The input data time is extracted from configured transmission, and all input number of requests received;According to input Data time and all input number of requests, calculate the average delay of input data;Congestion control door is calculated according to average delay Limit.
When caching is when having used memory space to be greater than preset threshold, data transfer controller triggers congestion control and calculates Method, wherein congestion avoidance algorithm is mainly used for, and obtains congestion control threshold according to configured transmission, and according to congestion control threshold Input speed is adjusted, stores data to caching according to input speed adjusted.Specifically, the configured transmission due to getting In include input data time and all input number of requests received etc. when writing data into caching, therefore, data pass Defeated control device can extract the input data time from configured transmission, and the institute received is extracted from configured transmission There is input number of requests, then, according to input data time and all input number of requests, calculates the mean time of input data Prolong, calculation formula is as follows:
Avg_delay=duration/write_total;
Wherein, avg_delay indicates that average delay, duration indicate the input data time, and write_total expression connects All input number of requests received.
Secondly, calculating congestion control threshold according to average delay, optionally, data transfer controller is according to average delay Calculate congestion control threshold the step of may include:
Obtain the calculation times of congestion control threshold;According to calculation times computing relay controlling elements;It is controlled according to delay The factor and average delay calculate congestion control threshold.Wherein, the calculation times of congestion control threshold are congestion avoidance algorithm touching The number of hair.Its calculation formula is as follows:
Write_thres=avg_delay* (1+ α)
α=exp (n, retreat_cnt)
Wherein, write_thres indicates that congestion control threshold, avg_delay indicate that average delay, α indicate delay control The factor, (i.e. statistics obtains the calculation times of retreat_cnt expression congestion control threshold from data transfer controller starting Congestion avoidance algorithm triggering number), exp indicate using natural constant e as the exponential function at bottom, n can with value be 1/4, or Person is other numerical value.
At this point, obtained congestion control threshold write_thres can be the approximation of the average speed of data transmission, with The number retreat_cnt of congestion avoidance algorithm triggering tend to be infinite, delay controlling elements α levels off to zero, it is meant that congestion The number retreat_cnt of control algolithm triggering is bigger, and the statistics of the configured transmission of historical data transmission is more accurate, so that congestion Control thresholding write_thres about levels off to the average value of statistics.
In step S104, when the input delay in configured transmission is greater than congestion control threshold, preset according to first defeated Enter speed to store the data of transmission to caching.
When caching is when having used memory space to be greater than preset threshold, congestion avoidance algorithm, at this time active client are triggered Input speed may be reduced to a very low velocity amplitude, for example, only receiving 3 or 4 inputs request etc. each second, i.e., Input delay write_delay can be a bigger value.
Since, including input delay, data transfer controller can be extracted from configured transmission in configured transmission Input delay, after obtaining congestion control threshold, data transfer controller may determine that whether input delay is greater than congestion control Thresholding processed illustrates that current network compares when input delay write_thres is greater than congestion control threshold write_thres Congestion starts slow growth algorithm, i.e., stores the data of transmission to caching according to the first default input speed, solve network and gather around In the case where plug or low speed, congestion is preferably avoided.
Wherein, input delay write_delay is the additional time delay of each input request, input delay write_ Delay is bigger, and the quantity for the input request that each second, data transfer controller received is fewer.Data transfer controller can Input speed is adjusted with adjusting the numerical value of input delay, to realize Congestion Avoidance.In addition, speed growth factor Write_inc is bigger, and input delay write_delay reduction is faster, and input speed is promoted faster.
It should be noted that it is a smaller value that speed growth factor write_inc, which can be set, so that when input Prolong write_delay slowly to reduce, data, which are stored to the input speed of caching, slowly to be increased.Optionally, the first default input Speed the comparison of the growth is gentle, achievees the effect that smooth, and rate curve can tend to straight line, can be by the first default input Speed is arranged so that input speed can just be write corresponding to full caching after preset time (such as 20 minutes or 30 minutes etc.) Speed;Or presupposition multiple (for example, 2 times or 3 times etc.) is set by the first default input speed and adjusts input in previous triggering The prefixed time interval of speed can just write speed corresponding to full caching, etc..Certainly, which can also To carry out flexible setting according to actual needs, specific value is not construed as limiting here.
In some embodiments, data transfer controller stores the data of transmission according to the first default input speed May include: to the step of caching
It obtains input and requests corresponding data;Calculate caching has used memory space within a preset period of time, reaches pre- If data transmission bauds needed for space size, the first default input speed is obtained;According to the first default input speed by data It stores to caching.
Specifically, the input that data transfer controller can be sent according to the client received is requested, and it is defeated to obtain this Enter to request corresponding data, and, caching is reached needed for pre-set space size using memory space within a preset period of time Data transmission bauds, wherein the preset time period and pre-set space size can carry out flexible setting according to actual needs, tool Body value is not construed as limiting here.For example, it may be what is cached after twenty minutes is just write using memory space needed for full caching institute Data transmission bauds.Then, which is set as the first default input speed, according to the first default input speed Degree stores data to caching, so as to make the utilization rate of caching keep stablizing, for example, remaining the utilization rate of caching 30%~80%, the concurrent quantity of network is effectively improved, the memory space of caching is taken full advantage of.
In some embodiments, data transfer controller according to configured transmission obtain congestion control threshold the step of it Afterwards, data transfer control method can also include:
It, will according to the second default input speed when the input delay in configured transmission is less than or equal to congestion control threshold Data are stored to caching, wherein the second default input speed is greater than the first default input speed.
Specifically, during being compared input delay with congestion control threshold, as input delay write_ When delay is less than or equal to congestion control threshold write_thres, illustrate current network not congestion, starting quickly increases at this time Long algorithm, into Fast growth phase.In Fast growth phase, data store the input speed to caching can be according to exponential type Increase, i.e., the second default input speed can be exponential type growth rate, can will count according to the second default input speed at this time According to storing to caching, therefore input delay write_delay can be rapidly achieved or be less than congestion control threshold Write_ thres。
In some embodiments, caching may include the first memory space and the second memory space etc., obtain caching After the step of having used memory space, data transfer control method can also include:
When the first memory space for taking caching using memory space of caching, presetting input speed according to third will Data are stored to the second memory space, wherein third presets input speed less than the first default input speed.
Specifically, caching may include one or more memory spaces, include that the first memory space and second is deposited with caching For storage space, wherein the first memory space is normal memory space, which can provide write-in number at any time According to the second memory space is emergency memory space, which is not written into data in the case where network non-congestion, only uses In the case where network congestion, occur providing write-in data when network disconnection.
Data transfer controller is in detecting caching during having used memory space, it can be determined that caching has made The first memory space of caching whether is taken with memory space, for example, vm_ when the first memory space of caching can be occupied full Buff_full is assigned a value of 1, and vm_buff_full is assigned a value of 0 when the first memory space of caching is not occupied full;Alternatively, can also Vm_buff_full is assigned a value of 0 when being occupied full with the first memory space that will be cached, and the first memory space of caching is not occupied Vm_buff_full is assigned a value of 1, etc. when full, for convenience, in embodiments of the present invention, will deposit with the first of caching Storage space, which is occupied full, is assigned a value of 1, the first memory space of caching is not occupied full for being assigned a value of 0 and is illustrated.
At this point, data transfer controller may determine that the value of vm_buff_full to determine that the first storage of caching is empty Between whether be occupied full, when the value of vm_buff_full be 1 when, illustrate caching take the first of caching using memory space Memory space, the slow service algorithm that opening network disconnects at this time enable the second memory space, while adjusting input speed, that is, press Input speed is preset according to third to store data to the second memory space.Wherein, third is preset input speed and be can be set to 1 second Clock inputs 1 time, either, according to information such as the size of the second memory space and the network ports, adjusts third and presets input speed It can guarantee that the second memory space is capable of providing storage service up to a few hours or a couple of days etc..Certainly, third presets input speed Flexible setting can be carried out according to actual needs, and specific value is not construed as limiting here.
From the foregoing, it will be observed that the embodiment of the present invention can use memory space by acquisition caching, whether to judge network Congestion, and (illustrate depositing for caching when memory space having been used to be greater than preset threshold when determining network congestion, such as when caching Storage space will be occupied full, and network compares congestion), start congestion avoidance algorithm at this time, obtains the configured transmission of data transmission, with And congestion control threshold is obtained according to configured transmission;Then, when the input delay in configured transmission is greater than congestion control threshold, Control input speed is increased slowly, stores the data of transmission to caching according to the first default input speed;To realize The purpose that network congestion is avoided to the input speed of caching is stored according to configured transmission adjust automatically data;The program can be with Guarantee the reliability of data transmission, and improves the efficiency of data transmission.
Citing, is described in further detail by the method according to described in above-described embodiment below.
For storing gateway, referring to Fig. 3, Fig. 3 is the system architecture diagram for storing gateway and carrying out data transmission control, In, storage gateway may include storage gateway module and storage gateway policy module etc., which may include depositing Gateway write buffer and storage gateway read buffer etc. are stored up, can also include storage gateway customer end module, storage in system architecture diagram Server-side command module, storage service end data module and final storage equipment etc..
Wherein, storage gateway customer end module is connectable to storage gateway module, mainly deposits with storage gateway module Storage gateway write buffer and storage gateway read buffer interact.
Storage gateway module, which can be, is deployed in storage equipment local, or is deployed in virtually depositing in the network environment of strange land It stores up in equipment.
Storing gateway write buffer is the local storage medium that can be accessed by storage gateway module, is written to for storing Gateway module is stored, but is not yet uploaded to the data of final storage EM equipment module.
Store gateway read buffer be the local storage medium that can be accessed by storage gateway module, for storage by The data that storage gateway module was read, or the data that storage is pre-read by storage gateway module.
Storage service sort command module is mainly used for, and to gateway module is stored, which can be used for controlling lower signaling The behavior of gateway module is stored, and receives the monitoring data that storage gateway module reports.
Storage service end data module can be deployed in final storage equipment local, for receiving storage gateway module transmission Data, and the data received landing is stored into EM equipment module to final.
Storage gateway policy module can be elementary tactics, local network conditions and network history transmission based on setting Situation etc., intelligent decision obtain the module of storage gateway module real-time Transmission rate.
Final storage EM equipment module may include the storage medium of one or more kinds of isomeries, be mainly used for storing data.
The present embodiment can pass through the input speed of balance storage gateway customer end module and the upload of storage gateway module Speed, so that storing gateway module under storage gateway write buffer and the lower situation of storage service end data intermodule bandwidth The input speed to storage gateway customer end module can be reduced automatically;In storage gateway module and storage service end data module Between bandwidth increase when, storage gateway module be capable of automatic lifting storage gateway customer end module input speed;In storage gateway When module and storage service end data intermodule are without network, storage gateway customer end module offer can be provided and serviced at a slow speed, Until network recovery, in network recovery, the input speed of storage gateway customer end module can be promoted rapidly, improves data The efficiency and reliability of transmission.
The data transmission stream journey to above system framework is illustrated below:
Firstly, storage gateway customer end module can initiate read request or write request to storage gateway module, net is stored It closes module and receives the read request or write request, when what is received is read request, storage gateway module is read according to storage gateway Whether caching hits, and decides whether to initiate read request or pre- read request to storage service end data module, i.e., when storage gateway is read When storing the data of reading in need in caching, storage gateway module can read data from storage gateway read buffer;Work as storage In gateway read buffer when the data of not stored reading in need, storage gateway module can initiate to read to storage service end data module Request or pre- read request, read data from storage service end data module according to read request or pre- read request.
When what is received is write request or snapshot request, the data write-in that storage gateway module will need to be written stores net Write buffer is closed, and the data are added in the asynchronous write request queue of storage gateway module, can be each by storage gateway module The version number that write request distribution is uniquely incremented by.
It stores gateway module and the read request received, asynchronous write request or snapshot request is sent to storage service end data Module, storage service end data module first trigger snapshot and generate calculation when receiving read request, asynchronous write request or snapshot request Method, records the network transmission situation (the including but not limited to configured transmissions such as transmitted data amount, propagation delay time) of the secondary request, and from Final storage EM equipment module reads data or writes data to final storage EM equipment module, completes the reading or write-in of data Afterwards, finally storage EM equipment module issues corresponding read request, asynchronous write to storage gateway module by storage service end data module Request or the response of snapshot request.
It stores gateway module upon receiving the response, records the network transmission situation of the secondary request and response.When receiving Be the response to read request when, storage gateway module will to storage gateway customer end module return read data, and by this Data are written in storage gateway read buffer according to cache policy.When what is received is the response to asynchronous write request, net is stored The write request in asynchronous write request queue will be handled according to cache policy by closing module.
The signalling process to above system framework is illustrated below:
When storage gateway policy module sends control signaling to storage service sort command module, storage service sort command mould Block issues control signaling to storage gateway module, after storage gateway module receives signaling, executes corresponding response according to the signaling Operation.
Below by above system framework caching situation and historic transmission situation data report flow into explanation:
Storing gateway module can be by local cache situation, history read request network transmission situation and history write request net The data such as network transmission situation are sent to storage service sort command module, and storage service sort command module protects the data received It deposits, which can judge that i.e. storage gateway policy module can be from storage for storage gateway policy module for dynamic strategy The data are obtained in server-side command module, and determine dynamic strategy according to the data.
It should be noted that the system architecture that storage gateway shown in Fig. 3 carries out data transmission control is only one and shows Example, system architecture and scene of description of the embodiment of the present invention etc. are the technologies in order to more clearly illustrate the embodiment of the present invention Scheme does not constitute the restriction for technical solution provided in an embodiment of the present invention, those of ordinary skill in the art it is found that with The differentiation of communication system and the appearance of new business scene, technical solution provided in an embodiment of the present invention ask similar technology Topic, it is equally applicable.
Referring to Fig. 4, Fig. 4 is the flow diagram of data transfer control method provided in an embodiment of the present invention.This method Process may include:
What S201, storage gateway detecting cached has used memory space;If caching has used memory space to be greater than first Preset threshold thens follow the steps S202;If caching has used memory space to be greater than the second preset threshold;It thens follow the steps S207。
Storage gateway can use memory space by storage gateway module detecting caching, wherein storage gateway mould Block can be in the input request for receiving client transmission, and that detects caching has used memory space;Either, net is stored It closes module and is also possible in real time or is spaced preset time, that detects caching has used memory space.
Data and solicited message of transmission in need etc. can be carried in input request, which, which can be, needs to be written The data of caching, input request may include write request or snapshot request etc., and it only includes one or more which, which can be, A memory space, the caching can be storage gateway write buffer.
Storage gateway is by storing after gateway module used memory space what is cached, to caching using depositing Storage space is judged, for example, caching can have been used memory space and the first preset threshold or the second preset threshold etc. Be compared, wherein the first preset threshold can less than the second preset threshold, first preset threshold and the second preset threshold Value can carry out flexible setting according to actual needs, and specific value is not construed as limiting here.
S202, storage gateway obtain the configured transmission of data.
When caching used memory space be less than or equal to the first preset threshold when, illustrate the non-congestion of current network, Storage gateway does not need adjustment input speed at this time, and current data can be maintained to transmit.
When caching is when having used memory space to be greater than the first preset threshold, illustrate currently stored gateway customer end module hair The input request sent is more, causes current network compared with congestion, at this point, storage gateway can be gathered around by storage gateway module starting Control algolithm is filled in, is obtained by congestion avoidance algorithm and the configured transmission for the data that caching is written is needed to adjust input speed.Example Such as, when caching used memory space be greater than memory space 90% space size when, obtain input delay, the visitor of client When the input data time at family end, all input number of requests of client, speed growth factor and gateway high speed writein The parameters such as average delay.
S203, storage gateway obtain congestion control threshold according to configured transmission.
After obtaining configured transmission, storage gateway can extract input number by storing gateway module from configured transmission According to the time, and all input number of requests received;According to input data time and all input number of requests, calculate defeated Enter the average delay of data;Its calculation formula is: average delay=input data time/all inputs number of requests.Then, root Congestion control threshold is calculated according to average delay, specifically, the number of congestion avoidance algorithm triggering is first obtained, further according to the number meter Delay controlling elements are calculated, its calculation formula is: delay controlling elements=exp (1/4, the number of congestion avoidance algorithm triggering);Most Congestion control threshold is calculated according to delay controlling elements and average delay afterwards, its calculation formula is: congestion control threshold=average Time delay * (1+ postpones controlling elements).
S204, storage gateway judge whether the input delay in configured transmission is greater than congestion control threshold;If so, executing Step S205;If it is not, thening follow the steps S206.
Gateway is stored after obtaining congestion control threshold, input delay is compared with congestion control threshold, is judged defeated Enter whether time delay is greater than congestion control threshold.
S205, storage gateway store data to caching according to the first default input speed.
When input delay is greater than congestion control threshold, illustrate that current network compares congestion, i.e. storage gateway write buffer It is lower with storage service end data intermodule bandwidth, the slow growth algorithm of gateways and starting is stored, i.e., according to the first default input speed The data of transmission are stored to caching, optionally, the first default input speed the comparison of the growth is gentle, and rate curve can become In straight line, what the first default input speed can be arranged via to preset time period caching has used memory space, reaches Corresponding growth rate when to pre-set space size, for example, the first default input speed can be arranged so as to input speed Degree can just write speed corresponding to full caching after 20 minutes.It is understood that the first default input speed can basis Actual needs carries out flexible setting, and specific value is not construed as limiting here.By the way that input speed is adjusted to the first default input Speed can make the utilization rate of caching keep stablizing, take full advantage of the memory space of caching, effectively improve the concurrent of network Quantity.
For example, as shown in figure 5, carrying out data transmission the signal of control in Fig. 5 by congestion avoidance algorithm for storage gateway The input performance data that figure, as storage gateway pass through storage gateway module real-time detection client, wherein horizontal axis is the time, The longitudinal axis is the number that client (i.e. storage gateway customer end module) executes input operation within the unit time.It can from Fig. 5 Out, storage gateway maintain at the beginning quite high input performance (for example, the maintenance of storage gateway is quite high at from 0 to 6000 second Input performance), i.e., storage gateway module and storage service end data intermodule bandwidth it is higher, at this time promoted storage gateway customer The input speed of end module, until when the uploading speed of storage gateway can not meet the input demand of storage gateway customer end module (for example, from 6001 to 10001 seconds when network congestion occur), i.e. storage gateway write buffer and storage service end data intermodule band Width is lower, for example, input delay is greater than congestion control threshold, triggers congestion avoidance algorithm at this time, reduces to storage gateway customer The input speed of end module, by network jitter several times, congestion avoidance algorithm will store the input speed of gateway customer end module Being adjusted to a suitable value makes the defeated of client for example, current input speed is adjusted to the first default input speed The uploading speed for entering speed and storage gateway reaches dynamic equilibrium, even if the input speed and storage of storage gateway customer end module The uploading speed of gateway module reaches dynamic equilibrium (for example, reaching dynamic from input speed after 10001 seconds with uploading speed is deposited Balance) so that client input is able to maintain superior performance.It should be noted that between from 6001 to 10001 seconds, when depositing When storing up gateway module and storage service end data intermodule without network, storage gateway customer end module can be continuously and provided at a slow speed Some period of time in service, and the input speed for storing gateway customer end module is able to ascend in network recovery, improve data biography Defeated reliability.
S206, storage gateway store data to caching according to the second default input speed.
When input delay is less than or equal to congestion control threshold, illustrates the non-congestion of current network, store gateway at this time Start rapid growth algorithm, stores data to caching according to the second default input speed, for example, the second default input speed can To be the speed increased according to exponential type.It is understood that the second default input speed can carry out according to actual needs Flexible setting, specific value are not construed as limiting here.
For example, in Fig. 5, from 0 to 6000 second and after 10001 seconds, does not occur network congestion phenomenon, store gateway Module and storage service end data intermodule bandwidth are higher, at this point it is possible to increase input speed, for example, by current input speed Degree is adjusted to the second default input speed, to improve the utilization rate for the caching for storing gateway.
S207, storage gateway preset input speed according to third and store data to caching.
The caching (i.e. storage gateway write buffer) for storing gateway may include the first memory space and the second memory space etc., Wherein, the first memory space can provide write-in data at any time, and the second memory space is not written into the case where network non-congestion Data in the case where being only used for network congestion, occur providing write-in data when network disconnection.Memory space has been used when caching When greater than the second preset threshold, illustrate the phenomenon that network disconnection occur, which can be set to the first storage sky Between size illustrate to occur network disconnection that is, when the first memory space for taking caching using memory space of caching Phenomenon.At this point, the slow service algorithm that storage gateway opening network disconnects, enables the second memory space, while adjusting input speed, Input speed is preset according to third to store data to the second memory space of caching.Wherein, the default input speed of third can To be set as inputting for 1 second 1 time, either, third, which presets input speed, can guarantee that the second memory space is capable of providing storage Service is up to a few hours or a couple of days etc..It is understood that third presets input speed can carry out flexibly according to actual needs Setting, specific value are not construed as limiting here.
Since the second memory space is arranged, after network congestion occurs in storage gateway, storage gateway still is able to as visitor Family end provides high-speed service and continues for quite a long time;In the case that network is fully disconnected, local cache also can Low-rate service is enough switched to, it is unavailable without will cause storage service, it therefore, can be by fast for of short duration network congestion Speed, which retransmits, to be solved, then not will do it congestion control;For severe network environment, it can also guarantee the stabilization for storing gateway service Property.
In the embodiment of the present invention, based on the congestion avoidance algorithm of storage gateway, when network performance is good, to store gateway High-performance storage service is provided;In network congestion or low speed, the smooth input burr of client.Network condition badly very To suspension, storage gateway still is able to continue externally to provide storage service longer period of time.In the net of storage gateway When network performance boost, the input performance of client can be promoted rapidly to adapt to bandwidth promotion, and to store gateway local The utilization rate of caching keeps stablizing, and improves the upper transfer performance of storage gateway.I.e. storage gateway can occur for a long time in network When suspension, network low speed or network congestion, storage gateway can adjust automatically input speed so that client obtains Smoother and more stable storage service.In addition, the congestion avoidance algorithm of storage gateway is the adjust automatically based on network state, Intervene manually without user, it is very convenient.
For convenient for better implementation data transfer control method provided in an embodiment of the present invention, the embodiment of the present invention is also provided A kind of device based on above-mentioned data transfer control method.The wherein meaning of noun and phase in the method for above-mentioned Data Transmission Controlling Together, specific implementation details can be with reference to the explanation in embodiment of the method.
Referring to Fig. 6, Fig. 6 is the structural schematic diagram of data transfer controller provided in an embodiment of the present invention, wherein counting According to transmission control unit may include memory space acquiring unit 301, parameter acquiring unit 302, thresholding acquiring unit 303 and First storage unit 304 etc..
Wherein, memory space acquiring unit 301, for obtain caching used memory space.
In the present embodiment, mainly according to present data transmission the case where, dynamic carry out congestion control.For example, obtaining caching Used the information such as memory space and configured transmission, dynamic carries out congestion control.Wherein, congestion control can be used for adjusting Data transmission in network amount makes volume of transmitted data approach the bearing capacity of current network, avoids network performance decline, etc..
Optionally, in one embodiment, memory space acquiring unit 301 is specifically used for: receiving the carrying that client is sent The input of data is requested;Memory space has been used according to input request caching.
Specifically, memory space acquiring unit 301, which can be, receives the input request that client is sent, wherein the input Data and the solicited message etc. of transmission in need can be carried in request, which can be the data for needing that caching is written, should Input request may include write request or snapshot request etc..When receiving input request, memory space acquiring unit 301 is obtained The memory space that used of caching is taken, i.e., what the detecting of memory space acquiring unit 301 cached has used memory space, wherein should Caching is the preset caching of data transfer controller itself, for example, write buffer or snapshot caching etc., which, which can be, is only wrapped A memory space is included, also may include multiple memory spaces, the size of multiple memory space can be consistent, can also be different It causes.
Optionally, in another embodiment, what the acquisition of memory space acquiring unit 301 cached has used memory space specific For: it is real-time or interval preset time acquisition caching to have used memory space.Wherein, which can be according to practical feelings Condition carries out flexible setting, and specific value is not construed as limiting here.It is understood that the acquisition for having used memory space of caching Mode can also be acquisition otherwise.
Parameter acquiring unit 302, for when caching is when having used memory space to be greater than preset threshold, acquisition data to be transmitted Configured transmission.
Get caching used memory space after, parameter acquiring unit 302 judgement caching use storage sky Between whether be greater than preset threshold, wherein preset threshold can carry out flexible setting according to actual needs, and specific value is here not It limits.For example, preset threshold can be set to the sky of the memory space 90% of caching when caching includes a memory space Between size;When caching includes the first memory space and the second memory space, preset threshold can be set to first depositing for caching Store up the space size, etc. in space.
When caching used memory space be less than or equal to preset threshold when, illustrate the non-congestion of current network, at this time First storage unit 304 can maintain current configured transmission to transmit data, for example, current input speed can be maintained Degree stores data to caching.When caching is when having used memory space to be greater than preset threshold, illustrate current network compared with congestion, Parameter acquiring unit 302 can execute the step of configured transmission for obtaining data transmission.
Wherein, the configured transmission for obtaining data transmission can be, and obtain the number for needing that caching (being stored into caching) is written According to configured transmission, i.e., the data can be need be written caching data.The configured transmission may include write data into it is slow Input delay when depositing, input data time when writing data into caching, all input number of requests, the speed received Average delay etc. when growth factor and high speed writein, it is to be understood that the configured transmission can also include other ginsengs Number, particular content are not construed as limiting here.
Thresholding acquiring unit 303, for obtaining congestion control threshold according to configured transmission.
After obtaining configured transmission, thresholding acquiring unit 303 can obtain congestion control threshold according to configured transmission, at certain In a little embodiments, as shown in fig. 7, thresholding acquiring unit 303 may include:
Subelement 3031 is extracted, for extracting the input data time from configured transmission, and is received all defeated Enter number of requests;
Time-delay calculation subelement 3032, for calculating input number according to input data time and all input number of requests According to average delay;
Thresholding computation subunit 3033, for calculating congestion control threshold according to average delay.
When caching is when having used memory space to be greater than preset threshold, thresholding acquiring unit 303 triggers congestion avoidance algorithm, Wherein, congestion avoidance algorithm is mainly used for, and obtains congestion control threshold according to configured transmission, and adjust according to congestion control threshold Input speed stores data to caching according to input speed adjusted.Specifically, due to being wrapped in the configured transmission that gets It includes input data time when writing data into caching and all input number of requests received etc., therefore, extracts subelement 3031 can extract the input data time from configured transmission, and all inputs received are extracted from configured transmission Then number of requests by time-delay calculation subelement 3032 according to input data time and all input number of requests, calculates input The average delay of data, calculation formula are as follows:
Avg_delay=duration/write_total;
Wherein, avg_delay indicates that average delay, duration indicate the input data time, and write_total indicates institute There is input number of requests.
Secondly, thresholding computation subunit 3033 calculates congestion control threshold according to average delay, optionally, thresholding calculates son Unit 3033 is specifically used for: obtaining the calculation times of congestion control threshold;According to calculation times computing relay controlling elements;According to Postpone controlling elements and average delay calculates congestion control threshold.
Wherein, the calculation times of congestion control threshold are the number of congestion avoidance algorithm triggering.Its calculation formula is as follows:
Write_thres=avg_delay* (1+ α)
α=exp (n, retreat_cnt)
Wherein, write_thres indicates that congestion control threshold, avg_delay indicate that average delay, α indicate delay control The factor, (i.e. statistics obtains the calculation times of retreat_cnt expression congestion control threshold from data transfer controller starting Congestion avoidance algorithm triggering number), exp indicate using natural constant e as the exponential function at bottom, n can with value be 1/4, or Person is other numerical value.
At this point, obtained congestion control threshold write_thres can be the approximation of the average speed of data transmission, with The number retreat_cnt of congestion avoidance algorithm triggering tend to be infinite, delay controlling elements α levels off to zero, it is meant that congestion The number retreat_cnt of control algolithm triggering is bigger, and the statistics of the configured transmission of historical data transmission is more accurate, so that congestion Control thresholding write_thres about levels off to the average value of statistics.
First storage unit 304, for when the input delay in configured transmission be greater than congestion control threshold when, according to first Default input speed stores the data of transmission to caching.
After congestion avoidance algorithm is triggered, the input speed of active client may be reduced to a very low speed Value, for example, only receiving 3 or 4 input requests etc. each second, i.e. input delay write_delay can be one bigger Value.
Since, including input delay, the first storage unit 304 can be extracted from configured transmission in configured transmission Input delay, after obtaining congestion control threshold, the first storage unit 304 may determine that whether input delay is greater than congestion control Thresholding illustrates that current network is relatively gathered around when input delay write_thres is greater than congestion control threshold write_thres Plug, starts slow growth algorithm, i.e., stores the data of transmission to caching according to the first default input speed, solve network congestion Or in the case where low speed, congestion is preferably avoided.
Wherein, input delay write_delay is the additional time delay of each input request, input delay write_ Delay is bigger, and the quantity for the input request that each second, data transfer controller received is fewer.First storage unit 304 can Input speed is adjusted with adjusting the numerical value of input delay, to realize Congestion Avoidance.In addition, speed growth factor Write_inc is bigger, and input delay write_delay reduction is faster, and input speed is promoted faster.
It should be noted that it is a smaller value that speed growth factor write_inc, which can be set, so that when input Prolong write_delay slowly to reduce, data, which are stored to the input speed of caching, slowly to be increased.Optionally, the first default input Speed the comparison of the growth is gentle, achievees the effect that smooth, and rate curve can tend to straight line, can be by the first default input Speed is arranged so that input speed can just be write corresponding to full caching after preset time (such as 20 minutes or 30 minutes etc.) Speed;Or presupposition multiple (for example, 2 times or 3 times etc.) is set by the first default input speed and adjusts input in previous triggering The prefixed time interval of speed can just write speed corresponding to full caching, etc..Certainly, which can also To carry out flexible setting according to actual needs, specific value is not construed as limiting here.
In some embodiments, the first storage unit 304 is specifically used for:
It obtains input and requests corresponding data;Calculate caching has used memory space within a preset period of time, reaches pre- If data transmission bauds needed for space size, the first default input speed is obtained;According to the first default input speed by data It stores to caching.
Specifically, the input that the first storage unit 304 can be sent according to the client received is requested, and the input is obtained Corresponding data are requested, and, caching is reached needed for pre-set space size using memory space within a preset period of time Data transmission bauds, wherein the preset time period and pre-set space size can carry out flexible setting according to actual needs, specifically Value is not construed as limiting here.For example, it may be number needed for what is cached after twenty minutes just write full caching using memory space According to transmission speed.Then, which is set as the first default input speed by the first storage unit 304, according to One default input speed stores data to caching, so as to make the utilization rate of caching keep stablizing, for example, making the benefit of caching 30%~80% is remained at rate, the concurrent quantity of network is effectively improved, takes full advantage of the memory space of caching.
In some embodiments, as shown in figure 8, data transfer controller further include:
Second storage unit 305, for pressing when the input delay in configured transmission is less than or equal to congestion control threshold Data are stored to caching according to the second default input speed, wherein the second default input speed is greater than the first default input speed.
Specifically, during being compared input delay with congestion control threshold, as input delay write_ When delay is less than or equal to congestion control threshold write_thres, illustrate current network not congestion, starting quickly increases at this time Long algorithm, into Fast growth phase.In Fast growth phase, data store the input speed to caching can be according to exponential type Increase, i.e., the second default input speed can be exponential type growth rate, and the second storage unit 305 can be pre- according to second at this time If input speed stores data to caching, therefore input delay write_delay can be rapidly achieved or be less than congestion control Thresholding Write_thres.
In some embodiments, as shown in figure 9, caching may include the first memory space and the second memory space etc., Data transfer controller further include:
Third storage unit 306, for pressing when the first memory space for taking caching using memory space of caching It presets input speed according to third to store data to the second memory space, wherein it is default less than first that third presets input speed Input speed.
Specifically, caching may include one or more memory spaces, include that the first memory space and second is deposited with caching For storage space, wherein the first memory space is normal memory space, which can provide write-in number at any time According to the second memory space is emergency memory space, which is not written into data in the case where network non-congestion, only uses In the case where network congestion, occur providing write-in data when network disconnection.
Third storage unit 306 is in detecting caching during having used memory space, it can be determined that caching has made The first memory space of caching whether is taken with memory space, for example, vm_ when the first memory space of caching can be occupied full Buff_full is assigned a value of 1, and vm_buff_full is assigned a value of 0 when the first memory space of caching is not occupied full;Alternatively, can also Vm_buff_full is assigned a value of 0 when being occupied full with the first memory space that will be cached, and the first memory space of caching is not occupied Vm_buff_full is assigned a value of 1, etc. when full, for convenience, in embodiments of the present invention, will deposit with the first of caching Storage space, which is occupied full, is assigned a value of 1, the first memory space of caching is not occupied full for being assigned a value of 0 and is illustrated.
At this point, third storage unit 306 may determine that the value of vm_buff_full to determine that the first storage of caching is empty Between whether be occupied full, when the value of vm_buff_full be 1 when, illustrate caching take the first of caching using memory space Memory space, the slow service algorithm that opening network disconnects at this time enable the second memory space, while adjusting input speed, that is, press Input speed is preset according to third to store data to the second memory space.Wherein, third is preset input speed and be can be set to 1 second Clock inputs 1 time, either, according to information such as the size of the second memory space and the network ports, adjusts third and presets input speed It can guarantee that the second memory space is capable of providing storage service up to a few hours or a couple of days etc..Certainly, third presets input speed Flexible setting can be carried out according to actual needs, and specific value is not construed as limiting here.
From the foregoing, it will be observed that the embodiment of the present invention can use storage by the acquisition caching of memory space acquiring unit 301 Space, come judge network whether congestion, and when determining network congestion, for example, when caching to have used memory space to be greater than default (illustrate that the memory space of caching will be occupied full, network compares congestion) when threshold value, starts congestion avoidance algorithm at this time, pass through ginseng Number acquiring unit 302 obtains the configured transmission of data transmission, and obtains congestion according to configured transmission by thresholding acquiring unit 303 Control thresholding;Then, when the input delay in configured transmission is greater than congestion control threshold, the first storage unit 304 controls defeated Enter speed to be increased slowly, stores the data of transmission to caching according to the first default input speed;To realize according to biography Defeated parameter adjust automatically data store the purpose that network congestion is avoided to the input speed of caching;The program can guarantee number According to the reliability of transmission, and improve the efficiency of data transmission.
The embodiment of the present invention also provides a kind of server, and as shown in Figure 10, it illustrates involved in the embodiment of the present invention The structural schematic diagram of server, specifically:
The server may include one or processor 401, one or more meters of more than one processing core The components such as memory 402, power supply 403 and the input unit 404 of calculation machine readable storage medium storing program for executing.Those skilled in the art can manage It solves, server architecture shown in Figure 10 does not constitute the restriction to server, may include than illustrating more or fewer portions Part perhaps combines certain components or different component layouts.Wherein:
Processor 401 is the control centre of the server, utilizes each of various interfaces and the entire server of connection Part by running or execute the software program and/or module that are stored in memory 402, and calls and is stored in memory Data in 402, the various functions and processing data of execute server, to carry out integral monitoring to server.Optionally, locate Managing device 401 may include one or more processing cores;Preferably, processor 401 can integrate application processor and modulatedemodulate is mediated Manage device, wherein the main processing operation system of application processor, user interface and application program etc., modem processor is main Processing wireless communication.It is understood that above-mentioned modem processor can not also be integrated into processor 401.
Memory 402 can be used for storing software program and module, and processor 401 is stored in memory 402 by operation Software program and module, thereby executing various function application and data processing.Memory 402 can mainly include storage journey Sequence area and storage data area, wherein storing program area can the (ratio of application program needed for storage program area, at least one function Such as sound-playing function, image player function) etc.;Storage data area, which can be stored, uses created data according to server Deng.In addition, memory 402 may include high-speed random access memory, it can also include nonvolatile memory, for example, at least One disk memory, flush memory device or other volatile solid-state parts.Correspondingly, memory 402 can also include Memory Controller, to provide access of the processor 401 to memory 402.
Server further includes the power supply 403 powered to all parts, it is preferred that power supply 403 can pass through power management system It unites logically contiguous with processor 401, to realize the function such as management charging, electric discharge and power managed by power-supply management system Energy.Power supply 403 can also include one or more direct current or AC power source, recharging system, power failure monitor electricity The random components such as road, power adapter or inverter, power supply status indicator.
The server may also include input unit 404, which can be used for receiving the number or character letter of input Breath, and generation keyboard related with user setting and function control, mouse, operating stick, optics or trackball signal are defeated Enter.
Although being not shown, server can also be including display unit etc., and details are not described herein.Specifically in the present embodiment, Processor 401 in server can according to following instruction, by the process of one or more application program is corresponding can It executes file to be loaded into memory 402, and runs the application program being stored in memory 402 by processor 401, thus Realize various functions, as follows:
Obtain caching has used memory space;When caching used memory space be greater than preset threshold when, obtain number According to the configured transmission of transmission;Congestion control threshold is obtained according to configured transmission;When the input delay in configured transmission is greater than congestion When controlling thresholding, the data of transmission are stored to caching according to the first default input speed.
Optionally, after the step of obtaining congestion control threshold according to configured transmission, data transfer control method further include: When the input delay in configured transmission is less than or equal to congestion control threshold, data are stored according to the second default input speed To caching, wherein the second default input speed is greater than the first default input speed.
Optionally, caching may include the first memory space and the second memory space, and obtain caching has used storage empty Between the step of after, data transfer control method further include: when caching using memory space taken caching first storage When space, input speed is preset according to third and stores data to the second memory space, wherein third is preset input speed and is less than First default input speed.
In the above-described embodiments, it all emphasizes particularly on different fields to the description of each embodiment, there is no the portion being described in detail in some embodiment Point, it may refer to the detailed description above with respect to data transfer control method, details are not described herein again.
It will appreciated by the skilled person that all or part of the steps in the various methods of above-described embodiment can be with It is completed by instructing, or relevant hardware is controlled by instruction to complete, which can store computer-readable deposits in one In storage media, and is loaded and executed by processor.
For this purpose, the embodiment of the present invention provides a kind of storage medium, wherein being stored with a plurality of instruction, which can be processed Device is loaded, to execute the step in any data transfer control method provided by the embodiment of the present invention.For example, this refers to Order can execute following steps:
Obtain caching has used memory space;When caching used memory space be greater than preset threshold when, obtain number According to the configured transmission of transmission;Congestion control threshold is obtained according to configured transmission;When the input delay in configured transmission is greater than congestion When controlling thresholding, the data of transmission are stored to caching according to the first default input speed.
The specific implementation of above each operation can be found in the embodiment of front, and details are not described herein.
Wherein, which may include: read-only memory (ROM, Read Only Memory), random access memory Body (RAM, Random Access Memory), disk or CD etc..
By the instruction stored in the storage medium, any data provided by the embodiment of the present invention can be executed and passed Step in transmission control method, it is thereby achieved that any data transfer control method institute provided by the embodiment of the present invention The beneficial effect being able to achieve is detailed in the embodiment of front, and details are not described herein.
A kind of data transfer control method, device and storage medium is provided for the embodiments of the invention above to have carried out in detail Thin to introduce, used herein a specific example illustrates the principle and implementation of the invention, and above embodiments are said It is bright to be merely used to help understand method and its core concept of the invention;Meanwhile for those skilled in the art, according to this hair Bright thought, there will be changes in the specific implementation manner and application range, in conclusion the content of the present specification should not manage Solution is limitation of the present invention.

Claims (15)

1. a kind of data transfer control method characterized by comprising
Obtain caching has used memory space;
When the configured transmission that the caching is when having used memory space to be greater than preset threshold, and acquisition data are transmitted;
Congestion control threshold is obtained according to the configured transmission;
When the input delay in the configured transmission is greater than the congestion control threshold, will be passed according to the first default input speed Defeated data are stored to the caching.
2. data transfer control method according to claim 1, which is characterized in that described to be obtained according to the configured transmission The step of congestion control threshold includes:
The input data time is extracted from the configured transmission, and all input number of requests received;
According to the input data time and all input number of requests, the average delay of input data is calculated;
Congestion control threshold is calculated according to the average delay.
3. data transfer control method according to claim 2, which is characterized in that described to be calculated according to the average delay The step of congestion control threshold includes:
Obtain the calculation times of congestion control threshold;
According to the calculation times computing relay controlling elements;
Congestion control threshold is calculated according to the delay controlling elements and the average delay.
4. data transfer control method according to claim 1, which is characterized in that the acquisition caching has used storage The step of space includes:
Receive the input request for the carrying data that client is sent;
Memory space has been used according to the input request caching.
5. data transfer control method according to claim 4, which is characterized in that described according to the first default input speed The data of transmission were stored to the step of caching and include:
It obtains the input and requests corresponding data;
Calculate the caching has used memory space within a preset period of time, the transmission of data needed for reaching pre-set space size Speed obtains the first default input speed;
The data are stored to the caching according to the described first default input speed.
6. data transfer control method according to any one of claim 1 to 5, which is characterized in that described according to After configured transmission obtains the step of congestion control threshold, the method also includes:
When the input delay in the configured transmission is less than or equal to the congestion control threshold, according to the second default input speed Degree stores the data to the caching, wherein the second default input speed is greater than the described first default input speed.
7. data transfer control method according to any one of claim 1 to 5, which is characterized in that the caching includes First memory space and the second memory space, it is described obtain caching the step of having used memory space after, the method is also Include:
When the caching is when taking the first memory space of the caching using memory space, according to the default input speed of third Degree stores the data to the second memory space, wherein the third presets input speed and is less than the described first default input Speed.
8. a kind of data transfer controller characterized by comprising
Memory space acquiring unit, for obtain caching used memory space;
Parameter acquiring unit, for when the caching is when having used memory space to be greater than preset threshold, acquisition data to be transmitted Configured transmission;
Thresholding acquiring unit, for obtaining congestion control threshold according to the configured transmission;
First storage unit, for when the input delay in the configured transmission is greater than the congestion control threshold, according to the One default input speed stores the data of transmission to the caching.
9. data transfer controller according to claim 8, which is characterized in that the thresholding acquiring unit includes:
Subelement is extracted, for extracting the input data time from the configured transmission, and all inputs received are asked Seek quantity;
Time-delay calculation subelement, for calculating input data according to the input data time and all input number of requests Average delay;
Thresholding computation subunit, for calculating congestion control threshold according to the average delay.
10. data transfer controller according to claim 9, which is characterized in that the thresholding computation subunit is specific For:
Obtain the calculation times of congestion control threshold;
According to the calculation times computing relay controlling elements;
Congestion control threshold is calculated according to the delay controlling elements and the average delay.
11. data transfer controller according to claim 8, which is characterized in that the memory space acquiring unit tool Body is used for:
Receive the input request for the carrying data that client is sent;
Memory space has been used according to the input request caching.
12. data transfer controller according to claim 11, which is characterized in that first storage unit is specifically used In:
It obtains the input and requests corresponding data;
Calculate the caching has used memory space within a preset period of time, the transmission of data needed for reaching pre-set space size Speed obtains the first default input speed;
The data are stored to the caching according to the described first default input speed.
13. the data transfer controller according to any one of claim 8 to 12, which is characterized in that the data pass Defeated control device further include:
Second storage unit, for when the input delay in the configured transmission be less than or equal to the congestion control threshold when, The data are stored to the caching according to the second default input speed, wherein the second default input speed is greater than institute State the first default input speed.
14. the data transfer controller according to any one of claim 8 to 12, which is characterized in that the caching packet Include the first memory space and the second memory space, the data transfer controller further include:
Third storage unit, for when the caching is when taking the first memory space of the caching using memory space, It presets input speed according to third to store the data to the second memory space, wherein it is small that the third presets input speed In the described first default input speed.
15. a kind of storage medium, which is characterized in that the storage medium is stored with a plurality of instruction, and described instruction is suitable for processor It is loaded, the step in 1 to 7 described in any item data transfer control methods is required with perform claim.
CN201810017412.4A 2018-01-09 2018-01-09 Data transmission control method, device and storage medium Active CN110022268B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201810017412.4A CN110022268B (en) 2018-01-09 2018-01-09 Data transmission control method, device and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201810017412.4A CN110022268B (en) 2018-01-09 2018-01-09 Data transmission control method, device and storage medium

Publications (2)

Publication Number Publication Date
CN110022268A true CN110022268A (en) 2019-07-16
CN110022268B CN110022268B (en) 2022-05-03

Family

ID=67187654

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201810017412.4A Active CN110022268B (en) 2018-01-09 2018-01-09 Data transmission control method, device and storage medium

Country Status (1)

Country Link
CN (1) CN110022268B (en)

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110391956A (en) * 2019-07-23 2019-10-29 中国工商银行股份有限公司 The identification monitoring method and device of network service processes state
CN110784414A (en) * 2019-10-30 2020-02-11 北京达佳互联信息技术有限公司 Data interaction method and device and electronic equipment
CN111324309A (en) * 2020-02-21 2020-06-23 北京天融信网络安全技术有限公司 Data storage method and electronic equipment
CN112737971A (en) * 2019-10-28 2021-04-30 腾讯科技(深圳)有限公司 Data processing method, device, storage medium and network equipment
CN112751778A (en) * 2019-10-30 2021-05-04 阿里巴巴集团控股有限公司 Data transmission control method and device, congestion detection and device and server system
CN113840272A (en) * 2021-10-12 2021-12-24 北京奕斯伟计算技术有限公司 Data transmission method, data transmission device and electronic device
CN114826374A (en) * 2022-03-28 2022-07-29 西安空间无线电技术研究所 Ka frequency band satellite high-speed data transmission system and method
CN113840272B (en) * 2021-10-12 2024-05-14 北京奕斯伟计算技术股份有限公司 Data transmission method, data transmission device and electronic device

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103782555A (en) * 2012-09-06 2014-05-07 华为技术有限公司 Network transmission time delay control method, service quality control entity and communication device
CN104301066A (en) * 2013-07-19 2015-01-21 华为技术有限公司 Data packet transmission method and device
CN105337891A (en) * 2015-11-02 2016-02-17 北京百度网讯科技有限公司 Traffic control method and traffic control device for distributed cache system
CN105912639A (en) * 2016-04-08 2016-08-31 浪潮(北京)电子信息产业有限公司 Automatic test method and apparatus for data write-in file system
WO2016206043A1 (en) * 2015-06-25 2016-12-29 Thomson Licensing Method and device for data transmission
CN106489136A (en) * 2014-06-02 2017-03-08 美光科技公司 For adjusting the system and method for bag transmission in expansible accumulator system agreement
CN106549876A (en) * 2015-09-22 2017-03-29 中兴通讯股份有限公司 Based on the method for controlling network congestion of ICAP agreements, device and client

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103782555A (en) * 2012-09-06 2014-05-07 华为技术有限公司 Network transmission time delay control method, service quality control entity and communication device
CN104301066A (en) * 2013-07-19 2015-01-21 华为技术有限公司 Data packet transmission method and device
CN106489136A (en) * 2014-06-02 2017-03-08 美光科技公司 For adjusting the system and method for bag transmission in expansible accumulator system agreement
WO2016206043A1 (en) * 2015-06-25 2016-12-29 Thomson Licensing Method and device for data transmission
CN106549876A (en) * 2015-09-22 2017-03-29 中兴通讯股份有限公司 Based on the method for controlling network congestion of ICAP agreements, device and client
CN105337891A (en) * 2015-11-02 2016-02-17 北京百度网讯科技有限公司 Traffic control method and traffic control device for distributed cache system
CN105912639A (en) * 2016-04-08 2016-08-31 浪潮(北京)电子信息产业有限公司 Automatic test method and apparatus for data write-in file system

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
L.GUAN 等: ""A ME-based approximation model for the performance evaluation of congestion control mechanism using queue thresholds"", 《THE FOURTH INTERNATIONAL CONFERENCE ONCOMPUTER AND INFORMATION TECHNOLOGY, 2004. CIT "04.》 *
王斌: ""机会网络路由策略及拥塞控制研究"", 《中国优秀硕士学位论文全文数据库·信息科技辑》 *

Cited By (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110391956A (en) * 2019-07-23 2019-10-29 中国工商银行股份有限公司 The identification monitoring method and device of network service processes state
CN110391956B (en) * 2019-07-23 2021-08-13 中国工商银行股份有限公司 Method and device for identifying and monitoring state of network service process
CN112737971A (en) * 2019-10-28 2021-04-30 腾讯科技(深圳)有限公司 Data processing method, device, storage medium and network equipment
CN112737971B (en) * 2019-10-28 2023-06-23 腾讯科技(深圳)有限公司 Data processing method, device, storage medium and network equipment
CN110784414A (en) * 2019-10-30 2020-02-11 北京达佳互联信息技术有限公司 Data interaction method and device and electronic equipment
CN112751778A (en) * 2019-10-30 2021-05-04 阿里巴巴集团控股有限公司 Data transmission control method and device, congestion detection and device and server system
CN111324309A (en) * 2020-02-21 2020-06-23 北京天融信网络安全技术有限公司 Data storage method and electronic equipment
CN113840272A (en) * 2021-10-12 2021-12-24 北京奕斯伟计算技术有限公司 Data transmission method, data transmission device and electronic device
CN113840272B (en) * 2021-10-12 2024-05-14 北京奕斯伟计算技术股份有限公司 Data transmission method, data transmission device and electronic device
CN114826374A (en) * 2022-03-28 2022-07-29 西安空间无线电技术研究所 Ka frequency band satellite high-speed data transmission system and method
CN114826374B (en) * 2022-03-28 2023-12-29 西安空间无线电技术研究所 Ka frequency band satellite high-speed data transmission system and method

Also Published As

Publication number Publication date
CN110022268B (en) 2022-05-03

Similar Documents

Publication Publication Date Title
CN110022268A (en) A kind of data transfer control method, device and storage medium
CN105302497B (en) A kind of buffer memory management method and system
CN110099009A (en) Dynamic current limiting method and computer-readable medium based on interface response time trend
CN106250305B (en) The self-adaptation control method of monitoring system data collection cycle under cloud computing environment
CN109491249A (en) It is a kind of that there are the design methods of multi-agent system event trigger controller when DoS attack
CN105446653B (en) A kind of data merging method and equipment
CN102137139A (en) Method and device for selecting cache replacement strategy, proxy server and system
CN106648456B (en) Dynamic copies file access method based on user's amount of access and forecasting mechanism
CN103729313B (en) The iostream amount control method of SSD caching and device
CN107783721A (en) The processing method and physical machine of a kind of data
CN105095495B (en) A kind of distributed file system buffer memory management method and system
CN107135088A (en) The method and apparatus that daily record is handled in cloud computing system
CN109814811A (en) A method of reducing NVMe SSD operating lag influences high-speed data storage apparatus writing speed
CN109729175A (en) Edge cooperative data unloading method under unstable channel condition
CN108924203A (en) Data copy self-adapting distribution method, distributed computing system and relevant device
CN107395735B (en) Delay and capacity reduction scheduling method and system for container cluster
WO2015018003A1 (en) Ltr/obff design scheme for ethernet adapter application
CN112714074B (en) Intelligent TCP congestion control method, system, equipment and storage medium
CN107341091A (en) Distributed memory system power consumption management method and device
CN106557434A (en) A kind of interface caching method and system
CN105515844A (en) System for improving software defined network reliability
CN106293500A (en) A kind of write operation control method, Apparatus and system
CN108459821A (en) A kind of method and device of data buffer storage
CN114301845B (en) Self-adaptive data center network transmission protocol selection method
CN114866475B (en) Network-on-chip congestion control method, system, device and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant