CN104092623A - Method and device used for carrying out overload protection on network node or server - Google Patents

Method and device used for carrying out overload protection on network node or server Download PDF

Info

Publication number
CN104092623A
CN104092623A CN201310110430.4A CN201310110430A CN104092623A CN 104092623 A CN104092623 A CN 104092623A CN 201310110430 A CN201310110430 A CN 201310110430A CN 104092623 A CN104092623 A CN 104092623A
Authority
CN
China
Prior art keywords
server
data flow
flow
cluster
network node
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201310110430.4A
Other languages
Chinese (zh)
Inventor
聂雅玲
马元琛
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hitachi Ltd
Original Assignee
Hitachi Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hitachi Ltd filed Critical Hitachi Ltd
Priority to CN201310110430.4A priority Critical patent/CN104092623A/en
Publication of CN104092623A publication Critical patent/CN104092623A/en
Pending legal-status Critical Current

Links

Abstract

The invention provides a method which transmits data from a terminal to a server through a transmission network and carries out overload protection on a network node or a server in a data transmission and processing system. The method comprises the steps that when the network node or the server is overloaded, the flow characteristics of a processed data flow are collected; when the network node or the server is overloaded, clustering is carried out on a data flow with the similar flow characteristics in the processed data flow to generate a cluster to be processed; and according to the flow characteristics of the data flow in the cluster to be processed, a reverse control policy is applied to the data flow in the cluster to be processed, so as to eliminate the overload of the network node or the server.

Description

Network node or server are carried out to the method and apparatus of overload protection
Technical field
The present invention relates to mass data transfers and treatment system; be specifically related to a kind of by data from terminal by transmission network to the method and apparatus that network node or server is carried out the data transmission processing system of server transmission overload protection; can prevention service device and flow surge and the negative effect thereof of network side, and avoided the negative effect of the phase mutual interference of terminal room by reverse cluster control.
Background technology
Along with the development of network and information technology, the application with mass data more and more occurs.As shown in Figure 4, data transmission processing system comprises mass data terminal 201, transmission network 202 and application server 203.Distributed mass data terminal 201 is sent to data on application server 203 and processes by transmission network 202.Due to the surge of data volume, in current data transmission processing system, there is overload in the server 205 in network node 204 and application server 203 again and again.
In the disclosed network traffics surge protection of patent application (US2010/0020687) technology; obtain multiple network traffics and network resource utilization information; determine a best allocated bandwidth to these network flows, in the time that flow has exceeded best allocated bandwidth, carry out packet loss processing according to priority.Only consider resource and the surge control of network side, do not considered resource and the flow surge of end side and server side.And single packet loss processing brings the risk of loss of data.
Therefore, need a kind of utilization to carry out comprehensive monitoring to the resource of terminal, transmission network and server, terminal flow is carried out to clustering control, and comprehensive various strategy, the overload problem that the surge of mass data terminal and generation thereof is caused is carried out the technology of network and server protection.
Summary of the invention
For the above-mentioned defect that overcomes prior art has proposed the present invention.Therefore; the object of the invention is to propose a kind of by data from terminal by transmission network to the method and apparatus that network node or server is carried out the data transmission processing system of server transmission overload protection; can prevention service device and flow surge and the negative effect thereof of network side, and avoided the negative effect of the phase mutual interference of terminal room by reverse cluster control.
To achieve these goals, according to the present invention, proposed a kind of by data from terminal by transmission network to the method for network node or server being carried out the data transmission processing system of server transmission overload protection, comprise: in the time of network node or server generation overload, collect the properties of flow of handled data flow; In the time of network node or server generation overload, the data flow with similar properties of flow in handled data flow is carried out to clustering and generate pending cluster; And according to the properties of flow of the data flow in pending cluster, the data flow for the treatment of in Processing Cluster is applied reverse control strategy, to eliminate the overload of network node or server.
Preferably, described properties of flow comprises: time-delay characteristics, packet loss characteristic or network congestion characteristic, described time-delay characteristics, packet loss characteristic and network congestion characteristic are characterized by delay parameter, packet loss parameter and the path congestion node parameter of data flow respectively.
Preferably, for according to the pending cluster of time-delay characteristics clustering, carry out time delay sending strategy in the terminal that sends at least one data flow in this pending cluster.
Preferably, for according to the pending cluster of packet loss characteristic clustering, carry out packet loss re-transmission policy in the terminal that sends at least one data flow in this pending cluster.
Preferably, for according to the pending cluster of network congestion characteristic clustering, carry out at least one in following oppositely control strategy: the terminal that carry out time delay sending strategy in the terminal that sends at least one data flow in this pending cluster, is sending at least one data flow in this pending cluster is carried out packet loss re-transmission policy and at least one the data flow execution route optimisation strategy in this pending cluster.
Preferably, described pending cluster comprises at least one data flow.
In addition, according to the present invention, proposed a kind of by data from terminal by transmission network to the device that network node or server is carried out the data transmission processing system of server transmission overload protection, comprise: in the time of network node or server generation overload, collect the unit of the properties of flow of handled data flow; In the time that overload occurs for network node or server, the data flow with similar properties of flow in handled data flow is carried out clustering and is generated the unit of pending cluster; And according to the properties of flow of the data flow in pending cluster, the data flow for the treatment of in Processing Cluster is applied reverse control strategy, to eliminate the unit of overload of network node or server.
Invention effect
According to the present invention, can prevention service device and flow surge and the negative effect thereof of network side, avoided the negative effect of the phase mutual interference of terminal room by reverse cluster control.
Brief description of the drawings
By reference to following combination accompanying drawing, to adopted detailed description of the preferred embodiment, it is more apparent that above-mentioned purpose of the present invention, advantage and feature will become, wherein:
Fig. 1 shows the pie graph that network node or server is carried out to the device of overload protection according to of the present invention.
Fig. 2 shows the flow chart that network node or server is carried out to the method for overload protection according to of the present invention.
Fig. 3 shows when when overload occurs server and treats Processing Cluster and apply the schematic diagram of reverse control strategy.
Fig. 4 shows the signal pie graph of the data transmission processing system of prior art.
Embodiment
The preferred embodiments of the present invention are described below with reference to the accompanying drawings.
Fig. 1 shows the pie graph that network node or server is carried out to the device of overload protection according to of the present invention.
As shown in Figure 1, the device 1 that network node or server are carried out to overload protection according to the present invention comprises status information capture unit 101, flow label unit 102, cluster generation unit 103, reverse flow control unit 104 and control strategy storehouse 105.
Fig. 2 shows the flow chart that network node or server is carried out to the method for overload protection according to of the present invention.
In this supposition: in the time that mass data terminal is transmitted data by transmission network to application server, network node N1 or Ni, server S 1 or Sk exist the risk of overload.
Now, server and network state information (step 401) are collected in status information capture unit 101, and collected server and network state information are as shown in table 1 below:
Table 1
State S1 Sk N1 Ni
Server parameter 1:CPU 80% 90% ? ?
Server parameter 2:RAM 60% 100% ? ?
Server parameter 3:I/O 40% 100% ? ?
Network parameter: congested node buffer memory ? ? 90% 96%
In table 1, data are carried out to real-time update.When reaching pre-overload threshold value, the relevant parameter of server S k (or network node) enters pre-overload judgement (step 402).As above in table 1, system can be at least according to a certain server parameter, cpu resource load condition 90% > 80% of for example Sk, or RAM, or I/O resource is in full load condition 100%, judges that Sk enters pre-overload.Comparatively speaking, the CPU of S1, RAM and I/O resource use value are lower, therefore can not be judged as pre-overload.System also can be used the combination of above parameter to carry out the judgement of pre-overload.
Then, the parameter (step 403) of object flow is collected in flow label unit 102.This collection operation is also analyzed for the handled data flow of server S k, marks relevant properties of flow, and for example time-delay characteristics, packet loss characteristic or network congestion characteristic, characterized by delay parameter, packet loss parameter, path congestion node parameter respectively.For example can utilize the time field of host-host protocol to know time delay, the timestamp of such as RSVP etc.Utilize the data sequence number UID of transmission or message field (MFLD) to know packet loss.Can know network path congested node according to IETF/CONEX RFC.Shown in relevant properties of flow table 2 specific as follows:
Table 2
Here P4 other parameters 4 that use that express possibility, Pi other parameter I that use that express possibility, representation parameter P4 and the different values such as x, y, z.
In step 404, whether determining server Sk transships.Value in reference shown in table 1, system can be at least according to a certain server parameter, and the cpu resource load condition of for example Sk increases to 95% >=90%, or RAM, or I/O resource is in full load condition 100%, judges that Sk enters overload.System also can be used the combination of above parameter to carry out the judgement of overload.
When the relevant parameter of server S k reaches overload protection thresholding;, while there is overload (step 404 be), cluster generation unit 103 carries out clustering to the data flow with similar properties of flow in handled data flow and generates pending cluster (step 405).
Here can carry out clustering mainly for single parameter to the application of some single parameter sensitivity.For example: some application is in real time very sensitive to delay parameter.Therefore, can detect for the delay parameter of numerous streams specially, select time delay grade not homogeneous turbulence generate different clusters and apply reverse control strategy after a while.
Certainly, also can carry out comprehensive selection to multiple parameters and carry out clustering
Data in reference table 2, F1 and F3 can become cluster 1, and this cluster has obvious time-delay characteristics, forms the pending cluster according to time-delay characteristics clustering.F1 and F2 can become cluster 2, and this cluster has obvious packet loss characteristic, form according to the pending cluster of packet loss characteristic clustering.F3 and Fn can become cluster 3, and this cluster has network congestion characteristic, according to the pending cluster of network congestion characteristic clustering.In this example, each cluster has been lifted two streams and has been generated as cluster, but the present invention is not limited thereto, and can be that single or multiple stream generates different clusters.
Reverse flow control unit 104 obtains corresponding oppositely control strategy from control strategy storehouse 105, and the data flow in pending cluster is applied reverse control strategy, to eliminate the overload (step 406 and 407) of network node or server.Relevant reverse control strategy can be as follows but be not limited to this:
Time delay sending strategy: send time shift message to making a start of object flow, the transmission delay differing in size in the insertion of making a start.
Packet loss re-transmission policy: part stream is carried out to packet loss processing, then retransmit in the time of the non-overload of server.
Routing optimality strategy: the stream that route is occurred to congested node carries out routing optimality.For example, in table 2, there is network congestion node N1 or Ni in F3 and F4, avoids N1 or Ni node thereby the transmission route of F3 and F4 can be updated to new route.For example use MPLS or Software Defined Network (SDN) technology etc.
Here for according to the pending cluster of time-delay characteristics clustering, can carry out time delay sending strategy in the terminal that sends at least one data flow in this pending cluster.For according to the pending cluster of packet loss characteristic clustering, can carry out packet loss re-transmission policy in the terminal that sends at least one data flow in this pending cluster.For according to the pending cluster of network congestion characteristic clustering, can carry out at least one in time delay sending strategy, packet loss re-transmission policy and path optimization's strategy.
Although the above situation that server k is occurred to transship is described, the present invention is not limited thereto, and the situation that can also occur to transship to network node similarly be processed.
Fig. 3 shows when when overload occurs server and treats Processing Cluster and apply the schematic diagram of reverse control strategy.
As shown in Figure 3, in the time of server overload, there are five cluster A, B, C, D and E with different characteristic.Because the stream in cluster B and D has obvious network congestion node diagnostic, therefore the reverse control that cluster B and D are correlated with.Relevant reverse control strategy can be one or several the combination in above strategy (time delay sending strategy, packet loss re-transmission policy and path optimization's strategy).
By the present invention; in the time of mass data processing, server and network node occur when congested; the stream by handle with similar features carries out cluster and the reverse control for cluster employing corresponding strategies, can protect server or network over loading, the normal operation of protection server or network.
Although below show the present invention in conjunction with the preferred embodiments of the present invention, one skilled in the art will appreciate that without departing from the spirit and scope of the present invention, can carry out various amendments, replacement and change to the present invention.Therefore, the present invention should not limited by above-described embodiment, and should be limited by claims and equivalent thereof.

Claims (7)

  1. By data from terminal by transmission network to a method of network node or server being carried out the data transmission processing system of server transmission overload protection, comprising:
    In the time of network node or the pre-overload of server generation, collect the properties of flow of handled data flow;
    In the time of network node or server generation overload, the data flow with similar properties of flow in handled data flow is carried out to clustering and generate pending cluster; And
    According to the properties of flow of the data flow in pending cluster, the data flow for the treatment of in Processing Cluster is applied reverse control strategy, to eliminate the overload of network node or server.
  2. 2. method according to claim 1, wherein,
    Described properties of flow comprises: time-delay characteristics, packet loss characteristic or network congestion characteristic, described time-delay characteristics, packet loss characteristic and network congestion characteristic are characterized by delay parameter, packet loss parameter and the path congestion node parameter of data flow respectively.
  3. 3. method according to claim 2, wherein,
    For according to the pending cluster of time-delay characteristics clustering, carry out time delay sending strategy in the terminal that sends at least one data flow in this pending cluster.
  4. 4. method according to claim 2, wherein,
    For according to the pending cluster of packet loss characteristic clustering, carry out packet loss re-transmission policy in the terminal that sends at least one data flow in this pending cluster.
  5. 5. method according to claim 2, wherein,
    For according to the pending cluster of network congestion characteristic clustering, carry out at least one in following oppositely control strategy: the terminal that carry out time delay sending strategy in the terminal that sends at least one data flow in this pending cluster, is sending at least one data flow in this pending cluster is carried out packet loss re-transmission policy and at least one the data flow execution route optimisation strategy in this pending cluster.
  6. 6. method according to claim 1, wherein,
    Described pending cluster comprises at least one data flow.
  7. By data from terminal by transmission network to a device that network node or server is carried out the data transmission processing system of server transmission overload protection, comprising:
    In the time of network node or the pre-overload of server generation, collect the unit of the properties of flow of handled data flow;
    In the time that overload occurs for network node or server, the data flow with similar properties of flow in handled data flow is carried out clustering and is generated the unit of pending cluster; And
    According to the properties of flow of the data flow in pending cluster, the data flow for the treatment of in Processing Cluster is applied reverse control strategy, to eliminate the unit of overload of network node or server.
CN201310110430.4A 2013-04-01 2013-04-01 Method and device used for carrying out overload protection on network node or server Pending CN104092623A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201310110430.4A CN104092623A (en) 2013-04-01 2013-04-01 Method and device used for carrying out overload protection on network node or server

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201310110430.4A CN104092623A (en) 2013-04-01 2013-04-01 Method and device used for carrying out overload protection on network node or server

Publications (1)

Publication Number Publication Date
CN104092623A true CN104092623A (en) 2014-10-08

Family

ID=51640305

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201310110430.4A Pending CN104092623A (en) 2013-04-01 2013-04-01 Method and device used for carrying out overload protection on network node or server

Country Status (1)

Country Link
CN (1) CN104092623A (en)

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1689278A (en) * 2001-12-18 2005-10-26 思科技术公司 Methods and apparatus for network congestion control
CN1984043A (en) * 2006-06-07 2007-06-20 华为技术有限公司 Method and system for controlling flow based on bandwidth adjusting mechanism
CN101060475A (en) * 2006-05-23 2007-10-24 华为技术有限公司 A method and system for regulating the link loads of media gateway control interface
US7403988B1 (en) * 2001-12-28 2008-07-22 Nortel Networks Limited Technique for autonomous network provisioning
CN101321173A (en) * 2008-07-21 2008-12-10 华为技术有限公司 Method, system and device for preventing network attack
US20100142524A1 (en) * 2007-07-02 2010-06-10 Angelo Garofalo Application data flow management in an ip network
CN102271092A (en) * 2011-09-09 2011-12-07 西安电子科技大学 Congestion control system and method based on node load quantization level

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1689278A (en) * 2001-12-18 2005-10-26 思科技术公司 Methods and apparatus for network congestion control
US7403988B1 (en) * 2001-12-28 2008-07-22 Nortel Networks Limited Technique for autonomous network provisioning
CN101060475A (en) * 2006-05-23 2007-10-24 华为技术有限公司 A method and system for regulating the link loads of media gateway control interface
CN1984043A (en) * 2006-06-07 2007-06-20 华为技术有限公司 Method and system for controlling flow based on bandwidth adjusting mechanism
US20100142524A1 (en) * 2007-07-02 2010-06-10 Angelo Garofalo Application data flow management in an ip network
CN101321173A (en) * 2008-07-21 2008-12-10 华为技术有限公司 Method, system and device for preventing network attack
CN102271092A (en) * 2011-09-09 2011-12-07 西安电子科技大学 Congestion control system and method based on node load quantization level

Similar Documents

Publication Publication Date Title
JP2017034678A5 (en)
JP4774357B2 (en) Statistical information collection system and statistical information collection device
JP5659125B2 (en) Relay device and relay method
CN106062726B (en) Flow aware buffer management for data center switches
EP2688256A1 (en) Network system, and switching method
US10033644B2 (en) Controlling congestion controlled flows
Siekkinen et al. A root cause analysis toolkit for TCP
JP2012257010A (en) Communication device, communication method, and remote monitoring system
US9319910B2 (en) Communication system
CN101350765B (en) Network flow detection method
Petrangeli et al. Network-based dynamic prioritization of http adaptive streams to avoid video freezes
US20160344611A1 (en) Method and control node for handling data packets
JP5894245B2 (en) An estimation method for loss rate in packetized networks
US9705745B2 (en) System and method for virtualizing software defined network (SDN)-based network monitoring
CN105007235B (en) A kind of jamming control method in wireless multimedia sensor network
JP2013030944A (en) Packet capture processing method and device
JP2013179678A (en) Router device
CN104092623A (en) Method and device used for carrying out overload protection on network node or server
JP2021512567A (en) Systems and methods for identifying candidate flows in data packet networks
CN109167734B (en) Method and device for identifying transmission control protocol state
CN109257282B (en) Data transmission method and device
CN109005126B (en) Data stream processing method, device and computer readable storage medium
JP5440200B2 (en) Relay device and bandwidth control method
JP5923914B2 (en) Network state estimation apparatus and network state estimation program
JP2013240017A (en) Network delay measuring device and network delay measuring method

Legal Events

Date Code Title Description
PB01 Publication
C06 Publication
SE01 Entry into force of request for substantive examination
C10 Entry into substantive examination
WD01 Invention patent application deemed withdrawn after publication

Application publication date: 20141008

WD01 Invention patent application deemed withdrawn after publication