CN108809764A - The network based on GPU accesses overtrick and connects number evaluation method under sliding window - Google Patents
The network based on GPU accesses overtrick and connects number evaluation method under sliding window Download PDFInfo
- Publication number
- CN108809764A CN108809764A CN201810618223.2A CN201810618223A CN108809764A CN 108809764 A CN108809764 A CN 108809764A CN 201810618223 A CN201810618223 A CN 201810618223A CN 108809764 A CN108809764 A CN 108809764A
- Authority
- CN
- China
- Prior art keywords
- overtrick
- sliding
- estimator
- aip
- candidate
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/08—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
- H04L43/0876—Network utilisation, e.g. volume of load or congestion level
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/48—Program initiating; Program switching, e.g. by interrupt
- G06F9/4806—Task transfer initiation or dispatching
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5061—Partitioning or combining of resources
- G06F9/5066—Algorithms for mapping a plurality of inter-dependent sub-tasks onto a plurality of physical CPUs
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2209/00—Indexing scheme relating to G06F9/00
- G06F2209/50—Indexing scheme relating to G06F9/50
- G06F2209/5018—Thread allocation
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Environmental & Geological Engineering (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
Overtrick is accessed the present invention provides the network based on GPU under sliding window and connects number evaluation method, and data structure is maintained in the video memory of GPU, is carried out overtrick connection number estimation parallel by GPU, is included the following steps：All messages in initialization sliding estimator matrix on GPU and one the candidate overtrick list of sliding, scanning timeslice, in timeslice at the end of estimate that overtrick connects number, window sliding.The present invention can detect overtrick and estimator connection number under sliding window, and estimated value is more acurrate, is not influenced by window start times, and the overtrick across time boundary will not be omitted；There is lower time delay than existing method, overtrick connection number can be estimated in time；Energy real-time parallel handles high-speed network flow, reduces computer capacity, improves the bulk velocity of algorithm, and sliding estimator matrix can be simultaneously by multiple IP address to updating without output mistake.
Description
Technical field
The invention belongs to express network management and parallel computing field, it is related under a kind of sliding window based on GPU's
The real-time overtrick of express network connects number estimating techniques.
Background technology
Overtrick is a kind of special host.Assuming that there are two networks：ANet and BNet.The two networks pass through border routing
Device ER is communicated.ANet can be the network of Metropolitan Area Network (MAN) either some country.BNet be another Metropolitan Area Network (MAN) either
Internet.All flows between ANet and BNet can be observed from ER.For a host " aip " in ANet,
Host in the BNet that following period of time is communicated by ER with aip is known as the connection number of aip.When the connection number of aip is big
When some specified threshold θ, aip is known as an overtrick.
Overtrick is related with many events in network, such as DDoS, scanning attack etc..It is also network management that host, which connects number,
With an important attribute in measurement.Since overtrick only accounts for the sub-fraction of whole hosts, it is one to carry out emphasis monitoring to overtrick
The efficient express network real-time management method of kind.
It can also be sliding time window that time window in overtrick defines, which can be discrete time window,.Existing overtrick
Connection number calculating method all carries out under discrete window.Under discrete time window, it is not overlapped between two neighboring window
Time.Fig. 1 describes the traffic partition situation under discrete window and sliding window.
Flow between network A Net and BNet can be divided according to the identical sequential time slices of size.Timeslice
Length can be 1 second, 1 minute or be set as other values according to concrete application.One Digital ID of each timeslice.Such as Fig. 1
Shown in top half, a sliding time window includes the continuous k timeslice since t-th of timeslice, is denoted as W (t, k).
Sliding time window moves forward a timeslice every time.Therefore identical containing k-1 between two neighboring sliding time window
Timeslice.When k is 1, there is no the overlapped period between two neighboring time window.Time window at this time is exactly
Discrete time window, as shown in the lower half portion of Fig. 1.
Network packet continuously transmits between network A Net and BNet.If by their discrete divisions, number is connected
Estimation result can be influenced by time window starting point.For example, it is a host in ANet to enable aip.As shown in Fig. 2,
Assuming that its only main-machine communication within the period from p0 to p1 and in BNet." communication " of this paper refers to aip in BNet
Host transmission data packet or receive data packet from the host in BNet.Pkt (aip, p0, p1) expressions are enabled to be arrived in period p0
The data packet stream communicated with aip in p1.
P0 is an inner time point of discrete time window W (t, 1), and p1 is inner one of discrete time window W (t+1,1)
Time point.P2 is enabled to indicate the time boundary of W (t, 1) and W (t+1,1).The data packet of aip can be divided into two parts：From p0 to p2
Data packet in period and the data packet from p2 to p1 in the period.Be denoted as respectively Pkt (aip, p0, p2) and Pkt (aip,
p2,p1).Under discrete time window, the connection number calculating of aip can generate mistake due to following two：
Half loses (Half loss, HL) is more than θ according to the connection number that Pkt (aip, p0, p2) is calculated；But from Pkt
The connection number that (aip, p2, p1) is calculated is less than θ.Aip is only detected as overtrick W (t, 1) is inner at this time.And the company of aip
Connect the part that number can be lost between period p2 to p1.
Complete lose (Full loss, FL) is from Pkt (aip, p0, p2) and the inner connections calculated separately of Pkt (aip, p2, p1)
Number is both less than θ, but from Pkt (aip；p0；P1) the inner connection number being calculated is more than θ.At this point, overtrick aip is in W (t, 1) and W
(t+1,1) inner to be all detected.
With | HL | and | FL | indicate the quantity of both hosts.| HL | and | FL | it is related with the size of timeslice.With Cs tables
Show the length of a timeslice.Cs is smaller, then the inner data packet numbers for including of Pkt (aip, p0, p2) and Pkt (aip, p2, p1)
It is smaller, | HL | and | FL | it can also reduce accordingly.But under discrete time window, Cs is exactly the time span measured, cannot
Arbitrarily change.Need to calculate the application that overtrick in longer period connects number for some, discrete time window can cause very big
Loss.
Cs can also influence the delay that overtrick connection number calculates.If the aip connections being calculated from Pkt (aip, p2, p1)
Number is more than θ, and the connection number of aip should immediately be calculated in p1.But under discrete time window, overtrick connects several when each
Between calculate at the end of window it is primary.Therefore delay is Cs-p1.
The disadvantages mentioned above of discrete time window is since its movable length is equal to the length of window.With discrete time window
Difference, for sliding window, the length of each timeslice can be much smaller than the length of window.Use below Csd indicate it is discrete when
Between the next timeslice of window length, i.e., the length of one discrete time window；When indicating that sliding window is next using Css
Between piece length.
Sliding time window has better measurement result than discrete time window.It moves forward the distance of Css every time,
And the storage of increment type and the connection number for updating host.Due to being influenced smaller, sliding window tool by time window starting point
There is accuracy rate more higher than discrete window.Since in the case of same time length of window, Css is much smaller than Csd, so sliding
Time window can more timely calculate overtrick connection number.
When having uniform window length with discrete time window, Css can be set to the value of very little to reduce | HL |,
| FL | and computing relay.If k is arranged to sufficiently large, under sliding window | HL | and | FL | it can be ignored.Such as we
One hour truthful data is had collected from the express network of 40Gb/s to test | HL | and | FL | value.This segment data flow from
The 13 of on October 21st, 2017：00 starts lasting one hour, and is also one of experimental data set below.As θ=1024, Csd
At=300 seconds, Css=1 seconds and k=300, each sliding window average detected to 574 overtricks.With sliding time window phase
Than under discrete time window | HL | under much average 50, | FL | it is average 14 more.
This example illustrates that sliding time window has higher accuracy rate than discrete time window well.Assuming that I
To calculate in ANet overtrick connection number (in BNet overtrick connection number can be calculated with identical algorithm, it is only necessary to by IP
Address swaps).
It is inner all using aip as source or place address in time window W (t, k) for a host aip in network A Net
, the data packet stream of aip is constituted by the data packet of border router ER, is denoted as Pkt (aip, t, k).
By extracting another address in the inner each data packets of Pkt (aip, t, k) in addition to aip, pair of aip can be obtained
End main frame stream is denoted as HS (aip, t, k).For some host bip in BNet, if bip time window W (t, k) it is inner with
Aip is communicated, then bip may be multiple in the inner appearance of HS (aip, t, k).Because aip receives multiple data packets from bip, or to bip
Send multiple data packets.
Host inner HS (aip, t, k) constitutes the opposite end host complexes of aip, is denoted as OP (aip, t, k).OP(aip,t,
K) number of element is less than or equal to the number of HS (aip, t, k) inner element in.Remember that the number of OP (aip, t, k) inner element is | OP
(aip,t,k)|.Then | OP (aip, t, k) | it is exactly connection numbers of the aip under time window W (t, k).For some in ANet
Host aip, if | OP (aip, t, k) | >=θ, then aip be known as an inner overtrick of sliding window W (t, k).
Threshold θ is the positive integer that user defines according to different application.For example, θ could be provided as the past period
The average of interior All hosts connection number, or the connection number of server under normal circumstances.
Overtrick connection number estimation is relatively easy under discrete window, because it does not require to preserve host in for the previous period
State.But there are following two problems for estimation result：
1) calculated value is influenced by time window starting point, cannot detect the overtrick across time window boundary.
2) result of calculation has prodigious delay.
The two disadvantages of discrete time window come from its move mode.The time that discrete time window moves every time
Length is equal to the length of own.When discrete time length of window is bigger, computing relay and error rate will be higher.When sliding
Between window solve the above problem well because the time span that sliding window moves every time can be much smaller than the length of its own
Degree.For example, when the length of time window is set as 300 seconds, sliding time window every time can forward slip 1 second.And from
Scattered time window must move 300 seconds.For the express network of 40Gb/s, discrete time window can be than sliding time window missing inspection
14 overtricks across time boundary.But overtrick connection number calculating can be more more complicated than under discrete window under sliding time window.
The state for the previous period of preservation host is needed under sliding time window and the calculating more more frequent than discrete time window
Overtrick connects number.Existing method cannot all be run under sliding window, because they need to start institute in each period
Some data structures are initialized.The connection number information recorded in the period before losing in this way.And these methods exist
Plenty of time calculating overtrick is occupied at the end of each time window connects number.It has been more than sliding window next time to calculate the time
The length of piece.
It is connected in number estimation method in existing overtrick, bit matrix is the number of most common preservation host connection number information
According to structure.In scan data packet, the appearance of different hosts is indicated by the way that certain positions of bit matrix are set as 1.In order to
Can recover overtrick at the end of each period, data packet can be mapped to according to special method in bit matrix some or
Certain bits.Chinese remainder theorem (CRT) is to be used for determining the method which bit can be set earliest.By using
CRT, each position in bit matrix can be shared by different hosts.But the calculating of CRT is complicated, limits the speed of this method
Degree.A kind of improved method is that data packet is mapped to bit square by the method extracted using the principle and Bit String of Bloom filter
Different bits in battle array.In this method secondary IP address extract specific position Bit String as data packet in bit matrix
One index.By the way that the index of different location is combined to restore overtrick.Using the Bit String in IP address as index, though
It is so easy to operate, but randomness is not high, each bit in bit matrix cannot be used with identical probability.Lead to the party
Method memory usage is low, and accuracy rate is high.This method can not connect number to the overtrick under sliding window and estimate simultaneously.
Current network bandwidth is increasing, and parallel computation becomes one of the necessary method of processing express network stream in real time.
Many existing methods attempt by using high-speed internal memory, such as Static RAM SRAM, to accelerate the processing speed of data packet.
But SRAM capacity is limited, limits application of these methods in large scale network.And overtrick connection number estimation needs to occupy
A large amount of resource, and the processing speed of CPU just becomes the new bottleneck of method.GPU (graphics processor) has higher than CPU
Parallel processing capability is because it contains more computing units.It can be real-time when carrying out overtrick connection number estimation using GPU
Handle high-speed network flow.It is a kind of to tolerate wrong Hash table to filter stream record, while recording host using bitmap
The method of connection number realizes the parallel computation of host connection number on GPU for the first time.But this method is needed in scan data packet
When need to store the IP address of each stream record, and cannot directly recover overtrick.Number is attached to All hosts and calculates meeting
Occupy the plenty of time.And network packet is directly passed to GPU by this method, occupies a large amount of transmission bandwidth.Server with
Communication between GPU becomes the bottleneck of this method.
Invention content
To solve the above problems, the invention discloses the overtrick connection number estimation sides under a kind of sliding window based on GPU
Method carries out real time high-speed net under sliding window using a kind of new structure come the connection number of the record host of increment type based on GPU
Network accesses overtrick connection number estimation.This method includes two data structures of setting：Slide estimator matrix and the candidate overtrick of sliding
List, sliding estimator matrix is for checking that overtrick connects number with record overtrick.Sliding estimator matrix is by the cunning of u rows v row
The matrix of dynamic estimator composition.Method includes integrally Four processes：Initialization sliding estimator matrix and sliding are candidate on GPU
Overtrick list, all messages in one timeslice of scanning estimate overtrick connection number, window sliding at the end of timeslice.This
The data structure of method is maintained in the video memory of GPU；Four processes are run on GPU, and overtrick connection is carried out parallel by GPU
Number estimation.
In order to achieve the above object, the present invention provides the following technical solutions：
The network based on GPU accesses overtrick and connects number evaluation method under sliding window, includes the following steps：
Data structure initializes, and the data structure includes sliding estimator matrix and the candidate overtrick list of sliding；
All data packets in a timeslice are scanned, number is connected using sliding estimator matrix record host, is scanning
When data packet, judge whether host is overtrick, and the overtrick detected is added in candidate host list；
At the end of timeslice, the connection of each host in candidate overtrick list is calculated according to sliding estimator matrix
Number, output connection number are more than the host and its estimated value of threshold value；After the connection number for having estimated all candidate overtrick hosts, update is slided
Dynamic estimator matrix and candidate overtrick list；
By one timeslice of window forward slip, starts to scan the data packet in next timeslice, estimate next cunning
Overtrick in dynamic window connects number.
Specifically, including the following steps：
The first step, arrange parameter
The sliding estimator matrix being made of the sliding estimator of u rows v row is set, and each estimator that slides includes：By n
The overtrick indicator of bit composition；Obscuring estimator by the sliding that g integer forms, (for IPv4,8) value of g is；By g '
(fetching of g ' is the integer more than 1024 to the linear estimator of sliding of a integer composition, and g ' values are bigger, and valuation result is more smart
Really)；Sliding candidate list for preserving the candidate overtrick detected is set, since overtrick number is uncertain, so sliding is waited
Select list meeting dynamic change at runtime.
Second step, the initialization data structure on GPU
The memory headroom of distribution sliding estimator matrix on the video memory of GPU, initialization sliding estimator matrix will slide
The integer that all slidings obscure estimator in estimator matrix is set as maximum value.All lines of slide in estimator matrix will be slided
The integer of property estimator is set as maximum value.One piece of video memory space is reserved on GPU, for preserving candidate overtrick list.Due to
Overtrick number only accounts for the sub-fraction of whole hosts, so, reserved video memory space does not have to very big.
Third walks, and scans IP data packets, extracts IP address pair
The data packet in each timeslice is scanned, the IP address of each data packet is extracted, with obtaining the IP of each data packet
Location pair.IP address to be shaped like<aip,bip>Two tuples.Aip is a host in the network to be measured, and bip is to pass through
The host that border router is communicated with aip.On the server first to these IP to caching.When buffering area is full or at one
Between data packet scanning in piece when completing, by the IP address in buffering area on server to copying GPU's to by PCIe buses
In video memory.
4th step starts GPU threads, scans IP address pair
After the completion of duplication, GPU starts handles the IP address pair received with IP address to the thread of identical quantity.Each
GPU threads read an IP address of the IP address to buffering area<aip,bip>To handling.Slide each of estimator matrix
Corresponding hash function RHi, the i ∈ [0, v-1] of row.Aip is mapped to any one sliding valuation of the i-th row by RHi (aip)
Device.For the i-th row, a sliding estimators of RHi (aip) can be updated, and preserve the connection number of aip.
5th step, at the end of timeslice, estimation overtrick connects number
After scanning through all data packets in the timeslice at the end of timeslice, proceeds by overtrick connection number and estimate
It calculates.Host in candidate overtrick list is grouped, every group of 1024 hosts.For every group of host, carried out on GPU respectively
Overtrick connects number estimation.Evaluation method is for one group of overtrick host, to start 1024 threads, per thread estimates one respectively
The connection number of a host.Per thread estimates that the method for candidate host connection number is as follows.If the cunning of all sliding estimators of the i-th row
The number of integer of the value of linear estimator less than k be | LDR (i) |k.It calculates and slides linear estimation in sliding estimator matrix
The probability that each integer of device is setFor host aip, by sliding estimator matrix per a line
In it is corresponding with aip sliding estimator the linear estimator of sliding merge：The sliding of each sliding estimator is linear
The operation that the integer of estimator presses the maximum integer of all same positions merges to obtain a new sliding linear estimation
Device.The linear estimator of sliding after the merging is scanned, the number of integer of the value less than k is obtained | ULE (aip) |k.According to formula Calculate the connection number of aip.Each host in candidate overtrick list as stated above by
One GPU thread process obtains its connection number estimated value.It exports host of all estimated values more than or equal to θ and its connection number is estimated
Calculation value just completes the connection number estimation of the overtrick under a sliding window, and θ is the threshold value of overtrick.
6th step, update sliding estimator matrix and candidate overtrick list：
Complete after the overtrick connection number estimation under a sliding window to sliding estimator matrix and candidate overtrick list into
Row update.Each bit of the overtrick indicator of sliding estimator each of in sliding estimator matrix is set as 0.For every
Each integer that sliding in a sliding estimator obscures estimator is added 1 if its value is less than maximum value.For every
Each integer of the linear estimator of sliding in a sliding estimator is added 1 if its value is less than maximum value.Scanning is waited
Each host in overtrick list is selected, determines whether remain as candidate overtrick in its sliding estimator matrix in the updated.Such as
Fruit is the bit for being then added into new candidate overtrick list and updating corresponding overtrick indicator.Scan through all times
After selecting overtrick host, current candidate overtrick Host List is replaced with new candidate overtrick Host List.
One timeslice of window sliding is repeated to execute since walking third by the 7th step
It has estimated the overtrick connection number in a sliding time window, and has updated sliding estimator matrix and candidate overtrick
After list, time window is slided into a timeslice, third step is returned to, starts to scan all data in next timeslice
Packet.
Further, the method that some sliding estimator is updated in the 4th step includes the following steps：
It is with a hash function H3 that the g ' of the linear estimator of sliding of bip Random Maps to the sliding estimator is a first
One in integer, and the integer is set as 0.Then with a random Harsh function H1, bip is mapped to [0,232- 1] it
Between an integer H1 (bip).Calculate the least significant bit LSB (H1 (bip)) of H1 (bip).Certain number least significant bit be exactly
Since right, the position of first bit " 1 " in this is several.If LSB (H1 (bip)) is less than an integer τ, no longer to the IP
Address is to being further processed.Wherein τ=log2 (θ/g), θ are the threshold values of overtrick.If LSB (H1 (bip)) be more than etc.
In an integer τ, then the sliding of bip Random Maps to the sliding estimator is obscured to one in g integer of estimator with H2
It is a, and the integer is set as 0.Every a line of estimator matrix is slided according to the method described above to IP address pair<aip,bip>
It is handled.
Further, after all being updated per a line in above-mentioned sliding estimator matrix, if LSB (H1 (bip)) is more than or equal to
One integer τ needs further to test whether aip is overtrick.
Further, whether the test aip is that the method for overtrick includes the following steps：
It will be merged per sliding estimator corresponding with aip in a line：By the overtrick instruction of each sliding estimator
Device by bit with operation merge to obtain one it is new merge after overtrick indicator, by the cunning of each sliding estimator
The maximum integer that the integer of dynamic model paste estimator presses all same positions merges to obtain the fuzzy valuation of a new sliding
Device.
Calculating the number that sliding obscures integer of all values less than k in estimator will no longer if quantity is less than ρ * g
It is further processed, whereinIf quantity is detected as one more than or equal to ρ * g, aip
Overtrick.When aip is detected as overtrick, need first to determine whether aip in the sliding time window has been detected as overtrick,
If being detected as overtrick, without being further processed, it is not such as detected as overtrick, then aip is added to candidate overtrick
List, and H4 (aip) bit of the overtrick indicator in sliding estimator corresponding with aip in every row is set as 1.
Further, determine that the method whether aip has been detected as overtrick in the sliding time window includes as follows
Step：
Aip is mapped to some random bit position H4 (aip) in overtrick indicator with a hash function H4, if
The position is 1 in overtrick indicator after above-mentioned merging, then it represents that aip is in candidate overtrick list；If in above-mentioned merging
The position is 0 in overtrick indicator afterwards, then it represents that aip is not in candidate overtrick list.
Compared with prior art, the invention has the advantages that and advantageous effect：
1. this method can detect overtrick and estimator connection number under sliding window, and other methods can only estimate discrete window
Overtrick under mouthful connects number.This method obviously has superiority, and estimated value is more acurrate, is not influenced by window start times, no
The overtrick across time boundary can be omitted.
2. this method has lower time delay than existing method, overtrick connection number can be estimated in time.We
Method all can connect number to overtrick at the end of each timeslice and estimate.Since the length of timeslice is far smaller than time window
Length, so this method can more timely estimate that overtrick connects number.
3. this method can handle high-speed network flow in real time.Sliding estimator matrix can be simultaneously by multiple IP address to more
Newly without output mistake.In order to reduce the processing time of algorithm at the end of each timeslice, this method is in scan data packet
When generate candidate overtrick list in real time.By calculating the connection number of host in candidate list at the end of timeslice, reduce
Computer capacity improves the bulk velocity of algorithm.The method of the present invention can be real in a large amount of network packet of GPU parallel processings
When estimation express network under overtrick connect number.
Description of the drawings
Fig. 1 is division state of the network flow under sliding window and discrete window；
Fig. 2 is the mistake caused by across time boundary under discrete time window；
Fig. 3 is sliding estimator matrix structure schematic diagram；
Fig. 4 is the IP address pair scanned on GPU in each timeslice；
At the end of Fig. 5 is each timeslice, estimate that overtrick connects number on GPU；
Fig. 6 is that the present invention connects number appraising model to overtrick under sliding window；
Fig. 7 is the sliding time window example being made of three timeslices；
Fig. 8 is hash function mapping citing；
Fig. 9 be specific implementation mode in initialize after sliding estimator matrix；
Figure 10 is sliding estimator matrix of first IP address to after handled in first timeslice；
Figure 11 is the sliding estimator matrix scanned through after preceding 3 timeslices；
Figure 12 is that overtrick has been calculated at the end of first sliding time window to connect number and updated all integers and bit
Sliding estimator matrix behind position.
Specific implementation mode
Technical solution provided by the invention is described in detail below with reference to specific embodiment, it should be understood that following specific
Embodiment is only illustrative of the invention and is not intended to limit the scope of the invention.
The network based on GPU accesses overtrick and connects number evaluation method, overall architecture under sliding window provided by the invention
As shown in fig. 6, being how to estimate that the overtrick under sliding window connects number in order to illustrate the present invention, it is given below one and specifically shows
Example.
All network flows are divided according to the timeslice of fixed length size, this example hypothesis there are 5 timeslices：
S1, s2, s3, s4, s5；Each sliding window is made of continuous three timeslices, shares 3 time window Ws (1,3), W (2,
3), (3,3) W, as shown in Figure 7；
The IP address for including in each timeslice is to as follows：
IPpair (s1)=[<a1,b1>,<a2,b2>,<a1,b2>,<a1,b3>,<a1,b4>]；
IPpair (s2)=[<a2,b1>,<a2,b2>,<a1,b5>,<a1,b3>,<a1,b6>,<a1,b4>]；
IPpair (s3)=[<a1,b7>,<a2,b1>,<a1,b5>,<a1,b3>,<a1,b6>,<a1,b4>,<a1,b8>,
<a1,b9>,<a1,b10>]；
IPpair (s4)=[<a1,b7>,<a2,b1>,<a1,b5>,<a3,b3>,<a1,b6>,<a1,b4>,<a1,b11
>,<a1,b12>]；
IPpair (s5)=[<a1,b7>,<a3,b1>,<a2,b5>,<a1,b13>,<a1,b6>,<a1,b14>,<a1,
b15>]；
Include two hosts in ANet in above-mentioned example：A1, a2 and a3；Include 15 hosts in BNet：B1 to b15.It is super
Point threshold value is set as 5；Estimator arranged in matrix is slided to arrange for 2 rows 2；The integer g ' that linear estimator includes each is slided to be set as
8；The integer that sliding obscures estimator and slides in linear estimator is set as 3 bits, and each integers maximums are 7.Sliding is estimated
Overtrick indicator in value device is set as 2bit.
It can be obtained according to statistical method, the opposite end host complexes and connection of host a1, a2, a3 under different sliding time windows
Number is respectively：
OP (a1,1,3)={ b1, b2, b3, b4, b5, b6, b7, b8, b9, b10 }
OP (a1,2,3)={ b3, b4, b5, b6, b7, b8, b9, b10, b11, b12 }
OP (a1,3,3)={ b3, b4, b5, b6, b7, b8, b9, b10, b11, b12, b13, b14, b15 }
| OP (a1,1,3) |=10
| OP (a1,2,3) |=10
| OP (a1,3,3) |=13
OP (a2,1,3)={ b1, b2 }
OP (a2,2,3)={ b1, b2 }
OP (a2,3,3)={ b1 }
| OP (a2,1,3) |=2
| OP (a2,2,3) |=2
| OP (a2,3,3) |=1
OP (a3,1,3)={ }
OP (a3,2,3)={ b3 }
OP (a3,3,3)={ b1, b3 }
| OP (a3,1,3) |=0
| OP (a3,2,3) |=1
| OP (a3,3,3) |=2
Value after the host in host and BNet in ANet is mapped by different hash functions is as shown in Figure 8.
Based on above-mentioned example, the method for the present invention specifically comprises the following steps：
Step 1：Arrange parameter
As shown in figure 3, the line number of sliding estimator and columns are respectively two rows two row, u in setting sliding estimator matrix
=2, v=2.Bit numerical digit 2 in each sliding estimator contained by overtrick indicator, i.e. n=2.Sliding obscures valuation
Integer number in device is 8, slides the integer number position 8 in linear estimator；Each integer is made of 3 bits.
Step 2：Initialization sliding estimator matrix
The distribution sliding estimator matrix on the video memory of GPU, and it is initialized.The bit of all overtrick indicators
Position 0 is all arranged in position, and all slidings obscure estimator and are set as maximum value 7 with the integer for sliding linear estimator.After initialization
Sliding estimator matrix it is as shown in Figure 9.
Step 3：IP address is distributed on GPU to caching and overtrick list memory space
One piece of memory space is distributed on the video memory of GPU for receiving the IP address pair transmitted from server.IP is set
20 IP address pair can be preserved in address caching.The space size for cached IP address pair then distributed on GPU is：160
Byte.One piece of video memory space is reserved on GPU, for preserving the candidate overtrick list detected.The candidate overtrick list of setting is most
10 candidate overtricks detected can be mostly preserved, then the reserved space size for preserving candidate overtrick list is 40 words
Section.
Step 4：Scan the data packet in a timeslice
When since a timeslice, the data packet in the timeslice is scanned, IP address pair is extracted from each IP data packets,
And by IP address to being saved in the caching on server.A total of 5 data packets in the 1st timeslice, from each data
Packet extraction IP address pair, obtains the IP address in the timeslice to sequence IPpair (s1).
Step 5：IP address is replicated to arriving GPU video memorys
It, will be on server when IP address scans through all IP data packets to caching full or one timeslice ending of arrival
IP address to caching by PCIe buses copy to the IP address on GPU to cache.One shares 5 in the 1st timeslice
IP address pair, and IP address is to can at most preserve 20 IP address pair in caching.So will just be cached at the end of timeslice
5 IP address in area are to copying on GPU, as shown in Figure 4.
Step 6：Start GPU threads, handles IP address pair
When having replicated the caching of IP address pair, starts on GPU and receive IP address to the identical thread of quantity, each
One IP address pair of thread process.In first timeslice, GPU cachings receive 5 IP address pair, so starting 5 on GPU
A thread, per thread handle an IP address pair.
Step 7：It finds and is sliding in estimator matrix with IP address to corresponding sliding estimator.
For each IP address pair, corresponding sliding estimator in the i-th row is found according to hash function RHi ().For
First IP address pair in one timeslice<a1,b1>, RH0 (a1)=0, RH1 can be learnt in hash function mapping table
(a1)=1.So IP address pair<a1,b1>The 1st sliding estimator of the first row of sliding estimator matrix can be mapped to
SE [0,0] is inner, and second sliding estimator SE [1,1] for being mapped to the second row is inner.
Step 8：Linear estimator is slided in update
In every row, update with IP address to the linear estimator of sliding in corresponding sliding estimator.According to Hash letter
Number H3 (), 0 is set as by the integer slided in linear estimator.For first IP address in first timeslice
It is right<a1,b1>, H3 (b1)=0 can be learnt in hash function mapping table.So sliding estimator SE [0,0] and SE [1,1]
In first integer of the linear estimator of sliding can be arranged to 0.
Step 9：Calculate the least significant bit of IP address, it is determined whether update sliding obscures estimator
IP address is subjected to Random Maps to second inner IP address using hash function H1 (), after then calculating mapping
Value least significant bit.Big least significant bit will be calculated to be compared with τ=log2 (θ/g).If least significant bit
Estimator is then obscured according to subsequent step update sliding more than τ.Sliding is not updated if least significant bit is less than τ obscures valuation
Device.It can be obtained according to Hash table, the least significant bit LSB (H1 (b1)) that second IP address of first IP address pair is calculated
It is 2.In this example, θ=5, g=8.The τ being calculated is less than 0, so needing to update the fuzzy estimator of sliding.
Step 10：Update sliding obscures estimator
In every row, updates and estimator is obscured to the sliding in corresponding sliding estimator with IP address.According to Hash letter
Number H2 (), 0 is set as by the integer slided in linear estimator.For first IP address in first timeslice
It is right<a1,b1>, H2 (b1)=1 can be learnt in hash function mapping table.So sliding estimator SE [0,0] and SE [1,1]
In second integer of the linear estimator of sliding can be arranged to 0.First IP address in first timeslice is handled
Sliding estimator matrix to after is as shown in Figure 10.
Step 11：The merging sliding for calculating IP address pair obscures estimator
After having updated the fuzzy estimator of sliding, all slidings corresponding with first IP address are obscured into estimators and are closed
And the sliding after being merged obscures estimator URE.Merging method is that each sliding is taken to obscure the maximum integer in estimator.
For first IP address pair<a1,b1>, sliding is obscured into estimator [0,0] and sliding obscures estimator [1,1] and merges,
It is URE={ 7,0,7,7,7,7,7,7 } that sliding after merging, which obscures integer set in estimator,.
Step 12：Estimator is obscured according to the sliding after merging, determines IP address to inner whether comprising overtrick.
After the fuzzy estimator URE of the sliding after merging is calculated, calculates URE mileage values and be less than timeslice in sliding window
The number of the integer of number | URE |k, will | URE |kCompared with ρ * g.If it is larger than or equal to ρ * g, then the IP address is to inner first
IP address is overtrick.In this example, rounding after ρ * g are calculated obtains 3.First IP address is scanned to rear | URE |kIt is 1, is less than 3, institute
Not to be overtrick at this time.The 4th IP address in the 1st timeslice is scanned through to rear, the merging sliding die for the a1 being calculated
Paste estimator is URE={ 7,0,0,0,7,7,7,7 }, and | URE |k=3.A1 is detected as overtrick at this time.
Step 13：After detecting overtrick, calculates and merge overtrick indicator
After detecting an overtrick, all overtrick indicators corresponding with first IP address are merged, are obtained
Overtrick indicator after merging.When merging, each overtrick indicator by bit and in the way of merge.In this example, scanning
A1 can be detected as overtrick by the 4th IP address in complete 1st timeslice to rear.After calculating SI [0,0] and SI [1,1] merging
Overtrick indicator USI={ 0,0 }.
Step 14：According to the overtrick indicator after merging, determine whether overtrick was added overtrick candidate list
Determine that overtrick is corresponding with which bit in overtrick indicator according to hash function H4 ().Test refers in merging overtrick
Show in device, whether which is arranged to 1.If being arranged to 1, then it represents that the overtrick is in candidate overtrick list；Such as
The fruit bit is 0, then it represents that the overtrick is to be added into candidate overtrick list.In this example, in the 1st timeslice is scanned through
A1 can be detected as overtrick by 4 IP address to rear.A1 is mapped to second bit of overtrick indicator by H4 ().In USI
Second bit is 0, so a1 be not added into candidate overtrick list.
Step 15：Candidate overtrick list is added in the overtrick detected
If step 12 detects overtrick, and determines that the overtrick was not added into candidate list in step 14, then need
The overtrick is added in candidate overtrick list.In this example, the 4th IP address in the 1st timeslice is scanned through to rear, can be incited somebody to action
A1 is detected as overtrick, and a1 is not in candidate overtrick list.Candidate overtrick list is added in a1.It is wrapped in candidate overtrick list at this time
The overtrick a1 detected containing one.
Step 16：After overtrick list is added in overtrick, the host is recorded in overtrick indicator
After candidate list is added in an overtrick, need the super of all sliding estimators corresponding with the overtrick host
The corresponding bit position of point indicator is set as 1.In this example, the 4th IP address in the 1st timeslice is scanned through to rear, can be incited somebody to action
A1 is detected as overtrick, and a1 is added in candidate overtrick list.SI [0,0] and SI [1,1] are corresponding with a1, and H4 (a1)=1.
Second bit of SI [0,0] and SI [1,1] are set as 1.Updated overtrick indicator is SI [0,0]={ 0,1 }, SI
[1,1]={ 0,1 }.
Step 17：All IP address in a caching are handled to rear, waiting is next to cache full or arrival time piece
Ending
When all complete IP address of GPU thread process to after, GPU wait for next time IP address it is full to caching, then opened from step 6
Begin, handles the IP address pair in new caching.If the caching is last in the last one timeslice of a sliding window
A IP address then starts to estimate overtrick connection number, as shown in Figure 5 to caching.When a sliding time window includes 3 in this example
Between piece.The sliding estimator matrix scanned through after preceding 3 timeslices is as shown in figure 11.
Step 18：It calculates in sliding estimator matrix is often gone and slides the probability that each integer of linear estimator is set
Statistics sliding estimator matrix slides linear estimator all values in often going are less than sliding window timeslice number
Integer number.The integer numbers of all linear estimators of sliding just obtain sliding and estimate in by obtained integer number divided by often going
Value device matrix slides the probability that each integer of linear estimator is set in often going.This example example, all lines of slide of the first row
Property estimator share 10 integer numerical value be less than 3；There are 7 in second row.In the often capable linear estimator of sliding one share 16 it is whole
Number.So it is 62.5% that the first row, which slides the probability that the integer of linear estimator is set, the second behavior 50%.
Step 19：Calculate the probability for merging and sliding that each integer is set in linear estimator
The probability multiplication that each integer that linear estimator is slided in every row is set obtains merging sliding linear estimation
The probability that each integer is set in deviceIn this example, the integer that the first row and the second row slide linear estimator is set
Probability be respectively 62.5% and 50%,
Step 20：Start GPU threads and calculates overtrick connection number
The candidate overtrick list of scanning calculates each overtrick connection number detected according to linear estimator is slided.On GPU
Start the thread of fixed quantity, per thread handles the overtrick in a candidate overtrick list.In this example, when scanning through third
Between after piece, there are one overtrick a1 in candidate overtrick list.Start a thread on GPU, calculates the connection number of a1.
Step 21：Linear estimator is slided in the merging for calculating overtrick
All linear estimators of sliding corresponding with overtrick are merged, the linear estimator of the sliding after being merged
ULE.When merging, the maximum value that linear estimator takes corresponding integer is each slided.In this example, scan through in the 3rd timeslice
All IP address start the connection number for calculating a1 to rear.A1 is corresponding with SLR [0,0] and SLR [1,1].Calculate SLR [0,0] and
The linear estimator ULE={ 0,0,0,0,0,0,0,0 } of sliding after SLR [1,1] merging.
Step 22：Calculate the integer number for merging and being set in the fuzzy estimator of sliding
After the fuzzy estimator ULE of the sliding after merging is calculated, calculates ULE mileage values and be less than timeslice in sliding window
The number of the integer of number | ULE |k.When | ULE |kWhen equal to g ', | ULE |kIt is set as g ' -1.|ULE|kNumber is connected for calculating.
In this example, all IP address in the 3rd timeslice are scanned through to rear, the linear estimator ULE=of sliding after calculating to merge
{0,0,0,0,0,0,0,0}。|ULE|k=7.
Step 23：Calculate connection number
It is calculated | ULE |kAfterwards, according to formulaIt calculates overtrick and connects number.This example
In,G '=8.Host a1's | ULE |k=7.A1 connection number estimated values are 13.64, rounding 13.?
In one sliding time window, the connection number of a1 is 10, close to estimated value.
Step 24：Export overtrick and its connection number
For each overtrick in candidate overtrick list, after its connection number has been calculated, all connection number estimated values are more than
The host and its connection number of overtrick threshold value copy on server, as further analyzing processing.Threshold is less than for estimated value
The host of value then not as overtrick, is no longer output on server.The connection number estimated value of a1 is 13 in this example.Overtrick has been calculated
A1 and its connection number estimated value are output in server memory space after connection number.
Step 25：Update overtrick indicator
After having estimated overtrick connection number, the bit of all overtrick indicators slided in estimator matrix is set as 0.
In this example, in scanning IP address clock synchronization, second bit of SI [0,0] and SI [1,1] are arranged to 1, need at this time its is extensive
It is 0 again.
Step 26：Update sliding obscures estimator
After having estimated overtrick connection number, all slidings in estimator matrix will be slided and obscure the integer progress of estimator more
Newly.If the integer value that sliding obscures estimator is less than maximum value, added 1.It slides and is obscured in estimator in this example
The maximum value of first integer is 7.Integer equal to 7 is not dealt with；For the integer less than 7, by itself plus 1.
Step 27：Linear estimator is slided in update
After having estimated overtrick connection number, the integer for sliding the linear estimator of all slidings in estimator matrix is carried out more
Newly.If an integer value for sliding linear estimator is less than maximum value, added 1.It is slided in this example in linear estimator
The maximum value of first integer is 7.Integer equal to 7 is not dealt with；For the integer less than 7, by itself plus 1.It has updated
Sliding estimator matrix later is as shown in figure 12.
Step 28：Determine each host in candidate's overtrick list update slide linear estimator after whether be still
Overtrick
Each host in candidate overtrick list is needed after having updated sliding estimator matrix according to after update
Sliding obscure estimator redefine whether it is still overtrick.According to updated sliding estimator matrix, calculate each
The merging sliding of host obscures estimator URE.Then URE mileage values are calculated and are less than the integer of timeslice number in sliding window
Number | URE |k, will | URE |kCompared with ρ * g.If it is larger than or equal to ρ * g, then the host in next sliding time window still
It is so overtrick.In this example, rounding after ρ * g are calculated obtains 3.According to updated sliding estimator matrix, the conjunction for the a1 being calculated
And slide and obscure estimator as URE={ 1,1,1,1,1,1,1,1 }, and | URE |k=7.A1 is in next sliding time window
Remain as overtrick.
Step 29：Delete non-overtrick
It is if a host in candidate overtrick list is not detected as overtrick in step 28, it is super from candidate
It is deleted in point list.
Step 30：Update overtrick indicator
If a host in candidate overtrick list is detected as overtrick in step 28, sliding estimator square is updated
Overtrick indicator corresponding with the overtrick host in battle array, avoid the host scan the IP address clock synchronization in new timeslice again by
Candidate overtrick list is added.Host a1 is detected as overtrick in step 26.SI [0,0] and SI [1,1] are corresponding with a1, and H4
(a1)=1.Second bit of SI [0,0] and SI [1,1] are set as 1.Updated overtrick indicator be SI [0,0]=
{ 0,1 }, SI [1,1]={ 0,1 }.
Step 31：Time window is slided into a timeslice, estimates the overtrick connection number of next time window
The overtrick connection number for having estimated a time window, after having updated sliding estimator matrix and candidate overtrick list,
Time window is slided into a timeslice, step 3 is repeated and starts to scan the IP address pair of next timeslice, estimate next cunning
Overtrick in dynamic time window connects number.
Continuous overtrick connection number estimation can be carried out to high-speed network flow under sliding window according to above-mentioned steps.
Real-time overtrick connection number calculating is an important content in network research field under sliding window.Incrementally updating and
It is its two difficult points to underestimate evaluation time.Algorithm proposed by the present invention solves the problems, such as this for the first time.In order to reduce every
The time of overtrick connection number is calculated at the end of a timeslice, the present invention obscures estimator when scan data packet using sliding
Generate a candidate overtrick list.It is a kind of connection number estimator of lightweight that sliding, which obscures estimator,.Due to its committed memory
Few processing speed is fast, and too many delay will not be brought when scanning number packet.At the end of each timeslice, the present invention utilizes line of slide
Property each candidate overtrick of estimator estimation connection number.The connection of each overtrick host can accurately be provided by sliding linear estimator
Number estimated value.
The technical means disclosed in the embodiments of the present invention is not limited only to the technological means disclosed in the above embodiment, further includes
By the above technical characteristic arbitrarily the formed technical solution of combination.It should be pointed out that for those skilled in the art
For, various improvements and modifications may be made without departing from the principle of the present invention, these improvements and modifications are also considered as
Protection scope of the present invention.
Claims (6)
1. the network based on GPU accesses overtrick and connects number evaluation method under sliding window, which is characterized in that include the following steps：
Data structure initializes, and the data structure includes sliding estimator matrix and the candidate overtrick list of sliding；
All data packets in a timeslice are scanned, number are connected using sliding estimator matrix record host, in scan data
Bao Shi judges whether host is overtrick, and the overtrick detected is added in candidate host list；
At the end of timeslice, the connection number of each host in candidate overtrick list is calculated according to sliding estimator matrix, it is defeated
Go out to connect host and its estimated value that number is more than threshold value；After the connection number for having estimated all candidate overtrick hosts, update sliding is estimated
It is worth device matrix and candidate overtrick list；
By one timeslice of window forward slip, starts to scan the data packet in next timeslice, estimate next sliding window
Overtrick in mouthful connects number.
2. the network based on GPU accesses overtrick and connects number evaluation method, feature under sliding window according to claim 1
It is, specifically comprises the following steps：
The first step, arrange parameter
The sliding estimator matrix being made of the sliding estimator of u rows v row is set, and each estimator that slides includes：By n bit
The overtrick indicator of composition；Estimator is obscured by the sliding that g integer forms；The sliding linear estimation being made of a integers of g '
Device；Sliding candidate list for preserving the candidate overtrick detected is set；
Second step, the initialization data structure on GPU
The memory headroom of distribution sliding estimator matrix on the video memory of GPU, initialization sliding estimator matrix will slide valuation
The integer that all slidings obscure estimator in device matrix is set as maximum value, and all slidings in sliding estimator matrix are linearly estimated
The integer of gauge is set as maximum value, remains for the video memory space for preserving candidate overtrick list in advance on GPU；
Third walks, and scans IP data packets, extracts IP address pair
The data packet in each timeslice is scanned, the IP address of each data packet is extracted, obtains the IP address pair of each data packet,
IP address is to being that structure is<aip,bip>Two tuples, wherein aip is a host in the network to be measured, and bip is logical
Cross the host that border router is communicated with aip；On the server first to these IP to caching, when buffering area is full or one
When data packet scanning in timeslice is completed, by the IP address in buffering area on server to copying GPU to by PCIe buses
Video memory in；
4th step starts GPU threads, scans IP address pair
After the completion of duplication, GPU starts handles the IP address pair received with IP address to the thread of identical quantity；
Each GPU threads read an IP address of the IP address to buffering area<aip,bip>To handling；
The every a line for sliding estimator matrix corresponds to hash function RHi, i a ∈ [0, v-1]；Aip is mapped to by RHi (aip)
Any one sliding estimator of i-th row, for the i-th row, a sliding estimators of RHi (aip) can be updated, and preserve aip's
Connect number；
5th step, at the end of timeslice, estimation overtrick connects number
After scanning through all data packets in the timeslice at the end of timeslice, overtrick connection number estimation is proceeded by：
Host in candidate overtrick list is grouped, every group of 1024 hosts are enterprising in GPU respectively for every group of host
Row overtrick connects number estimation；Evaluation method is：
For one group of overtrick host, start 1024 threads, per thread estimates the connection number of a host, per thread respectively
The method for estimating candidate host connection number is as follows：
If the number of integer of the value less than k of the linear estimator of sliding of all sliding estimators of the i-th row is | LDR (i) |k；
It calculates and slides the probability that each integer of linear estimator is set in sliding estimator matrix
For host aip, the linear estimator of sliding of sliding estimator corresponding with aip in estimator matrix being slided per a line
It merges：The integer of the linear estimator of sliding of each sliding estimator is pressed to the maximum integer of all same positions
Operation merges to obtain a linear estimator of new sliding；
The linear estimator of sliding after the merging is scanned, the number of integer of the value less than k is obtained | ULE (aip) |k；According to formula Calculate the connection number of aip；
Each host in candidate overtrick list obtains its connection number estimated value by a GPU thread process as stated above；
It exports host of all estimated values more than or equal to θ and its connection number estimated value just completes overtrick under a sliding window
Number estimation is connected, θ is the threshold value of overtrick；
6th step, update sliding estimator matrix and candidate overtrick list：
Sliding estimator matrix and candidate overtrick list are carried out more after completing the overtrick connection number estimation under a sliding window
Newly：
Each bit of the overtrick indicator of sliding estimator each of in sliding estimator matrix is set as 0；
Each integer that estimator is obscured for the sliding in each sliding estimator, if its value is less than maximum value, by it
Add 1；
For each integer of the linear estimator of sliding in each sliding estimator, if its value is less than maximum value, by it
Add 1；
Each host in the candidate overtrick list of scanning, determines whether remain as time in its sliding estimator matrix in the updated
Overtrick is selected, if it is, being added into new candidate overtrick list and updating the bit of corresponding overtrick indicator；
After scanning through all candidate overtrick hosts, replaces current candidate overtrick host with new candidate overtrick Host List and arrange
Table；One timeslice of window sliding is repeated to execute since walking third by the 7th step
It has estimated the overtrick connection number in a sliding time window, and has updated sliding estimator matrix and candidate overtrick list
Afterwards, time window is slided into a timeslice, returns to third step, starts to scan all data packets in next timeslice.
3. the network based on GPU accesses overtrick and connects number evaluation method, feature under sliding window according to claim 2
It is：The method that some sliding estimator is updated in 4th step includes the following steps：
First with a hash function H3 by a integers of g ' of the linear estimator of sliding of bip Random Maps to the sliding estimator
In one, and the integer is set as 0；
Then with a random Harsh function H1, bip is mapped to [0,232- 1] an integer H1 (bip) between；
The least significant bit LSB (H1 (bip)) of H1 (bip) is calculated, if LSB (H1 (bip)) is less than an integer τ, no longer
To the IP address to being further processed, wherein τ=log2 (θ/g), θ are the threshold values of overtrick；If LSB (H1 (bip))
The sliding of bip Random Maps to the sliding estimator is then obscured g integer of estimator by more than or equal to one integer τ with H2
In one, and the integer is set as 0；
Every a line of estimator matrix is slided according to the method described above to IP address pair<aip,bip>It is handled.
4. the network based on GPU accesses overtrick and connects number evaluation method, feature under sliding window according to claim 3
It is：After all update per a line in sliding estimator matrix, if LSB (H1 (bip)) more than or equal to an integer τ, need into
One pacing tries whether aip is overtrick.
5. the network based on GPU accesses overtrick and connects number evaluation method, feature under sliding window according to claim 4
It is：Whether the test aip is that the method for overtrick includes the following steps：
It will be merged per sliding estimator corresponding with aip in a line：The overtrick indicator of each sliding estimator is pressed
Bit with operation merge to obtain one it is new merge after overtrick indicator, by the sliding die of each sliding estimator
The maximum integer that the integer of paste estimator presses all same positions merges to obtain the fuzzy estimator of a new sliding；
The number that sliding obscures integer of all values less than k in estimator is calculated, if quantity will be carried out no longer less than ρ * g
It is further processed, whereinIf quantity is detected as an overtrick more than or equal to ρ * g, aip；
When aip is detected as overtrick, need first to determine whether aip in the sliding time window has been detected as overtrick,
If being detected as overtrick, without being further processed, it is not such as detected as overtrick, then aip is added to candidate overtrick
List, and H4 (aip) bit of the overtrick indicator in sliding estimator corresponding with aip in every row is set as 1.
6. the network based on GPU accesses overtrick and connects number evaluation method, feature under sliding window according to claim 5
It is：Determine whether the method that overtrick has been detected as in the sliding time window includes the following steps aip：
Aip is mapped to some random bit position H4 (aip) in overtrick indicator with a hash function H4, if above-mentioned
The position is 1 in overtrick indicator after merging, then it represents that aip is in candidate overtrick list；If after above-mentioned merging
The position is 0 in overtrick indicator, then it represents that aip is not in candidate overtrick list.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810618223.2A CN108809764B (en) | 2018-06-15 | 2018-06-15 | GPU-based network access over-point connection number estimation method under sliding window |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810618223.2A CN108809764B (en) | 2018-06-15 | 2018-06-15 | GPU-based network access over-point connection number estimation method under sliding window |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108809764A true CN108809764A (en) | 2018-11-13 |
| CN108809764B CN108809764B (en) | 2021-11-02 |
Family
ID=64086211
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810618223.2A Active CN108809764B (en) | 2018-06-15 | 2018-06-15 | GPU-based network access over-point connection number estimation method under sliding window |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108809764B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111585904A (en) * | 2020-04-30 | 2020-08-25 | 南京汉德数字孪生技术研究院有限公司 | Bidirectional coflow scheduling method based on prediction of concurrent connection number |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101038602A (en) * | 2007-04-19 | 2007-09-19 | 复旦大学 | Clock deviation arrangement method driven by production yield under technique parametric variation |
| CN101227318A (en) * | 2007-12-04 | 2008-07-23 | 东南大学 | Method for overtrick real-time detection of high speed network flow quantity |
| CN104866781A (en) * | 2015-05-27 | 2015-08-26 | 广西师范大学 | Privacy protection method for community detection application-oriented social network data publication |
| US9239983B1 (en) * | 2011-09-02 | 2016-01-19 | Lockheed Martin Corporation | Quantifying a condition with a neural network |
-
2018
- 2018-06-15 CN CN201810618223.2A patent/CN108809764B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101038602A (en) * | 2007-04-19 | 2007-09-19 | 复旦大学 | Clock deviation arrangement method driven by production yield under technique parametric variation |
| CN101227318A (en) * | 2007-12-04 | 2008-07-23 | 东南大学 | Method for overtrick real-time detection of high speed network flow quantity |
| US9239983B1 (en) * | 2011-09-02 | 2016-01-19 | Lockheed Martin Corporation | Quantifying a condition with a neural network |
| CN104866781A (en) * | 2015-05-27 | 2015-08-26 | 广西师范大学 | Privacy protection method for community detection application-oriented social network data publication |
Non-Patent Citations (4)
| Title |
|---|
| JIE XU，ET AL.，: ""Abstract Super Points from Core Network by Unique Candidate List"", 《2017 INTERNATIONAL CONFERENCE ON GREEN INFORMATICS (ICGI)》 * |
| JIE XU，ET AL.，: ""High Speed Network Super Points Detection Based on Sliding Time Window by GPU"", 《2017 IEEE INTERNATIONAL SYMPOSIUM ON PARALLEL AND DISTRIBUTED PROCESSING WITH APPLICATIONS AND 2017 IEEE INTERNATIONAL CONFERENCE ON UBIQUITOUS COMPUTING AND COMMUNICATIONS (ISPA/IUCC)》 * |
| 周爱平,: ""高速网络流量测量关键问题研究"", 《中国博士学位论文全文数据库-信息科技辑》 * |
| 程光，: ""基于自适应抽样的超点检测算法"", 《中国科学E辑:信息科学》 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111585904A (en) * | 2020-04-30 | 2020-08-25 | 南京汉德数字孪生技术研究院有限公司 | Bidirectional coflow scheduling method based on prediction of concurrent connection number |
| CN111585904B (en) * | 2020-04-30 | 2022-07-08 | 南京汉德数字孪生技术研究院有限公司 | Bidirectional coflow scheduling system based on prediction of concurrent connection number |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108809764B (en) | 2021-11-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104516825B (en) | Cache memory system and its operating method | |
| CN102915347B (en) | A kind of distributed traffic clustering method and system | |
| US10257456B2 (en) | Hardware friendly virtual frame buffer | |
| WO2019024623A1 (en) | Flow measurement method, device and system | |
| CN104156400B (en) | The storage method and device of a kind of mass network flow data | |
| CN101841438B (en) | Method or system for accessing and storing stream records of massive concurrent TCP streams | |
| CN108090079A (en) | A kind of chat record processing method and processing device of instant messenger | |
| CN104699422A (en) | Determination method and determination device of cache data | |
| CN107241305A (en) | A kind of network protocol analysis system and its analysis method based on polycaryon processor | |
| CN112671611A (en) | Sketch-based large stream detection method and device | |
| CN105938480A (en) | RFID redundant data cleansing method and system based on DTBF | |
| CN113839835B (en) | Top-k flow accurate monitoring system based on small flow filtration | |
| CN106501695A (en) | A kind of high-tension cable local discharge signal acquisition processing system and method | |
| CN106874319A (en) | The distributed statistical method and device of click volume | |
| Zhou et al. | Per-flow counting for big network data stream over sliding windows | |
| CN110532307A (en) | A kind of date storage method and querying method flowing sliding window | |
| CN108809764A (en) | The network based on GPU accesses overtrick and connects number evaluation method under sliding window | |
| Nyang et al. | Recyclable counter with confinement for real-time per-flow measurement | |
| US8886878B1 (en) | Counter management algorithm systems and methods for high bandwidth systems | |
| JP3446704B2 (en) | Shaper and scheduling method used therefor | |
| CN105611269B (en) | Real-time disparity computation system based on FPGA | |
| Lou et al. | An efficient adaptive denoising sketch for per-flow traffic measurement | |
| CN106502592A (en) | Solid state hard disc caching block recovery method and system | |
| CN104394091B (en) | A kind of network redundancy method for recognizing flux based on uniform sampling | |
| CN116303585A (en) | Flag bit-based data stream counting method, flag bit-based data stream counting equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |