CN101800771B - Copy selection method based on kernel density estimation - Google Patents

Abstract

The invention discloses a copy selection method based on kernel density estimation, belonging to the computer network technology field. The method comprises the following steps: divide copies in a network into old copies and new copies, for old copies, select a best old copy according to history data by utilizing a kernel density estimation policy; for new copies, select a best new copy according to current bandwidth condition of a node in which the new copies exist; calculate and compare the best new copy and the best old copy, therefore, select a best copy from multiple copies which have a same logical file name. The copy selection method based on kernel density estimation is suitable to a dynamic low-side network and is especially suitable to the condition with frequently changing network state. User access delay and bandwidth consumption can be reduced by the method and the network performance is increased.

Description

A kind of copy selection method based on Density Estimator

Technical field

The invention belongs to technical field of the computer network, relate to a kind of copy selection method based on Density Estimator be applied in dynamic low side network.

Background technology

Along with the continuous progress of computer networking technology, net list has revealed the advantages such as powerful data-handling capacity, parallel data transfer function and internal metadata management.When mass data is shared in the world by network, very important effect has been played in the replica management service.The replica management technology can reduce network delay and bandwidth consumption in data transmission procedure.It mainly comprises: replica location, copy are selected and copy creating.

Copy is selected technology, is, when a lot of copies are arranged in network, by selecting most suitable copy, to reach the fastest purpose of access response.The key issue that copy is selected is the response time for each physical copy of each its correspondence of logic copy prediction.In the low side network, the transmission rate of data is faster, and the response time of copy is just shorter.

At present, copy selection method mainly is divided into two kinds.A kind of is to take the copy performance as basis, selects the method for copy by the response time of prediction copy.Such as people such as X.Shen, use the I/O model to decide the data response time, but the method is very complicated, is difficult to measurement result exactly.

Another method is based on the method for historical information predicated response time, and this method is widely adopted.Wherein, by people such as Wolski, proposed a kind of copy selection method based on historical information end to end, but this method can not be predicted in the actual file transmission of using GridFTP exactly.And the people such as Vazhkudai and Schopf utilize the GridFTP daily record data to formulate regression prediction method.The people such as Rashedur M.Rahman have proposed a kind of copy selection method of use k-Nearest Neighbor (KNN) principle, yet, even one newly adds the best node of copy of network also may not can be selected, because this method based on KNN is only selected optimal node from the historical data of copy, and the copy newly added does not have enough historical datas.In addition, all methods recited above all need abundant historical data and memory space.

In addition, by people such as Hu, propose the IBL algorithm, can utilize a small amount of data to carry out copy effectively and select.The similitude of this system of selection based between request example and training example.It uses Euclidean distance to calculate the distance between two examples.The tactful GRESS that also has another network architecture based on open.But these methods all are only applicable to Data Grid, and are not suitable for dynamic low side network.

Summary of the invention

The objective of the invention is, for solving how to select the problem of best copy from a plurality of copies of low side dynamic network, to propose a kind of copy selection method based on Density Estimator.This method can reduce user's access delay and bandwidth consumption, utilizes a small amount of historical data to select best copy, improves the entire system performance.

For achieving the above object, the technical solution adopted in the present invention is as follows:

A kind of copy selection method based on Density Estimator.

At first carry out related definition, specific as follows:

Definition 1: the node of request access file is called user node.

Definition 2: a threshold value λ is set, if the jumping figure between node and user node is less than λ, this node is exactly the node close with user node so, is called the adjacent node of user node.

Definition 3: a time threshold σ is set, if the node line duration that the creation-time of a copy is less than σ or this copy place apart from time interval of current time is less than σ, this copy is called latest copy; Otherwise this copy is called old copy.

Definition 4: at t _ithe number sum of the copy on the some nodes of other node visit is called the offered load of this access node constantly, is designated as load _i.The historical data of the offered load of this node is designated as { (t ₀, load ₀), (t ₁, load ₁) ..., (t _n, load _n).Total daily record time that T is node, T=t _n-t ₀.

Definition 5: the offered load that makes the prediction of node is predictLoad, and

wherein, total daily record time that T is node, load _imean t _ithe offered load of moment node.

On basis defined above, carry out following work, its flow process as shown in Figure 1:

It is { replica that setting tool has the copy of identity logic filename ₁, replica ₂..., replica _n, if be that user node u is from copy { replica ₁, replica ₂..., replica _nin select best copy, concrete steps are as follows:

Step 1, according to the definition 3, the copy in network is divided into to old copy and latest copy.

That is, at copy set { replica ₁, replica ₂..., replica _nin, if certain copy replica wherein _i, the creation-time of 1≤i≤n is less than time threshold σ apart from the time interval of current time, or the node line duration at this copy place is less than time threshold σ, and this copy is latest copy; Otherwise this copy is old copy.Mode like this, find out institute in copy set have been friends in the past copy and latest copy.If have been friends in the past copy in copy set,, for had been friends in the past copy, perform step two; If only have latest copy in copy set, perform step three.

Step 2, employing Density Estimator strategy, dope following network bandwidth according to historical data, selects best old copy from had been friends in the past copy.

For the copy set { oldReplica that m Geju City copy is arranged ₁, oldReplica ₂..., oldReplica _m, to every Geju City copy oldReplica wherein _i, 1≤i≤m, be at first the available bandwidth of the node prediction network at copy place, every Geju City.Preferably adopt following methods to realize:

The computing formula of the available bandwidth between node u and node v is as follows:

$\mathrm{available}\_\mathrm{bandwidth}(u,v)=\frac{\mathrm{bandwidth}(u,v)}{\mathrm{predictLoad}}---\left(1\right)$

Wherein, the offered load of the prediction that predictLoad is node v, bandwidth (u, v) is the network bandwidth between node u and node v.The offered load of prediction is larger, and available bandwidth is less.Utilize formula (1) to dope the available bandwidth of every Geju City copy, and be recorded as available_bandwidth (u, o _i), wherein, o _iold copy oldReplica _ithe node at place.

Then, utilize the Density Estimator strategy, calculate the possibility that every Geju City copy is accessed again---

1≤i≤m.Preferably adopt following methods to realize:

According to definition 2, determine all adjacent node { closeNode of user node ₀, closeNode ₁... closeNode _q.All copies of being accessed by user node and its adjacent node in network are { accessReplica ₁, accessReplica ₂..., accessReplica _d.At given nearest t in the time period, X={x ₁, x ₂..., x _k, Y={y ₁, y ₂..., y _k.X, the element in Y is respectively x _j=accessReplica _i, y _j=(accessReplica _i, p), 1≤j≤k, 1≤i≤d, wherein, (accessReplica _i, p) mean copy accessReplica _iaccessed accessReplica once by node p _i∈ { accessReplica ₁, accessReplica ₂..., accessReplica _d, p ∈ { u, closeNode ₀, closeNode ₁... closeNode _q.The sample that X is the cuclear density function, the node of the each access of Y record and accessed copy.

Utilize the accessed possibility of the every Geju City of Density Estimator policy calculation copy

formula as follows:

Wherein, H is parameter, H=h* ξ, wherein, and ξ=hop+1, hop≤λ, h is one and is greater than zero constant, value is as far as possible little, preferably is less than 0.1.This is because if H is too small, and estimated result will be by noise effect; If H is excessive, estimated result will too smoothly even be tending towards being uniformly distributed.ξ is for adjusting the H value.Hop is the jumping figure between user node and its adjacent node.Weights ω _jmeet

k () is positioned at x _j, j=1,2 ..., the kernel function of d, for example, can be used gaussian kernel function, and formula is as follows:

$K\left(u\right)={(-1)}^r{H}_r\left(u\right)\frac{1}{{\left(2\mathrm{\π}\right)}^{\frac{d}{2}}}\exp (-\frac{1}{2}u)---\left(3\right)$

Wherein, H _r(u) be the Hermite multinomial.

Afterwards, to { oldReplica ₁, oldReplica ₂..., oldReplica _min each copy calculate f (oldReplica _i), i ∈ 1,2 ..., m}, computing formula is as follows:

Wherein, bandwidth is the network total bandwidth.

Finally, according to the every Geju City copy obtained, and the computing formula of the best copy in old copy, select best old copy:

$f\left(\mathrm{bestOldReplica}\right)=\underset{i}{\mathrm{Max}}\left(f\left(\mathrm{oldRepli}{\mathrm{ca}}_i\right)\right),i\∈\{\mathrm{1,2}...m\}---\left(4\right)$

Forward step 3 after completing to.

Step 3, judged whether latest copy.If there is no latest copy, forward step 4 to; If latest copy is arranged,, according to latest copy place node current bandwidth situation, select best latest copy, preferably adopt following methods to realize:

Suppose to have l latest copy { newReplica ₁, newReplica ₂..., newReplica _l, l=n-m wherein.At { newReplica ₁, newReplica ₂..., newReplica _lmiddle g (the y that calculates _i), g (y _i) be node y _i(1≤i≤l) current available bandwidth:

$g\left(y_i\right)=\frac{\mathrm{bandwidth}(u,y_i)}{{\mathrm{cl}}_i},i\∈\{i,2,...l\}---\left(6\right)$

Wherein, cl _inode y _ithe current network load, y _i(1≤i≤l) is copy newReplica _ithe node at (1≤i≤l) place.

Best latest copy computing formula is as follows:

$g\left(\mathrm{bestNewNode}\right)=\underset{i}{\mathrm{Max}}\left(g\left(y_i\right)\right),i\∈\{\mathrm{1,2},...l\}---\left(7\right)$

The latest copy of selecting thus on node bestNewNode is best latest copy.Then perform step four.

If step 4 does not have latest copy, the old copy of the best that step 2 is selected is best copy; If there is no old copy, the best latest copy that step 3 is selected is best copy; If existing latest copy, the copy of haveing been friends in the past again, the available_bandwidth of more best old copy (u, bestOldReplica _i) value of value and the g (bestNewNode) of best latest copy, if available_bandwidth is (u, bestOldReplica _i) value large, best old copy is best copy; If g (bestNewNode) value is large, best latest copy is best copy.

Beneficial effect

The present invention contrasts prior art, by adopting the Density Estimator strategy, utilizes a small amount of historical data to select best copy, has effectively reduced user's access delay and bandwidth consumption, has improved the performance of network.The present invention is applicable to dynamic low side network, the especially situation of network state frequent variations.

The accompanying drawing explanation

The flow chart that Fig. 1 is the inventive method.

Embodiment

Below in conjunction with embodiment, the present invention will be further described.

Embodiment

Suppose as a logical resource LFN of a user node u request, have the copy of 10 these logical resources in network.

Step 1, according to the definition 3, establish σ=900s, all copies are divided into to latest copy and old copy, comprising 2 latest copy { newReplica ₀, newReplica ₁and 8 Geju City copy { oldReplica ₀, oldReplica ₁..., oldReplica ₁.Calculate the current network load of each latest copy, latest copy newReplica ₀, newReplica ₁the current network load be respectively 3,4; Calculate every Geju City copy { oldReplica ₀, oldReplica ₁..., oldReplica ₇predicted network load predictLoad be respectively 5,6,10,2,5,8,3,6.

Step 2, at first be the available bandwidth of the node prediction network at copy place, every Geju City, utilize formula (1) to predict the available bandwidth of every Geju City copy and be recorded as available_bandwidth (u, o _i), i ∈ 0,1 ... 7}, wherein, o _i, 0≤i≤7th, old copy oldReplica _i, the node at 0≤i≤7 places, u and o _i, the bandwidth between 0≤i≤7 is followed successively by: 800,500,1000,1000,500,1000,1000,1000; Corresponding available_bandwidth (u, o _i), the value of 0≤i≤7 is followed successively by 160,83,100,500,100,125,333,167; Secondly, establish λ=3, obtain all 3 adjacent node { closeNode of user node according to λ ₀, closeNode ₁, closeNode ₂.The copy that user node and its adjacent node were accessed is { oldReplica ₀, oldReplica ₃, oldReplica ₆.Record the old copy visit data in t=4800s, Y is { (oldReplica ₀, u), (oldReplica ₀, u), (oldReplica ₀, u), (oldReplica ₀, u), (oldReplica ₀, u), (oldReplica ₀, u), (oldReplica ₀, u), (oldReplica ₀, u), (oldReplica ₀, u), (oldReplica ₀, closeNode ₀), (oldReplica ₀, closeNode ₀), (oldReplica ₀, closeNode ₀), (oldReplica ₀, closeNode ₀), (oldReplica ₀, closeNode ₀), (oldReplica ₀, closeNode ₀), (0ldReplica ₀, closeNode ₀), (oldReplica ₀, closeNode ₀), (oldReplica ₀, closeNode ₀), (oldReplica ₀, closeNode ₀), (oldReplica ₀, closeNode ₀), (oldReplica ₀, closeNode ₀), (oldReplica ₃, u), (oldReplica ₃, u), (oldReplica ₃, u), (oldReplica ₃, u), (oldReplica ₃, u), (oldReplica ₃, u), (oldReplica ₃, u), (oldReplica ₃, u), (oldReplica ₃, u), (oldReplica ₃, u), (oldReplica ₃, u), (oldReplica ₃, u), (oldReplica ₃, u), (oldReplica ₃, u), (oldReplica ₃, u), (oldReplica ₃, u), (oldReplica ₃, u), (oldReplica ₃, u), (oldReplica ₃, u), (oldReplica ₃, u), (oldReplica ₃, u), (oldReplica ₃, u), (oldReplica ₃, u), (oldReplica ₃, u), (oldReplica ₃, u), (oldRepelia ₃, u), (oldReplica ₃, u), (oldReplica ₃, u), (oldReplica ₃, u), (oldReplica ₃, u), (oldReplica ₃, u), (oldReplica ₃, u), (oldReplica ₃, u), (oldReplica ₃, closeNode ₀), (oldReplica ₃, closeNode ₀), (oldReplica ₂, closeNode ₀), (oldReplica ₂, closeNode ₀), (oldReplica ₂, closeNode ₀), (oldReplica ₂, closeNode ₀), (oldReplica ₂, closeNode ₀), (oldReplica ₆, u), (oldReplica ₆, u), (oldReplica ₆, closeNode ₃), (oldReplica ₆, closeNode ₃), (oldReplica ₆, closeNode ₃), (oldReplica ₆, closeNode ₃), (oldReplica ₆, closeNode ₃), (oldReplica ₆, closeNode ₃), (oldReplica ₆, closeNode ₃), (oldReplica ₆, closeNode ₃), (oldReplica ₆, closeNode ₃), (oldReplica ₆, closeNode ₃), (oldReplica ₆, closeNode ₃), (oldReplica ₆, closeNode ₃), (oldReplica ₆, closeNode ₃), (oldReplica ₆, closeNode ₃), (oldReplica ₆, closeNode ₃), (oldReplica ₆, closeNode ₃), (oldReplica ₆, closeNode ₃), (oldReplica ₆, closeNode ₃), (oldReplica ₆, closeNode ₃), (oldReplica ₆, closeNode ₃), (oldReplica ₆, closeNode ₃), (oldReplica ₆, closeNode ₃), (oldReplica ₆, closeNode ₃), (oldReplica ₆, closeNode ₃), (oldReplica ₆, closeNode ₃), (oldReplica ₆, closeNode ₃), (oldReplica ₆, closeNode ₃), (oldReplica ₆, closeNode ₃), (oldReplica ₆, closeNode ₃), (oldReplica ₆, closeNode ₃), (oldReplica ₆, closeNode ₃), (oldReplica ₆, closeNode ₃), (oldReplica ₆, closeNode ₃), (oldReplica ₆, closeNode ₃), (oldReplica ₆, closeNode ₃), (oldReplica ₆, closeNode ₃), (oldReplica ₆, closeNode ₃), (oldReplica ₆, closeNode ₃), (oldReplica ₆, closeNode ₃), (oldReplica ₆, closeNode ₃), (oldReplica ₆, closeNode ₃), (oldReplica ₆, closeNode ₃) the sample X of Density Estimator is

OldReplica ₀accessed 21 times altogether, comprise that it is by node u access 9 times, by node closeNode ₀access 12 times oldReplica ₃accessed 40 times altogether, comprise that it is by node visit u33 time, by node closeNode ₀access 2 times, by node closeNode ₂access 5 times; OldReplica ₆accessed 44 times altogether, comprise that it is by node u access 2 times, by node closeNode ₂42 access time.The not accessed mistake of other old copy.Get h=0.015, for each Geju City copy oldReplica _i, i ∈ 0,1 ... 7} utilizes the computing formula of Density Estimator strategy to obtain the possibility that it is accessed again

i ∈ 0,1 ... 7}.

finally, utilize formula (4) to calculate f (oldReplica _i), i ∈ 0,1 ..., 7}, wherein the total bandwidth bandwidth value of network is 1000.f(oldReplica ₀)＝0.032，f(oldReplica ₃)＝0.44，f(oldReplica ₆)＝0.271，f(oldReplica ₁)＝f(oldReplica ₂)＝f(oldReplica ₄)＝f(oldReplica ₅)＝f(oldReplica ₇)＝0。Utilize formula (5) to obtain f (oldReplica ₃), i ∈ 0,1 ..., the value maximum of 7}, old copy oldReplica ₃for the old copy of the best.

Step 3, to all latest copy { newReplica ₀, newReplica ₁, utilize formula (6) to obtain g (y ₀)=333, g (y ₁)=250, wherein newReplica ₀the node at place is y ₀; NewReplica ₁the node at place is y ₁.According to formula (7), select newReplica ₀for best latest copy.

Step 4, comparison g (y ₀) and available_bandwidth (u, o ₃) value, available_bandwidth (u, o ₃) value large, therefore determine oldReplica ₃for best copy.

Claims (1)

1. the copy selection method based on Density Estimator, is characterized in that, in the low side dynamic network, at first carries out related definition, specific as follows:

Definition 1: the node of request access file is called user node;

Definition 2: a threshold value λ is set, if the jumping figure between node and user node is less than λ, this node is exactly the node close with user node so, is called the adjacent node of user node;

Definition 3: a time threshold σ is set, if the node line duration that the creation-time of a copy is less than σ or this copy place apart from time interval of current time is less than σ, this copy is called latest copy; Otherwise this copy is called old copy;

Definition 4: at t _ithe number sum of the copy on the some nodes of other node visit is called the offered load of this access node constantly, is designated as load _i; The historical data of the offered load of this node is designated as { (t ₀, load ₀), (t ₁, load ₁) ..., (t _n, load _n); Total daily record time that T is node, T=t _n-t ₀; I is integer, and N is integer;

Definition 5: the offered load that makes the prediction of node is predictLoad, and

i is positive integer, and N is positive integer; Wherein, total daily record time that T is node, load _imean t _ithe offered load of moment node;

On basis defined above, carry out following work:

It is { replica that setting tool has the copy of identity logic filename ₁, replica ₂..., replica _n, n is positive integer, if be that user node u is from copy { replica ₁, replica ₂..., replica _nin select best copy, concrete steps are as follows:

Step 1, according to the definition 3, the copy in network is divided into to old copy and latest copy;

That is, at copy set { replica ₁, replica ₂..., replica _nin, if certain copy replica wherein _i, the creation-time of 1≤i≤n is less than time threshold σ apart from the time interval of current time, or the node line duration at this copy place is less than time threshold σ, and this copy is latest copy; Otherwise this copy is old copy; Mode like this, find out institute in copy set have been friends in the past copy and latest copy; If have been friends in the past copy in copy set,, for had been friends in the past copy, perform step two; If only have latest copy in copy set, perform step three;

Step 2, employing Density Estimator strategy are selected best old copy from had been friends in the past copy according to historical data;

For the copy set { oldReplica that m Geju City copy is arranged ₁, oldReplica ₂..., oldReplica _m, to every Geju City copy oldReplica wherein _i, 1≤i≤m, be at first the available bandwidth of the node prediction network at copy place, every Geju City;

Wherein, described is that the method for available bandwidth of node prediction network at copy place, every Geju City is as follows:

The computing formula of the available bandwidth between node u and node v is as follows:

$\mathrm{available}\_\mathrm{bandwidth}(u,v)=\frac{\mathrm{bandwidth}(u,v)}{\mathrm{predictLoad}}---\left(1\right)$

Wherein, the offered load of the prediction that predictLoad is node v, bandwidth (u, v) is the network bandwidth between node u and node v; The offered load of prediction is larger, and available bandwidth is less; Utilize formula 1 to dope the available bandwidth of every Geju City copy, and be recorded as available_bandwidth (u, o _i), wherein, o _iold copy oldReplica _ithe node at place;

Then, utilize the Density Estimator strategy, calculate the possibility that every Geju City copy is accessed again

1≤i≤m; Concrete grammar is as follows:

According to definition 2, determine all adjacent node { closeNode of user node ₀, closeNode ₁... closeNode _q, q is integer; All copies of being accessed by user node and its adjacent node in network are { accessReplica ₁, accessReplica ₂..., accessReplica _d, d is positive integer;

At given nearest t in the time period, X={x ₁, x ₂..., x _k, Y={y ₁, y ₂..., y _k, k is positive integer; X, the element in Y is respectively x _j=accessReplica _i, y _j=(accessReplica _i, p), 1≤j≤k, 1≤i≤d, wherein, (accessReplica _i, p) mean copy accessReplica _iaccessed accessReplica once by node p _i∈ { accessReplica ₁, accessReplica ₂..., accessReplica _d, d is positive integer, p ∈ { u, closeNode ₀, closeNode ₁... closeNode _q, q is integer; The sample that X is the cuclear density function, the node of the each access of Y record and accessed copy;

The accessed possibility of the every Geju City of Density Estimator policy calculation copy formula as follows:

Wherein, H is parameter, H=h* ξ, wherein, and ξ=hop+1, hop≤λ, h is one and is greater than zero constant; ξ is for adjusting the H value; Hop is the jumping figure between user node and its adjacent node; Weights ω _jmeet

k () is positioned at x _j, j=1,2 ..., the kernel function of d;

Afterwards, to { oldReplica ₁, oldReplica ₂..., oldReplica _min each copy calculate f (oldReplica _i, i ∈ 1,2 ..., m}, computing formula is as follows:

Wherein, bandwidth is the network total bandwidth, and i is positive integer;

Finally, according to the every Geju City copy obtained, and the computing formula of the best copy in old copy, select best old copy:

$f\left(\mathrm{bestOldReplica}\right)=\underset{i}{\mathrm{Max}}\left(f\left(\mathrm{oldRepli}{\mathrm{ca}}_i\right)\right),i\∈\{\mathrm{1,2}...m\}---\left(4\right)$

Forward step 3 after completing to;

Step 3, judged whether latest copy; If there is no latest copy, forward step 4 to; If latest copy is arranged,, according to latest copy place node current bandwidth situation, select best latest copy, select the method for best latest copy:

Suppose to have l latest copy { newReplica ₁, newReplica ₂..., newReplica _l, wherein, l is positive integer, l=n-m; At { newReplica ₁, newReplica ₂..., newReplica _lmiddle g (the y that calculates _i), g (y _i) be node y _i(1≤i≤l) current available bandwidth:

$g\left(y_i\right)=\frac{\mathrm{bandwidth}(u,y_i)}{{\mathrm{cl}}_i},i\∈\{i,2,...l\}---\left(5\right)$

Wherein, cl _inode y _ithe current network load, y _i(1≤i≤l) is copy newReplica _ithe node at (1≤i≤l) place;

Best latest copy computing formula is as follows:

$g\left(\mathrm{bestNewNode}\right)=\underset{i}{\mathrm{Max}}\left(g\left(y_i\right)\right),i\∈\{\mathrm{1,2},...l\}---\left(6\right)$

The latest copy of selecting thus on node bestNewNode is best latest copy, then performs step four;

If step 4 does not have latest copy, the old copy of the best that step 2 is selected is best copy; If there is no old copy, the best latest copy that step 3 is selected is best copy; If existing latest copy, the copy of haveing been friends in the past again, the available_bandwidth of more best old copy (u, bestOldReplica _i) value of value and the g (bestNewNode) of best latest copy, if available_bandwidth is (u, bestOldReplica _i) value large, best old copy is best copy; If g (bestNewNode) value is large, best latest copy is best copy.

Images