CN103389946B - Go flaking method and system - Google Patents

Abstract

The present invention provides one to remove flaking method, first the method determines the expection read/write bandwidth of each data object accessing operation in access forecasting sequence, the described read/write bandwidth that prefetches is with data object length corresponding to this access operation and access step-length of beating as variable, the matching bandwidth obtained based on matching bandwidth function.Then from the beginning of from access forecasting sequence, certain accesses operation, the read/write bandwidth that prefetches of continuous multiple access operations and each data object accessing operation for exceeding predetermined number is less than bandwidth threshold, and these data objects accessing operation are sequentially written to new continuous print memory space.The method can accurately judge fragmentation data, in the case of little effect current system performance, improves reading performance, and the environment in change has preferable adaptability.

Description

Go flaking method and system

Technical field

The invention belongs to Computer Storage field, the method particularly relating to process fragmentation of data.

Background technology

Fragmentation refers to that continuous print file data the most relatively is stored after storage medium in logic, and its storage position becomes Obtain relative distribution, bring the operation of a large amount of random read-write when data access.Storage system based on disk for current main flow For system, fragmentation can cause the continuous decline of read/write bandwidth, accordingly, it would be desirable to one removes flaking method, reduces fragmentation Impact, reduces random read-write and accesses.

Existing go flaking method be typically input access forecasting sequence, according to this access forecasting sequence identification fragmentation Fragmentation data is also written to new memory area by data, thus reaches the purpose of fragmentation.Described access forecasting sequence It is a kind of prediction to the most most possible access order, there is multiple Forecasting Methodology, but inherently can be divided into two kinds: (1) static prediction, as the logic storage order according to data is predicted；(2) dynamic prediction, as according to current accessed sequential prediction.

During removing fragmentation, it is usually required mainly for solve two problems: (1) which type of data object is fragmentation number According to, if i.e. according to accessing forecasting sequence access current storage system, the access of which data object can cause read/write bandwidth to go out Existing more decline；(2) which data should be written to new storage position, thus is reducing what random read-write in the future accessed Meanwhile, the current performance of little effect system as far as possible.Existing subject matter present in flaking method is gone to show as follows Several aspects: first, the judgement to fragmentation data is not accurate enough, it is impossible to effectively improving performance；Secondly, Fragmentation is removed In, the data volume needing movement is too much, and during causing operation, declining significantly occurs in systematic function.

Summary of the invention

Therefore, it is an object of the invention to overcome the defect of above-mentioned prior art, it is provided that a kind of new remove flaking method.

It is an object of the invention to be achieved through the following technical solutions:

On the one hand, the invention provides one goes flaking method, described method to include:

Step 1) determine the expection read/write bandwidth of each data object accessing operation in access forecasting sequence, described pre- Taking read/write bandwidth is with data object length corresponding to this access operation and access step-length of beating as variable, based on matching bandwidth Function and the matching bandwidth that obtains；

Step 2) from accessing in forecasting sequence from the beginning of certain access operation, for exceeding continuous multiple access of predetermined number The read/write bandwidth that prefetches of operation and each data object accessing operation is less than bandwidth threshold, and these are accessed the data of operation Object is sequentially written to new continuous print memory space.

In the above-mentioned methods, described step 1) in, described matching bandwidth function can be to need the storage of fragmentation In system, with data object length and access step-length of beating as variable, beat step based in different pieces of information object length and access The long lower benchmark read/write bandwidth data measured, the bandwidth function expression obtained by approximating method.

In the above-mentioned methods, described matching bandwidth function can obtain through the following steps:

Step a), in the storage system needing fragmentation, beats step-length y for becoming with data object length x and access Amount, measures the benchmark read/write bandwidth data under one group of difference x and y；

Step b) is a selected master variable from x, y, and another is time variable；

Step c) find with master variable closest to master variable value；

Step d) finds the interval of time variable；

Step e) is based on the benchmark read/write bandwidth data matching x under measured different x and y, when y is any value Read/write bandwidth f (x, y).

In the above-mentioned methods, in described step c), when master variable is x, find x_iMake Δ x=h (x_i)-h (x) absolute Value minimum；When master variable is y, find y_iMake Δ y=h (y_i) absolute value of-h (y) is minimum；

In described step d), when secondary variable is y, find y_jMake y ∈ [y_j, y_j+1], when secondary variable is x, find x_jMake x ∈[x_j, x_j+1]；

In step e), when secondary variable is y, make $f(x,y)=f_{\mathrm{ij}+1}+\frac{h\left(y\right)-h\left(y_j\right)}{h\left(y_{j+1}\right)-{\mathrm{hy}}_j}|f_{\mathrm{ij}}-$ $f_{\mathrm{ij}+1}|+g\left(\mathrm{\Δx}\right),$ Wherein, (Δ x) represents the further correction to fitting result to g；When secondary variable is x, make $f(x,y)=f_{\mathrm{ij}}+\frac{h\left(x\right)-h\left(x_j\right)}{h\left(x_{j+1}\right)-h\left(x_j\right)}|f_{\mathrm{ij}}-f_{i+1j}|+g\left(\mathrm{\Δy}\right),$ Wherein, (Δ y) represents to enter fitting result g One step correction.

In the above-mentioned methods, may also include step 3) from accessing in forecasting sequence from the beginning of certain access operation, for exceeding Continuous multiple access operations of predetermined number and the read/write bandwidth that prefetches of each data object accessing operation are more than bandwidth threshold Value, then skip these and access operation, continues with ensuing access and operates.

In the above-mentioned methods, may also include described step 4), for accessing in forecasting sequence not by step 2) or step 3) Subsequence s that process, that be made up of continuous multiple access operations_i..., s_i+mIf the overall bandwidth of this subsequence is less than certain The hypothesis continuous bandwidth of individual threshold value and this subsequence is more than certain threshold value, then by this subsequence, each accesses the data pair operated As being written to new continuous print memory space, and the access entrance of these data objects is changed to new position；Wherein, described Assume that continuous bandwidth represents if the bandwidth after being deposited continuously by the data object accessing operation in this subsequence；Subsequence s_i..., s_i+mOverall bandwidthl_iFor s_iThe length of data object, f_iFor s_iData object pre- Phase read/write bandwidth, m is the number accessing operation in this subsequence.

In the above-mentioned methods, described bandwidth threshold can go as required fragmentation storage system maximum bandwidth value or Continuous bandwidth determines, described continuous bandwidth is to beat in the case of step-length is 0 in access, obtains based on matching bandwidth function Matching bandwidth.

Another aspect, the present invention also provides for one and goes fragmentation system, described system to include:

Bandwidth determines device, accesses the expection read/write of each data object accessing operation in forecasting sequence for determining Bandwidth, described in prefetch read/write bandwidth be with data object length corresponding to this access operation and access step-length of beating as variable, base The matching bandwidth obtained in matching bandwidth function；

Remove crumb units, for from accessing in forecasting sequence from the beginning of certain access operation, for exceeding the company of predetermined number Continue multiple read/write bandwidth that prefetches accessing operation and each data object accessing operation and be less than bandwidth threshold, these are accessed The data object of operation is sequentially written to new continuous print memory space.

In said system, described matching bandwidth function can be in the storage system needing fragmentation, with data Object length and access step-length of beating is variable, based in different pieces of information object length with access the benchmark measured under step-length of beating Read/write bandwidth data, the bandwidth function expression obtained by approximating method.

In said system, described in go crumb units to can be additionally used in from accessing from the beginning of certain accesses operation in forecasting sequence, The read/write bandwidth that prefetches of continuous multiple access operations and each data object accessing operation for exceeding predetermined number is more than Bandwidth threshold, then skip these and access operation, continues with ensuing access and operates.

In said system, described in go crumb units to can be additionally used in the continuous multiple visits for being unsatisfactory for exceeding predetermined number Ask operation data object prefetch read/write bandwidth both greater than or less than bandwidth threshold, operated by continuous multiple access and constitute Subsequence s_i..., s_i+mIf the overall bandwidth of this subsequence is less than certain threshold value and the hypothesis continuous bandwidth of this subsequence More than certain threshold value, then each data object accessing operation in this subsequence is written to new continuous print memory space, and The access entrance of these data objects is changed to new position；Wherein, described hypothesis continuous bandwidth represents if by this sub-sequence Row access the bandwidth after the data object of operation is deposited continuously；Subsequence s_i..., s_i+mOverall bandwidthl_iFor s_iThe length of data object, f_iFor s_iThe expection read/write bandwidth of data object, m is this sub-sequence Row access the number of operation.

Compared with prior art, provided by the present invention remove flaking method, can accurately judge fragmentation number According to, in the case of little effect current system performance, improving reading performance, the environment in change has preferable adaptability.

Accompanying drawing explanation

Embodiments of the present invention is further illustrated referring to the drawings, wherein:

Fig. 1 is the schematic flow sheet removing flaking method according to one embodiment of the invention；

Fig. 2 is the schematic flow sheet removing flaking method according to another embodiment of the present invention.

Detailed description of the invention

In order to make the purpose of the present invention, technical scheme and advantage are clearer, below in conjunction with accompanying drawing by concrete real The present invention is described in more detail to execute example.Should be appreciated that specific embodiment described herein only in order to explain the present invention, It is not intended to limit the present invention.

Access forecasting sequence and be actually a series of access operation, and read/write bandwidth is main and each access operates The size of the data object accessed and access change in location distance before or after accessing this data object (or can also It is referred to as access to beat step-length) relevant.Generally, the degree of fragmentation of the data object that read/write bandwidth is less in forecasting sequence is accessed Higher, it is contemplated that read/write bandwidth is a kind of posteriority index, it is impossible to obtain in advance.Therefore, in an embodiment of the present invention, Have employed read/write bandwidth approximating method, calculate the pre-of the data object that in access forecasting sequence, each access operation is accessed Phase read/write bandwidth (the most also referred to as accesses the matching bandwidth that operation is corresponding), and carries out fragmentation based on this index The judgement of degree.

Fig. 1 gives the schematic flow sheet removing flaking method according to an embodiment of the invention.The method includes: step Rapid 1) the expection read/write bandwidth of the data object every time accessing operation in access forecasting sequence is determined；Step 2) from accessing prediction In sequence certain access operation start, if it exceeds predetermined number continuous multiple access operation data objects prefetch reading/ Write bandwidth and be less than bandwidth threshold, these data objects accessing operation are sequentially written to new continuous print memory space.

With reference to Fig. 1, more specifically, in step 1) determine the pre-of the data object that in access forecasting sequence, access operates every time Phase read/write bandwidth.The described read/write bandwidth that prefetches is the data object length corresponding with this access operation and access step-length of beating is Variable, the matching bandwidth obtained based on matching bandwidth function.According to one embodiment of present invention, described step 1) can include The following step:

Step 11) receive the access operation accessed in forecasting sequence one by one, and it is saved in queue S: < s₁..., s_nIn >.? Including but not limited to being obtained by certain mode (such as metadata access, index search etc.) in the information preserved in queue S Each original position o accessing the accessed data object of operation_iAnd length l of data object_i.The access preserved in queue S is pre- Order-checking row can be the forecasting sequence (static prediction or dynamic prediction) directly obtained.It may also be pre-through pretreatment Order-checking row (such as removing the forecasting sequence of repetition)；Such as, if the kth received accesses operation s_kWith s in queue S₁..., s_k-1One of them access same data object, the most not by this access operate s_kAdd in enqueue S.

Step 12) calculate each access operation s in queue S_iCorresponding matching bandwidth f_i。

As mentioned above, described matching bandwidth is actually with data object length and the access of this access operation correspondence Step-length of beating is variable, the bandwidth obtained based on matching bandwidth function.Described matching bandwidth function is to remove fragmentation at needs Storage system in, with data object length and access step-length of beating as variable, based on one group in different pieces of information object length and Access the benchmark read/write bandwidth data of the measurement beated under step-length, the bandwidth function expression obtained by approximating method.At this In one embodiment of invention, obtain the expression formula of matching bandwidth function through the following steps:

Step a), in the storage system needing fragmentation, beats step-length y for becoming with data object length x and access Amount, measures the benchmark read/write bandwidth data under one group of difference x and y, as shown in table 1.Wherein x_i, y_jThe difference representing x and y takes Value, measuring corresponding read/write bandwidth through reality is f_ij, described different x and y refer to the incremental manner of variable, include but not limited to Difference is incremented by, and geometric ratio is incremented by.

Table 1

Parameter	y0	y1	...	yn
					x0	f00	f01	...	fon
x1	f10	f11	...	f1n
					...	...	...	...	...
xm	fm0	fm2	...	fmn

Step b) is a selected master variable from x, y, and another is time variable.

Step c) find with master variable closest to master variable value (such as, x₀～x_mIn a value), i.e. work as master When variable is x, find x_iMake Δ x=h (x_i) absolute value of-h (x) is minimum, when master variable is y, finds y_iMake Δ y=h (y_i) absolute value of-h (y) is minimum, wherein function h (x) is including but not limited to linear function (such as h (x)=x), logarithmic function (as H (x)=log₂X) etc..

Step d) finds the interval of time variable, i.e. when secondary variable is y, finds y_jMake y ∈ [y_j, y_j+1], when secondary change When amount is for x, find x_jMake x ∈ [x_j, x_j+1]。

Step e) based on read/write bandwidth f that benchmark data matching x, y are during any value (x, y): in secondary variable-value district Between take a rational value in corresponding bandwidth interval in proportion as fitting result, i.e. when secondary variable is y, make $f(x,y)=f_{\mathrm{ij}+1}+\frac{h\left(y\right)-h\left(y_j\right)}{h\left(y_{j+1}\right)-{\mathrm{hy}}_j)}|f_{\mathrm{ij}}-f_{\mathrm{ij}+1}|+g\left(\mathrm{\&Delta;x}\right),$ Wherein, (Δ x) represents fitting result g Revise further.When secondary variable is x, make $f(x,y)=f_{\mathrm{ij}}+\frac{h\left(x\right)-h\left(x_j\right)}{h\left(x_{j+1}\right)-h\left(x_j\right)}|f_{\mathrm{ij}}-f_{i+1j}|+g\left(\mathrm{\&Delta;y}\right),$ Its In, (Δ y) represents the further correction to fitting result to g.

Obtained the expression formula of matching bandwidth function by above-mentioned steps a)-step e), wherein comprise two variable data Object length x and access are beated step-length y.Described data object length can be the physical length in units of byte, it is also possible to Being the logic length of self-defined unit, described access step-length of beating refers to access position before or after this data object accessing Change distance, this distance can be the physical distance in units of byte, it is also possible to be the logical reach of self-defined unit.

Such as, measured benchmark data can be to read band data, can use for data object length x and access step-length of beating From small to large by 2 integer power as value mode, i.e. take x_i=2ⁱ, y_j=2^j, according to x_i, y_jThe most from first to last read one Big file, using meansigma methods as f_ij.For each x_iAlso need to benchmark during measuring sequence reading (i.e. accessing step-length y=0 of beating) Band data, the band data (continuous bandwidth can be referred to as) when the most there is not fragment.Here may select master variable is Accessing size of data x, secondary variable is step-length y of beating.Then value (the x closest to master variable is found₀～x_mIn a value) and Secondary variable-value is interval.Make h (x)=log₂X, Δ x=i-log₂X, then i=[log₂X+0.5] time Δ x minimum.Order, Secondary variable-value interval is [2^j, 2^j+1], as y=0, interval is [0,1].Make g (Δ x)=Δ x × (f_ij-f_ij+1), the most as above Literary composition is described, can obtain the wide fitting function that reads tape $f(x,y)=f_{\mathrm{ij}}+1+\frac{h\left(y\right)-h\left(y_j\right)}{h\left(y_{j+1}\right)-{\mathrm{hy}}_j}|f_{\mathrm{ij}}-f_{\mathrm{ij}+1}|+$ $g\left(\mathrm{\&Delta;x}\right).$

Then, according to obtained matching bandwidth function, calculate each access in queue S and operate s_iCorresponding matching bandwidth f_i=f (l_i, | o_i-o_i-1-l_i-1|).Visible, this matching bandwidth is actually according to length (such as, the l of data object_i) and visit Ask before or after this object access change in location distance (such as, | o_i-o_i-1-l_i-1|), obtain matching bandwidth by above-mentioned The calculated numerical value of function.Receive when such as accessing forecasting sequence, arrive delay upper bound (for avoiding fragmentation The threshold value introducing too much delay and arrange), or start to go to step 2 when receiving queue S is full etc.) continue executing with.

With continued reference to Fig. 1, in step 2) from accessing in forecasting sequence from the beginning of certain access operation, if it exceeds predetermined number The read/write bandwidth that prefetch of data objects of continuous multiple access operations be both less than bandwidth threshold, these are accessed the number of operation New continuous print memory space sequentially it is written to according to object.So, the method by matching bandwidth selection calculated above to that A little data objects carry out fragmentation, are negatively affected presently written bandwidth to reduce as far as possible.If it exceeds predetermined number The read/write bandwidth that prefetches of continuous multiple data object accessing operation is both greater than bandwidth threshold, then skip these and access operation, continue Continuous process is ensuing accesses operation.

Wherein, described bandwidth threshold can be static threshold, can go the maximum belt of the storage system of fragmentation as required Width values determines.Bandwidth threshold such as can be set to the percentage ratio of storage system maximum bandwidth, such as storage system maximum bandwidth More than 60%；When described bandwidth threshold can also be dynamic threshold, can determine according to continuous bandwidth, this continuous bandwidth is from plan Closing function and calculate gained, i.e. accessing step-length of beating is matching bandwidth f (x, 0) when 0.Such as can be set to bandwidth threshold to set It is set to the percentage ratio of continuous bandwidth, such as more than the 70% of continuous bandwidth.

Avoid the write operation impact on systematic entirety energy, again to more effectively reduce system fragmentation program simultaneously In one embodiment, in step 2) according to three conditions, access forecasting sequence is divided into the three continuous subsequences of class, and to all kinds of sons Sequence carries out different disposal.As described above, queue S: < s₁..., s_n> preserve is received access forecasting sequence, from Head scans receiving queue S to tail, according to following three conditions this access forecasting sequence is divided into multiple subsequence:

Condition 1:s_i..., s_i+mMeet f_i+1≥T_fi+1..., f_i+m≥T_fi+m, m >=T_m, f_i+m+1＜ T_fi+m+1Or i+m=n； Here show that certain accesses operation (such as s from accessing forecasting sequence_i) start, exceed a certain number of (i.e. T_m) the most multiple The matching bandwidth accessing operation is both greater than bandwidth threshold, say, that these access the degree of fragmentation of the data object operated relatively Low, this kind of subsequence can be ignored.Now s_i..., s_i+mCorresponding data object is without being written to new continuous print storage position Put, to s_i..., s_i+mIt is not required to any process, continues to identify next subsequence.Wherein, T_fiRepresent for s_iMatching bandwidth Bandwidth threshold, T_mFor an integer thresholds set in advance.As described above, bandwidth threshold T_fiCan be that described bandwidth threshold can To be static threshold, as may be configured as needing more than the 60% of the storage system maximum bandwidth of fragment；It can also be dynamic threshold During value, as may be configured as more than the 70% of continuous bandwidth, it is plan when 0 that this continuous bandwidth refers to access step-length of beating Crossed belt width f (x, 0).

Condition 2:s_i..., s_i+mMeet f_i＜ T_fi..., f_i+m＜ T_fi+m, m >=T '_m, f_i+m+1≥T_fi+m+1Or i+m=n；This In show from from accessing forecasting sequence, certain accesses operation (such as s_i) start, exceed a certain number of (such as T '_m) continuous many The individual matching bandwidth accessing operation is both less than bandwidth threshold, say, that these access the degree of fragmentation of the data object operated Higher, need to go fragmentation to process this kind of subsequence.Now s_i..., s_i+mCorresponding data object needs to be written to New continuous print storage.Wherein, T_fiRepresent for s_iThe bandwidth threshold of matching bandwidth, T '_mFor an integer set in advance Threshold value.

For meeting the subsequence s of condition 2_i..., s_i+m, can be by s_i..., s_i+mCorresponding data object is sequentially written to New continuous print memory space, and the access entrance of these data objects is changed to new position.Wherein, write data pair As reading from locally stored position, it is also possible to obtained from other equipment by network.

Condition 3, except remaining situation of condition 1 and condition 2.Such as, s_i..., s_i+mIt is unsatisfactory for condition 1 or condition 2, simultaneously I+m=n, or s_i+m+1..., s_i+m+kMeet condition 1 or condition 2.

The subsequence of condition, such as s is divided for being unsatisfactory for (1) and (2)_i..., s_i+m, calculate the overall band of this sequence Width, namely performs all overall bandwidth accessing operation generation rather than the bandwidth of certain access operation in this sequence.If More than certain threshold value, (such as threshold value a) then illustrates the data object degree of fragmentation that this sequence accesses to the overall bandwidth of this sequence The highest, it is not necessary to process this sequence.Whereas if overall bandwidth is less than threshold value a, then calculate the hypothesis continuous bandwidth of this sequence, should Assume that continuous bandwidth refers to if the bandwidth after being deposited continuously by the data object accessing operation in this sequence.If overall bandwidth is little In threshold value a and assume that (such as threshold value b) illustrates if the data pair will answered these sequence pair continuous bandwidth more than certain threshold value To be effectively improved bandwidth as being written to Coutinuous store region, the data object therefore can answered these sequence pair is carried out accordingly Process, i.e. this subsequence s_i..., s_i+mCorresponding data object is written to new continuous print memory space.If overall bandwidth It is less than threshold value b, even if illustrating to be written to the data object that these sequence pair are answered continuously less than threshold value a and hypothesis continuous bandwidth Memory area can not improve bandwidth, does not processes this sequence.Wherein, the setting of threshold value a and threshold value b can refer to Bandwidth threshold described in literary composition is arranged.

Wherein overall bandwidth and hypothesis continuous bandwidth calculate as follows:

s_i..., s_i+mOverall bandwidth numerical value $f=\frac{l_i+\&CenterDot;\&CenterDot;\&CenterDot;+l_{i+m}}{\frac{l_i}{f_i}+\&CenterDot;\&CenterDot;\&CenterDot;+\frac{l_{i+m}}{f_{i+m}}};$

Acquisition can write s_i..., s_i+mCoutinuous store space original position o of corresponding data object also calculates hypothesis even Continuous bandwidth f '=f (l_i+…+l_i+m, | o-o '_i-1-l_i-1|), if f is ＜ T_fAnd f ' ＞ T '_f, by s_i..., s_i+mCorresponding data Object is sequentially written to new continuous print memory space, and the access entrance of these data objects is changed to new position.Its In, o '_i-1And l_i-1For forecasting sequence s_i-1The up-to-date original position (new position may be had been written into) of corresponding data object And size, T_fWith T '_fSetting can refer to bandwidth threshold mentioned above and arrange, write data object can be from locally stored Position is read, it is also possible to obtained from other equipment by network.

When queue S is disposed, after emptying queue S, continue to another and access forecasting sequence.

Above-mentioned fragment method is gone in order to be more fully understood that, more detailed to going Fragmentation to carry out below in conjunction with table 2 and Fig. 2 Thin illustration.Table 2 gives the concrete access forecasting sequence that Fragmentation is to be received, wherein comprises this Access in forecasting sequence and access sequence number i of operation and the long l of data object of correspondence_iDegree and original position o of this data object_i。

Table 2

Sequence number	1	2	3	4	5	6	7	8	9	10	11	12	13	14
															li	3	2	5	6	4	3	3	5	4	3	2	3	3	3
oi	10	20	30	50	3	10	7	35	40	13	16	20	7	23

Wherein, a length of the 12 of receiving queue S are set, during original state, it is assumed that it is 0 that last stored accesses position, specifically Execution process is:

1. perform step 201, receive the access operation 1～14 accessed in forecasting sequence successively, often receive one and access behaviour Go to step 202.

2. perform step 202, it is determined whether be the access operation repeated；If it is step 201 is gone to；If it is not, Then go to step 203.

Specifically, if currently received access operation with queue in received access operation accessed data object phase With, then ignore currently received access and operate, go to step 201, to avoid the access band of the same data object of double counting Width, such as, when receiving access operation 6,13, accesses operation 6,13 and 1,7 accessed data objects identical (original position and data Length is identical).

3. perform step 203, calculate the matching bandwidth of this access operation.

Specifically, the currently received data pair accessing operation the accessed data object tail entry relative to queue S are obtained The step-length of beating of elephant, by data object size and the step-length input such as matching bandwidth function obtained above of beating of acquisition, and will Data object size, the information such as data object original position and calculated matching bandwidth recorded queue tail, such as, right In accessing in operation 1, data object size is 3, and step-length of beating is 10 (to be jumped to data object initiate from last visit position 0 Position 10), a width of f of matching band₁=f (3,10), accesses in operation 2, and data object size is 2, and step-length of beating is 7 (from access The data object end position 13 of operation 1 jumps to data object original position 20), matching bandwidth f₂=f (2,7).

4. perform step 204, it is judged that process whether queue condition meets.Such as completely or arrive delay upper bound when queue Time (avoiding fragmentation to introduce too much delay), from first to last process queue, otherwise continue to access forecasting sequence.

Step 201～204 performs repeatedly, until after access forecasting sequence receives, or queue the most completely reaches to process bar Part, queue S state (accessing 6,13 is repeated accesses, ignores reception) as shown in table 3:

Table 3

Entries in queues	s1	s2	s3	s4	s5	s6
							li	3	2	5	6	4	3
oi	10	20	30	50	3	7
							fi	F (3,10)	F (2,7)	F (5,8)	F (6,15)	F (4,53)	F (3,0)
Entries in queues	s7	s8	s9	s10	s11	s12
							li	5	4	3	2	3	3
oi	35	40	12	15	20	23
							fi	F (5,25)	F (4,0)	F (3,32)	F (2,0)	F (3,3)	F (3,0)

5. performing step 205, scan queue S also the most according to condition divides subsequence.

Specifically, from first to last scan queue item s₁～s₁₂, according to three conditions division subsequences: condition 1, s_i..., s_i+mMeet f_i+1≥T_fi+1..., f_i+m≥T_fi+m, m >=T_m, f_i+m+1＜ T_fi+m+1Or m=12；Condition 2, s_i+1..., s_i+mMeet f_i ＜ T_fi..., f_i+m＜ T_fi+m, m >=T ' m, f_i+m+1≥T_fi+m+1Or m=12；Condition 3, except remaining situation of condition 1 and condition 2 (it is unsatisfactory for condition 1 or condition 2, m=12 or s simultaneously_m+1..., s_m+kMeet condition 1 or condition 2), wherein, T_fi=0.75 × f(l_i, 0), T_m, T '_mRound numbers 4.After scanned, comparative result is as shown in table 4, wherein, and s₁～s₅Matching bandwidth is less, meets Condition 2, s₆～s₈Matching amount of bandwidth differs, and meets condition 3, s₉～s₁₂Matching bandwidth is relatively big (except s₉), meet condition 1, simultaneously Subsequence division result is made corresponding labelling in queue.

Table 4

6. perform step 206, it is judged that subsequence meets condition；If meeting bar 1, then go to step 209；If meeting bar 2, then device step 208；If meeting bar 3, then device step 207.

Specifically, according to the labelling result of step 205, s₁～s₅Meet condition 2, go to step 208.

7. perform step 208, by s₁～s₅Data object be written to new memory space

Specifically, find and can write s₁～s₅The original position in Coutinuous store space of data object and write, it is assumed that Find original position 100 in this example, by s₁～s₅Data object be respectively written into 100～102,103～104,105～ 109,110～115, in the memory space of 116～119, and the access entrance of these data objects is changed to new storage position Put.

8. perform step 209, by s₁～s₅(queue head pointer is pointed to s6) is removed from queue

9. perform step 210 (now queue not empty, go to step 206)

10. perform step 206, it is judged that subsequence meets condition.

Specifically, according to the labelling result of step 205, s₆～s₈Meet condition 3, go to step 207.

11. perform step 207, it is judged that subsequence s₆～s₈Whether meet Writing condition.

Specifically, acquisition can write s₆～s₈Memory space original position o of data object, calculate overall bandwidthCalculate and assume write bandwidth f '=f (l₆+l₇+l₈, | o-o '₅-l₅|), it is computed, o=120 (s₅Ending Position), o '₅=116 (the newly written positions of data object of s5), l₅=4, then f '=f (12,0), threshold value T_f=T '_f=0.75 × f(l₆+l₇+l₈, 0) and=0.75 × f (12,0) (s_i～s_i+mCorresponding threshold value T_f=T '_f=0.75 × f (l_i+…+l_i+m, 0)).This Time, meet f ＜ T_fAnd f ' ＞ T '_f, turn 208.

12. perform step 208, by s₆～s₈Data object be written to new Coutinuous store space

Specifically, by s₆～s₈Data object be sequentially respectively written into 120～122,123～127,128～131 deposit In storage space, and the access entrance of these data objects is changed to new storage position.

13. perform step 209, by subsequence s₆～s₈Remove from queue and (queue head pointer is pointed to s₉)

14. perform step 210 (now queue not empty turns 206)

15. perform step 206, it is judged that subsequence meets condition

Specifically, according to the labelling result of step 205, s₉～s₁₂Meet condition 1, turn 209.

16. perform step 209, by s₉～s₁₂(queue head pointer is pointed to s1) is removed from queue

17. perform step 210 (now queue is empty, turns 201)

Inventor also, in content finds storage system, utilizes the backup load under true environment to carry out above-mentioned method Test.Test result shows, along with the growth of data volume, reads bandwidth and improves 12%～60%, and data redundancy controls 1%～2%, essentially eliminate the reading continuous downward trend of bandwidth；In same system, utilize the data syn-chronization of two weeks by a definite date The test result of load shows, reads bandwidth and improves 5～8 times, it is to avoid the reading bandwidth that fragmentation of data causes serious Decline.

Although the present invention has been described by means of preferred embodiments, but the present invention is not limited to described here Embodiment, the most also include done various changes and change.

Claims (7)

1. go a flaking method, described method to include:

Step 1) determine and access the expection read/write bandwidth of each data object accessing operation in forecasting sequence, described expection reads/ Writing bandwidth is with data object length corresponding to this access operation and access step-length of beating as variable, based on matching bandwidth function The matching bandwidth obtained, described matching bandwidth function is in the storage system needing fragmentation, with data object length and Access step-length of beating is variable, based in different pieces of information object length with access the benchmark read/write bandwidth measured under step-length of beating Data, the bandwidth function expression obtained by approximating method；

Step 2) from accessing in forecasting sequence from the beginning of certain access operation, for exceeding continuous multiple access operations of predetermined number And the expection read/write bandwidth of each data object accessing operation is less than bandwidth threshold, and these are accessed the data object of operation Sequentially it is written to new continuous print memory space；

Wherein, described matching bandwidth function obtains through the following steps:

Step a) is in the storage system needing fragmentation, with data object length x and access step-length y of beating as variable, surveys Measure the benchmark read/write bandwidth data under one group of difference x and y；

Step b) is a selected master variable from x, y, and another is time variable；

Step c) find with master variable closest to master variable value；

Step d) finds the interval of time variable；

Step e) is based on the benchmark read/write bandwidth data matching x under measured different x and y, and y is read/write during any value Bandwidth f (x, y)；

Wherein, in described step c), when master variable is x, find x_iMake Δ x=h (x_i) absolute value of-h (x) is minimum；Work as main transformer When amount is for y, find y_iMake Δ y=h (y_i) absolute value of-h (y) is minimum；

In described step d), when secondary variable is y, find y_jMake y ∈ [y_j,y_j+1], when secondary variable is x, find x_jMake x ∈ [x_j,x_j+1]；

In step e), when secondary variable is y, make Wherein, g Δ x represents the further correction to fitting result；When secondary variable is x, makeWherein, (Δ y) represents to enter fitting result g One step correction.

Method the most according to claim 1, also includes step 3) from accessing in forecasting sequence from the beginning of certain access operation, The expection read/write bandwidth of continuous multiple access operations and each data object accessing operation for exceeding predetermined number is more than Bandwidth threshold, then skip these and access operation, continues with ensuing access and operates.

Method the most according to claim 2, also includes step 4), for accessing in forecasting sequence not by step 2) or step 3) subsequence s that process, that be made up of continuous multiple access operations_i,…,s_i+mIf the overall bandwidth of this subsequence is less than certain The hypothesis continuous bandwidth of individual threshold value and this subsequence is more than certain threshold value, then by this subsequence, each accesses the data pair operated As being written to new continuous print memory space, and the access entrance of these data objects is changed to new position；Wherein, described Assume that continuous bandwidth represents if the bandwidth after being deposited continuously by the data object accessing operation in this subsequence；Subsequence s_i,…,s_i+mOverall bandwidthl_iFor s_iThe length of data object, f_iFor s_iThe expection of data object Read/write bandwidth, m is the number accessing operation in this subsequence.

Method the most according to claim 2, described bandwidth threshold goes the maximum belt of the storage system of fragmentation as required Width values or continuous bandwidth determine, described continuous bandwidth is to beat in the case of step-length is 0 in access, based on matching bandwidth letter The matching bandwidth counted and obtain.

5. go a fragmentation system, described system to include:

Bandwidth determines device, accesses the expection read/write bandwidth of each data object accessing operation in forecasting sequence for determining, Described expection read/write bandwidth is with data object length corresponding to this access operation and access step-length of beating as variable, based on plan Crossed belt width function and the matching bandwidth that obtains, described matching bandwidth function is in the storage system needing fragmentation, with number It is variable according to object length and access step-length of beating, based in different pieces of information object length with access the base measured under step-length of beating Quasi-read/write bandwidth data, the bandwidth function expression obtained by approximating method；

Remove crumb units, for from accessing in forecasting sequence from the beginning of certain access operation, for exceeding the continuous many of predetermined number These, less than bandwidth threshold, are accessed operation by the individual expection read/write bandwidth of operation and each data object accessing operation that accesses Data object be sequentially written to new continuous print memory space；

Wherein, described matching bandwidth function obtains through the following steps:

Step a) is in the storage system needing fragmentation, with data object length x and access step-length y of beating as variable, surveys Measure the benchmark read/write bandwidth data under one group of difference x and y；

Step b) is a selected master variable from x, y, and another is time variable；

Step c) find with master variable closest to master variable value；

Step d) finds the interval of time variable；

Step e) is based on the benchmark read/write bandwidth data matching x under measured different x and y, and y is read/write during any value Bandwidth f (x, y)；

Wherein, in described step c), when master variable is x, find x_iMake Δ x=h (x_i) absolute value of-h (x) is minimum；Work as main transformer When amount is for y, find y_iMake Δ y=h (y_i) absolute value of-h (y) is minimum；

In described step d), when secondary variable is y, find y_jMake y ∈ [y_j,y_j+1], when secondary variable is x, find x_jMake x ∈ [x_j,x_j+1]；

In step e), when secondary variable is y, make Wherein, g Δ x represents the further correction to fitting result；When secondary variable is x, makeWherein, (Δ y) represents to enter fitting result g One step correction.

System the most according to claim 5, described in go crumb units to be additionally operable to from accessing forecasting sequence certain accesses behaviour Work starts, for exceeding continuous multiple access operations and the expection read/write of each data object accessing operation of predetermined number Band is wider than bandwidth threshold, then skip these and access operation, continues with ensuing access and operates.

System the most according to claim 6, described in go crumb units to be additionally operable to the company for being unsatisfactory for exceeding predetermined number The expection read/write bandwidth of continuous multiple data object accessing operation both greater than or less than bandwidth threshold, by continuous multiple access The subsequence s that operation is constituted_i,…,s_i+mIf the overall bandwidth of this subsequence is less than certain threshold value and the hypothesis of this subsequence Continuous bandwidth is more than certain threshold value, then each data object accessing operation in this subsequence is written to new continuous print storage Space, and the access entrance of these data objects is changed to new position；Wherein, described hypothesis continuous bandwidth represents if will This subsequence accesses the bandwidth after the data object of operation is deposited continuously；Subsequence s_i,…,s_i+mOverall bandwidthl_iFor s_iThe length of data object, f_iFor s_iThe expection read/write bandwidth of data object, m is this sub-sequence Row access the number of operation.