CN102156730B - File storage dynamic aggregation based optimization method - Google Patents

File storage dynamic aggregation based optimization method Download PDF

Info

Publication number
CN102156730B
CN102156730B CN 201110086026 CN201110086026A CN102156730B CN 102156730 B CN102156730 B CN 102156730B CN 201110086026 CN201110086026 CN 201110086026 CN 201110086026 A CN201110086026 A CN 201110086026A CN 102156730 B CN102156730 B CN 102156730B
Authority
CN
China
Prior art keywords
file
copy
website
node
data
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Fee Related
Application number
CN 201110086026
Other languages
Chinese (zh)
Other versions
CN102156730A (en
Inventor
王纪军
唐巍
洪昕
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
State Grid Corp of China SGCC
State Grid Jiangsu Electric Power Co Ltd
Information and Telecommunication Branch of State Grid Jiangsu Electric Power Co Ltd
Original Assignee
State Grid Jiangsu Electric Power Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by State Grid Jiangsu Electric Power Co Ltd filed Critical State Grid Jiangsu Electric Power Co Ltd
Priority to CN 201110086026 priority Critical patent/CN102156730B/en
Publication of CN102156730A publication Critical patent/CN102156730A/en
Application granted granted Critical
Publication of CN102156730B publication Critical patent/CN102156730B/en
Expired - Fee Related legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Landscapes

  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The invention discloses a file storage dynamic aggregation based optimization method. In the optimization method, a copy of a file which is frequently accessed by a node is acquired and is packed and stored at a local node by dynamically combining the file, so that frequency of remotely reading the file by the node is reduced and user data access delay is reduced. The method comprises the following steps of: 1, determining whether data is overlapped on the basis of a data original, detecting whether a request file sent by a user is in a local node, if the data is overlapped, judging that the request file is in the local node, if the data is not overlapped, turning into the step 2; 2, detecting whether sufficient storage space is locally available to accommodate a request file based on a copy placement optimization algorithm, if yes, selecting a remote node with minimum performance cost to obtain a request file copy, is not, turning into the step 3, wherein the cost comprises file transmission time and queuing time on the remote node; and 3, based on a storage space optimization algorithm, calculating times t1 of reading the request file by the node, calculating times t2 of reading less files from all files stored in the local node, if t1 is more than t2, deleting the t2 and clearing space for storing the request file. The cost of the copy selected to be replicated by the method is the minimum at the present and the cost is also the minimum in later time.

Description

Optimization method based on file storage dynamic aggregation
Technical field
The invention belongs to field of distributed storage, specifically a kind of optimization method based on file storage dynamic aggregation.
Background technology
Distributed storage has been set up framework and the isomery distributional environment of integrated storage, processing, access, transmission and the service of mass data, data copy the efficient that not only can improve data access, and can improve load equilibrium and the reliability of system.
Traditional method is that a data owner is originally copied copy to different websites, and this situation causa essendi is in the past or present some copies demand and treatment scheme is fairly simple.Development along with Data Replication Technology in Mobile, many variations have occured in situation, on same data source object, can mark off different copy source copies, these different copy source copies independently have a plurality of copies of oneself, this situation is called " cutting apart copy replication " (Multi-Intersected Copy Replication is called for short MICR) more.Although occur for the research of cutting apart copy replication more, still have subproblem not solve, mainly be unmodified main notebook data do not propagate with parallel link cut apart the copy replication problem more.
In addition, based on the replication strategy of Economic Model, be to determine the copy creating position and carry out the copy selection that according to the reverse auction agreement it is with the price guideline of data transmission period as auction.There is such problem in this model when the assessment data reproduction value: node is often made a strategic decision according to number one, thereby not necessarily obtains overall optimum efficiency.
Summary of the invention
For the overall performance that makes distributed storage reaches optimum, the purpose of this invention is to provide a kind of optimization method based on file storage dynamic aggregation, the copy that this optimization method is selected to be replicated is not only at the current Least-cost that expends, expend cost in time afterwards also for minimum, obtain overall optimum efficiency, reduce the number of times of the long-range file reading of node, reduce the user data access delay.
The objective of the invention is to be achieved through the following technical solutions:
A kind of optimization method based on file storage dynamic aggregation, it is characterized in that this optimization method passes through the file dynamic combined, often copy and the packing of the file of access are stored in local node to obtain node, reduce the number of times of the long-range file reading of node, reduce the user data access delay, concrete steps are as follows:
1) the based on data master determines whether that originally data are overlapping, and whether the demand file that the inspection user sends has data overlapping at local node, and then there is local node in this demand file; Countless when overlapping, then enter step 2);
The optimization algorithm of 2) placing based on copy, check whether this locality holds demand file with enough storage spaces, storage space is arranged, then select one to expend the minimum remote node of cost and obtain this demand file copy, cost include file transmission time and the queuing time on remote node; Without storage space, then turn step 3).
3) based on the optimized algorithm of storage space, the number of times of the long-range read requests file of computing node t 1Read the number of times of minimum file in the All Files of calculating book ground node storage t 2If, t 1 t 2, then delete the latter and vacating space storage resource request file.
Among the present invention, the based on data master determines whether that originally the overlapping algorithm of data is as follows:
Total n the node in the system of setting up departments , each node contains m file, and the user sends file request to a certain node when access, suppose that request comprises a plurality of files, and this request can be expressed as a file set
Figure 698042DEST_PATH_IMAGE002
Due to node
Figure 127886DEST_PATH_IMAGE003
There is data owner originally
Figure 290883DEST_PATH_IMAGE004
, so by
Figure 769269DEST_PATH_IMAGE004
The set that generates
Figure 412740DEST_PATH_IMAGE005
Be the object that need to copy
Figure 934857DEST_PATH_IMAGE006
If the data owner of demand file originally is
Figure 663779DEST_PATH_IMAGE007
, when the user accesses, to arbitrarily
Figure 680276DEST_PATH_IMAGE004
With
Figure 178254DEST_PATH_IMAGE007
, two kinds of situations can be arranged:
Figure 868342DEST_PATH_IMAGE008
Perhaps
Figure 84560DEST_PATH_IMAGE009
, namely
Figure 904748DEST_PATH_IMAGE004
With
Figure 257232DEST_PATH_IMAGE007
May there be overlapping data also may not have overlapping data.
When
Figure 386731DEST_PATH_IMAGE009
, when namely having data overlapping, there is local node in this demand file.When
Figure 824666DEST_PATH_IMAGE008
, namely countless when overlapping, enter content 2.Here with the set of crossover data content Expression:
Figure 655536DEST_PATH_IMAGE011
, can release thus:
Figure 690357DEST_PATH_IMAGE012
, the purpose of describing like this is to make to work as
Figure 615587DEST_PATH_IMAGE004
With
Figure 777578DEST_PATH_IMAGE007
Not carrying out overlapping Data duplication when same website all exists propagates.
The optimization algorithm of placing based on copy is as follows:
For certain file , suppose that its copy trace utilization cost is
Figure 291605DEST_PATH_IMAGE014
, wherein, the cost that creates a Copy is , the expense of access copy is
Figure 926166DEST_PATH_IMAGE016
, normal number
Figure 688586DEST_PATH_IMAGE017
With
Figure 60049DEST_PATH_IMAGE018
Expression creates and access copy shared weight in total cost, then Replica Placement Strategy is exactly under the constraint condition that satisfies regulation, so that the cost summation of copy creating and access
Figure 352807DEST_PATH_IMAGE020
Minimum.
Suppose the website that creates a Copy Storage space be
Figure 141957DEST_PATH_IMAGE021
, then the average storage space of all websites is , and
Figure 92912DEST_PATH_IMAGE023
Average storage space and website have been reflected
Figure 384085DEST_PATH_IMAGE013
The ratio of storage space.
Suppose:
Figure 540260DEST_PATH_IMAGE004
Be illustrated in certain period website
Figure 274998DEST_PATH_IMAGE013
The request number of times of certain copy, website
Figure 883834DEST_PATH_IMAGE024
Access copy website
Figure 244277DEST_PATH_IMAGE013
An expense be
Figure 254958DEST_PATH_IMAGE025
(available website
Figure 160597DEST_PATH_IMAGE013
With
Figure 256729DEST_PATH_IMAGE024
Between shortest path time representation), so
Figure 886775DEST_PATH_IMAGE026
Can find out that the file request number of times is more, function
Figure 17542DEST_PATH_IMAGE016
Be worth larger; If
Figure 94083DEST_PATH_IMAGE027
, namely website does not send the request of access to wave file, and it does not just produce the expense of access copy so.Have according to above-mentioned analysis
Figure 677511DEST_PATH_IMAGE028
(1)
Therefore, the copy Placement Problems can be converted into following optimization problem:
Objective function:
Figure 114177DEST_PATH_IMAGE029
(2)
Constraint condition:
Figure 99451DEST_PATH_IMAGE030
(3)
Figure 346892DEST_PATH_IMAGE031
Figure 417617DEST_PATH_IMAGE032
(4)
Figure 657974DEST_PATH_IMAGE033
Figure 497754DEST_PATH_IMAGE034
(5)
Wherein, objective function (2) respectively asks website to deposit the cost weighted sum of website minimum with its nearest copy so that the copy creating cost of each website adds,
Figure 916097DEST_PATH_IMAGE017
With Size be empirical value, arrange according to copy storage space and the copy request number of times influence degree to the response time; The copy number that constraint condition (3) expression can create is
Figure 252586DEST_PATH_IMAGE035
Individual; Constraint condition (4) represents that each request website corresponding only copy website is its service; Constraint condition (5) is if the expression copy is positioned at website
Figure 946873DEST_PATH_IMAGE024
, website so
Figure 536117DEST_PATH_IMAGE013
The copy request can only be assigned to website
Figure 581434DEST_PATH_IMAGE024
Calculating exists
Figure 166523DEST_PATH_IMAGE036
Transmission cost in time cycle is to determine that whether reapposing copy exchanges optimum average response time for.Therefore, new objective function is changed to by formula (2)
Figure 715316DEST_PATH_IMAGE037
(6)
Compare with formula (2), increased by one in the new target function type (8) and adjusted cost
Figure 741041DEST_PATH_IMAGE038
Cumulative, it is right to realize
Figure 8074DEST_PATH_IMAGE036
Iterative computation in time cycle, each argument symbol have been added a upper footnote
Figure 128346DEST_PATH_IMAGE039
Wherein, if
Figure 797225DEST_PATH_IMAGE036
Website in cycle
Figure 993851DEST_PATH_IMAGE024
Copy need to reappose, then
Figure 482601DEST_PATH_IMAGE040
, otherwise
Figure 672143DEST_PATH_IMAGE041
Figure 195528DEST_PATH_IMAGE042
Be illustrated in
Figure 297476DEST_PATH_IMAGE036
Cycle is interior with point leaving from station
Figure 539102DEST_PATH_IMAGE024
The copy of nearest website is transferred to website
Figure 266755DEST_PATH_IMAGE024
Cost.
Optimized algorithm based on storage space is as follows:
1, file f i With respect to operation j k The degree of correlation be defined as operation j k Accessed f i Total degree, be expressed as
2, file f i With respect to the local set of a group job The operation that is defined as among the operation set J of the degree of correlation accessed file f i Total degree, namely
Figure 646418DEST_PATH_IMAGE045
If the 3 operation set expressions with local runtime are J Local , file f i With respect to local operation collection J Local The degree of correlation be designated as
Figure 174833DEST_PATH_IMAGE046
Then the target of the copy replication strategy of file dynamic combined be keep as far as possible have higher G Local File be stored in this locality, and the deletion have lower G Local File, be the establishment vacating space of latest copy.
In the cloud storage system, a plurality of files can occur often by the situation of group interview, have undoubtedly the stronger degree of association between these files, yet these files may be distributed on a plurality of nodes.The copy that the present invention selects to be replicated expends cost also for minimum not only at the current Least-cost that expends in the time afterwards, obtain overall optimum efficiency.
Algorithm of the present invention is by with the file dynamic combined, obtain node often copy and the packing of a plurality of files of access be stored in local node, reduce the number of times of the long-range file reading of node, finally reach the target that reduces the user data access delay.
Description of drawings
Fig. 1 is the structured flowchart of reality of the present invention.
Embodiment
A kind of optimization method based on file storage dynamic aggregation of the present invention, this optimization method passes through the file dynamic combined, often copy and the packing of the file of access are stored in local node to obtain node, reduce the number of times of the long-range file reading of node, reduce the user data access delay, concrete steps are as follows:
1) the based on data master determines whether that originally data are overlapping, and whether the demand file that the inspection user sends has data overlapping at local node, and then there is local node in this demand file; Countless when overlapping, then enter step 2);
Total n the node in the system of setting up departments , each node contains m file, and the user sends file request to a certain node when access, suppose that request comprises a plurality of files, and this request can be expressed as a file set
Figure 116561DEST_PATH_IMAGE002
Due to node
Figure 332779DEST_PATH_IMAGE003
There is data owner originally
Figure 402235DEST_PATH_IMAGE004
, so by
Figure 692402DEST_PATH_IMAGE004
The set that generates
Figure 369371DEST_PATH_IMAGE005
Be the object that need to copy
Figure 994256DEST_PATH_IMAGE006
If the data owner of demand file originally is , when the user accesses, to arbitrarily With
Figure 938575DEST_PATH_IMAGE007
, two kinds of situations can be arranged:
Figure 50757DEST_PATH_IMAGE008
Perhaps , namely With
Figure 824175DEST_PATH_IMAGE007
May there be overlapping data also may not have overlapping data.
When
Figure 161002DEST_PATH_IMAGE009
, when namely having data overlapping, there is local node in this demand file.When
Figure 189001DEST_PATH_IMAGE008
, namely countless when overlapping, enter content 2.Here with the set of crossover data content
Figure 308267DEST_PATH_IMAGE010
Expression:
Figure 763519DEST_PATH_IMAGE011
, can release thus:
Figure 850293DEST_PATH_IMAGE012
, the purpose of describing like this is to make to work as With Not carrying out overlapping Data duplication when same website all exists propagates.
The optimization algorithm of 2) placing based on copy, check whether this locality holds demand file with enough storage spaces, storage space is arranged, then select one to expend the minimum remote node of cost and obtain this demand file copy, cost include file transmission time and the queuing time on remote node; Without storage space, then turn step 3).
For certain file , suppose that its copy trace utilization cost is
Figure 324819DEST_PATH_IMAGE014
, wherein, the cost that creates a Copy is
Figure 694621DEST_PATH_IMAGE015
, the expense of access copy is
Figure 788479DEST_PATH_IMAGE016
, normal number
Figure 585533DEST_PATH_IMAGE017
With
Figure 381320DEST_PATH_IMAGE018
Expression creates and access copy shared weight in total cost, then Replica Placement Strategy is exactly under the constraint condition that satisfies regulation, so that the cost summation of copy creating and access Minimum.
Suppose the website that creates a Copy
Figure 205554DEST_PATH_IMAGE047
Storage space be
Figure 509144DEST_PATH_IMAGE021
, then the average storage space of all websites is
Figure 220748DEST_PATH_IMAGE022
, and
Figure 289198DEST_PATH_IMAGE023
Average storage space and website have been reflected
Figure 428056DEST_PATH_IMAGE013
The ratio of storage space.
Suppose:
Figure 932855DEST_PATH_IMAGE004
Be illustrated in certain period website
Figure 448150DEST_PATH_IMAGE013
The request number of times of certain copy, website
Figure 371107DEST_PATH_IMAGE024
Access copy website
Figure 680865DEST_PATH_IMAGE013
An expense be
Figure 672961DEST_PATH_IMAGE025
(available website With
Figure 503831DEST_PATH_IMAGE024
Between shortest path time representation), so
Figure 984491DEST_PATH_IMAGE026
Can find out that the file request number of times is more, function
Figure 729462DEST_PATH_IMAGE016
Be worth larger; If
Figure 320980DEST_PATH_IMAGE027
, namely website does not send the request of access to wave file, and it does not just produce the expense of access copy so.Have according to above-mentioned analysis
Figure 218529DEST_PATH_IMAGE028
(1)
Therefore, the copy Placement Problems can be converted into following optimization problem:
Objective function:
(2)
Constraint condition:
(3)
Figure 234917DEST_PATH_IMAGE031
Figure 986972DEST_PATH_IMAGE032
(4)
Figure 809435DEST_PATH_IMAGE033
Figure 263419DEST_PATH_IMAGE034
(5)
Wherein, objective function (2) respectively asks website to deposit the cost weighted sum of website minimum with its nearest copy so that the copy creating cost of each website adds,
Figure 462319DEST_PATH_IMAGE017
With
Figure 68881DEST_PATH_IMAGE018
Size be empirical value, arrange according to copy storage space and the copy request number of times influence degree to the response time; The copy number that constraint condition (3) expression can create is
Figure 62245DEST_PATH_IMAGE035
Individual; Constraint condition (4) represents that each request website corresponding only copy website is its service; Constraint condition (5) is if the expression copy is positioned at website
Figure 3525DEST_PATH_IMAGE024
, website so
Figure 6116DEST_PATH_IMAGE013
The copy request can only be assigned to website
Figure 467184DEST_PATH_IMAGE024
Calculating exists
Figure 631449DEST_PATH_IMAGE036
Transmission cost in time cycle is to determine that whether reapposing copy exchanges optimum average response time for.Therefore, new objective function is changed to by formula (2)
Figure 60025DEST_PATH_IMAGE037
(6)
Compare with formula (2), increased by one in the new target function type (8) and adjusted cost Cumulative, it is right to realize
Figure 916303DEST_PATH_IMAGE036
Iterative computation in time cycle, each argument symbol have been added a upper footnote
Figure 517048DEST_PATH_IMAGE039
Wherein, if
Figure 164412DEST_PATH_IMAGE036
Website in cycle Copy need to reappose, then
Figure 678887DEST_PATH_IMAGE040
, otherwise
Figure 450534DEST_PATH_IMAGE041
Figure 588123DEST_PATH_IMAGE042
Be illustrated in
Figure 470628DEST_PATH_IMAGE036
Cycle is interior with point leaving from station
Figure 760795DEST_PATH_IMAGE024
The copy of nearest website is transferred to website
Figure 703344DEST_PATH_IMAGE024
Cost.
3) based on the optimized algorithm of storage space, the number of times of the long-range read requests file of computing node t 1Read the number of times of minimum file in the All Files of calculating book ground node storage t 2If, t 1 t 2, then delete the latter and vacating space storage resource request file.
Optimized algorithm based on storage space is as follows:
1, file f i With respect to operation j k The degree of correlation be defined as operation j k Accessed f i Total degree, be expressed as
Figure 328229DEST_PATH_IMAGE043
2, file f i With respect to the local set of a group job
Figure 14425DEST_PATH_IMAGE044
The operation that is defined as among the operation set J of the degree of correlation accessed file f i Total degree, namely
Figure 159099DEST_PATH_IMAGE045
If the 3 operation set expressions with local runtime are J Local , file f i With respect to local operation collection J Local The degree of correlation be designated as
Figure 272548DEST_PATH_IMAGE046
Then the target of the copy replication strategy of file dynamic combined be keep as far as possible have higher G Local File be stored in this locality, and the deletion have lower G Local File, be the establishment vacating space of latest copy.
The copy that the present invention selects to be replicated expends cost also for minimum not only at the current Least-cost that expends in the time afterwards, obtain overall optimum efficiency.

Claims (1)

1. optimization method based on file storage dynamic aggregation, it is characterized in that this optimization method passes through the file dynamic combined, often copy and the packing of the file of access are stored in local node to obtain node, reduce the number of times of the long-range file reading of node, reduce the user data access delay, concrete steps are as follows:
1) the based on data master determines whether that originally data are overlapping, and whether the demand file that the inspection user sends has data overlapping at local node, and then there is local node in this demand file; Countless when overlapping, then enter step 2); The based on data master determines whether that originally the overlapping algorithm of data is as follows:
Total n node S={S in the system sets up departments 1, S 2... S n, each node contains m file, and the user sends file request to a certain node when access, suppose that request comprises a plurality of files, and this request can be expressed as a file set D={d 1... d kD p;
Due to node S iThis R of data owner is arranged i, so by R iThe set R that generates is the object R={R that need to copy i; 1≤i≤n}; If this is R for the data owner of demand file j, when the user accesses, to R arbitrarily iAnd R j, two kinds of situations can be arranged:
Figure FDA00002367930400011
Perhaps Be R iWith R jMay there be overlapping data also may not have overlapping data;
When When namely having data overlapping, there is local node in this demand file; When Be countless when overlapping, enter step 2); With crossover data content set R IjExpression: R Ij=R i∩ R j, then: Even work as R iAnd R jNot carrying out overlapping Data duplication when same website all exists propagates;
The optimization algorithm of 2) placing based on copy, check whether this locality holds demand file with enough storage spaces, storage space is arranged, then select one to expend the minimum remote node of cost and obtain this demand file copy, cost include file transmission time and the queuing time on remote node; Without storage space, then turn step 3); The optimization algorithm of placing based on copy is as follows:
For certain file i, suppose that its copy trace utilization cost is Z i, wherein, the cost that creates a Copy is Z 1, the expense of access copy is Z 2, normal number α iAnd β iExpression creates and access copy shared weight, then Z in total cost i=α iZ 1+ β iZ 2Replica Placement Strategy is exactly under the constraint condition that satisfies regulation, so that the cost summation Z=∑ Z of copy creating and access iMinimum;
The storage space of supposing the website i that creates a Copy is S i, then the average storage space of all websites is
Figure FDA00002367930400021
And
Figure FDA00002367930400022
The ratio that has reflected the storage space of average storage space and website i;
Suppose: R iBe illustrated in certain period the request number of times to certain copy of website i, the expense of website j access copy website i is C Ij, the shortest path time representation between available website i and the j, so Z 2=R iC IjBe that the file request number of times is more, function Z 2Be worth larger; If R i=0, namely website does not send the request of access to wave file, and it does not just produce the expense of access copy so; Have according to above-mentioned analysis
Z = S ‾ Σ i = 1 n α i S i + Σ i = 1 n β i Σ j = 1 n R j C ij X ij - - - ( 1 )
Therefore, the copy Placement Problems can be converted into following optimization problem:
Objective function:
min Z = S ‾ Σ i = 1 n α i S i + Σ i = 1 n β i Σ j = 1 n R j C ij X ij - - - ( 2 )
Constraint condition:
Σ j = 1 n y j = p - - - ( 3 )
Σ j = 1 n x ij = 1 ( i = 1 , . . . , n ) - - - ( 4 )
x ij-y j≤0(i＝1，…,n;j＝1，…，n)(5)
Wherein, objective function (2) respectively asks website to deposit the cost weighted sum of website minimum with its nearest copy so that the copy creating cost of each website adds, α iAnd β iSize be empirical value, arrange according to copy storage space and the copy request number of times influence degree to the response time; The copy number that constraint condition (3) expression can create is p; Constraint condition (4) represents that each request website corresponding only copy website is its service; Constraint condition (5) is if the expression copy is positioned at website j, and the copy request of website i can only be assigned to website j so;
Calculating is at the transmission cost of K in the time cycle, to determine that whether reapposing copy exchanges optimum average response time for; Therefore, new objective function is changed to by formula (2)
min Z = Σ k = 1 n Z ( k ) + Σ k = 2 K Σ j = 1 n a j ( k ) b j ( k ) - - - ( 6 )
Compare with formula (2), increased by one in the new target function type (6) and adjusted cost a jb jCumulative, realize that to the iterative computation of K in the time cycle each argument symbol has added a upper footnote (k); Wherein, if need to reappose at the copy of K website j in the cycle, a then j=1, otherwise a j=0; b jThe copy that is illustrated in the K cycle website that leaving from station some j is nearest is transferred to the cost of website j;
3) based on the optimized algorithm of storage space, the number of times t of the long-range read requests file of computing node 1Read the number of times t of minimum file in the All Files of calculating book ground node storage 2If, t 1T 2, then delete the latter and vacating space storage resource request file;
Optimized algorithm based on storage space is as follows:
File f iWith respect to operation j kThe degree of correlation be defined as operation j kAccessed f iTotal degree, be expressed as
Figure FDA00002367930400032
File f iWith respect to the local set of group job J=(J 1, J 2..., J n) the operation that is defined as among the operation set J of the degree of correlation accessed file f iTotal degree, namely
Figure FDA00002367930400033
If be J with the operation set expression of local runtime Local, file f iWith respect to local operation collection J LocalThe degree of correlation be designated as
Figure FDA00002367930400034
Then the target of the copy replication strategy of file dynamic combined is to keep as far as possible to have higher G LocalFile be stored in this locality, and deletion has low G LocalFile, be the establishment vacating space of latest copy.
CN 201110086026 2011-04-07 2011-04-07 File storage dynamic aggregation based optimization method Expired - Fee Related CN102156730B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN 201110086026 CN102156730B (en) 2011-04-07 2011-04-07 File storage dynamic aggregation based optimization method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN 201110086026 CN102156730B (en) 2011-04-07 2011-04-07 File storage dynamic aggregation based optimization method

Publications (2)

Publication Number Publication Date
CN102156730A CN102156730A (en) 2011-08-17
CN102156730B true CN102156730B (en) 2013-03-20

Family

ID=44438229

Family Applications (1)

Application Number Title Priority Date Filing Date
CN 201110086026 Expired - Fee Related CN102156730B (en) 2011-04-07 2011-04-07 File storage dynamic aggregation based optimization method

Country Status (1)

Country Link
CN (1) CN102156730B (en)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102902762B (en) * 2012-09-25 2015-08-19 华为技术有限公司 A kind of methods, devices and systems of deleting duplicated data
CN109561152B (en) * 2018-12-13 2020-07-24 深圳先进技术研究院 Data access request response method, device, terminal and storage medium
CN118626315B (en) * 2024-08-09 2024-10-18 云南缮阳生物科技有限公司 Biological polypeptide database sharing interaction method and system

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1996305A (en) * 2006-12-22 2007-07-11 华为技术有限公司 Data saving and reading method and device therefore, and data transmission system
CN101187931A (en) * 2007-12-12 2008-05-28 浙江大学 Distribution type file system multi-file copy management method
CN101902498A (en) * 2010-07-02 2010-12-01 广州鼎甲计算机科技有限公司 Network technology based storage cloud backup method
CN101909068A (en) * 2009-06-02 2010-12-08 华为技术有限公司 Method, device and system for managing file copies
CN101916289A (en) * 2010-08-20 2010-12-15 浙江大学 Method for establishing digital library storage system supporting mass small files and dynamic backup number

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP4320195B2 (en) * 2003-03-19 2009-08-26 株式会社日立製作所 File storage service system, file management apparatus, file management method, ID designation type NAS server, and file reading method
US7966288B2 (en) * 2008-01-02 2011-06-21 Sandisk Il Ltd. Device and method for integrating file systems
CN101751309B (en) * 2009-12-28 2011-06-29 北京理工大学 Optimized transcript distributing method in data grid
CN101753625B (en) * 2009-12-28 2012-09-05 北京理工大学 Method for deployment of copy service and copy establishment in peer-to-peer network environment

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1996305A (en) * 2006-12-22 2007-07-11 华为技术有限公司 Data saving and reading method and device therefore, and data transmission system
CN101187931A (en) * 2007-12-12 2008-05-28 浙江大学 Distribution type file system multi-file copy management method
CN101909068A (en) * 2009-06-02 2010-12-08 华为技术有限公司 Method, device and system for managing file copies
CN101902498A (en) * 2010-07-02 2010-12-01 广州鼎甲计算机科技有限公司 Network technology based storage cloud backup method
CN101916289A (en) * 2010-08-20 2010-12-15 浙江大学 Method for establishing digital library storage system supporting mass small files and dynamic backup number

Also Published As

Publication number Publication date
CN102156730A (en) 2011-08-17

Similar Documents

Publication Publication Date Title
AU2009223346B2 (en) Web-based multiuser collaboration
US9311731B2 (en) Dynamic graph system for a semantic database
CN103067297B (en) A kind of dynamic load balancing method based on resource consumption prediction and device
CN102546782A (en) Distribution system and data operation method thereof
CN103329151A (en) Recommendations based on topic clusters
Zhang et al. Clustering based virtual machines placement in distributed cloud computing
CN101370025A (en) Storing method, scheduling method and management system for geographic information data
CN103595780A (en) Cloud computing resource scheduling method based on repeat removing
US9734162B2 (en) Geospatial queries using associated distribution data
JP2017021772A (en) Copula-theory based feature selection
CN102063330A (en) Performance data acquisition method for large-scale parallel program
Newton et al. Performance assessment of urban precinct design: a scoping study
CN101504663A (en) Swarm intelligence based spatial data copy self-adapting distribution method
Dang et al. Combination of replication and scheduling in data grids
CN102156730B (en) File storage dynamic aggregation based optimization method
CN112214328A (en) Load balancing method for transaction processing in isomorphic multilink
US11107037B2 (en) Method and system of sharing product data in a collaborative environment
Xie et al. Multi-objective optimization of data deployment and scheduling based on the minimum cost in geo-distributed cloud
Jaiswal et al. Minimum carbon dioxide emission based selection of traffic route with unsignalised junctions in tandem network
CN103984737A (en) Optimization method for data layout of multi-data centres based on calculating relevancy
Xiao et al. Graph Neural Network-Based Design Decision Support for Shared Mobility Systems
Marzuni et al. Cross-MapReduce: Data transfer reduction in geo-distributed MapReduce
CN105740249A (en) Processing method and system during big data operation parallel scheduling process
Alsaryrah et al. A fast iot service composition scheme for energy efficient qos services
CN103780426A (en) Consistency maintenance methods for cloud storage data and cloud storage system

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
C14 Grant of patent or utility model
GR01 Patent grant
ASS Succession or assignment of patent right

Owner name: STATE ELECTRIC NET CROP.

Free format text: FORMER OWNER: JIANGSU ELECTRIC POWER COMPANY

Effective date: 20130718

Owner name: JIANGSU ELECTRIC POWER COMPANY

Effective date: 20130718

C41 Transfer of patent application or patent right or utility model
COR Change of bibliographic data

Free format text: CORRECT: ADDRESS; FROM: 210024 NANJING, JIANGSU PROVINCE TO: 100031 XICHENG, BEIJING

TR01 Transfer of patent right

Effective date of registration: 20130718

Address after: 100031 Xicheng District West Chang'an Avenue, No. 86, Beijing

Patentee after: State Grid Corporation of China

Patentee after: JIANGSU ELECTRIC POWER Co.

Address before: 210024 Shanghai Road, Jiangsu, China, No. 215, No.

Patentee before: JIANGSU ELECTRIC POWER Co.

ASS Succession or assignment of patent right

Owner name: JIANGSU ELECTRIC POWER COMPANY INFORMATION COMMUNI

Free format text: FORMER OWNER: JIANGSU ELECTRIC POWER COMPANY

Effective date: 20150403

TR01 Transfer of patent right

Effective date of registration: 20150403

Address after: 100031 Xicheng District West Chang'an Avenue, No. 86, Beijing

Patentee after: State Grid Corporation of China

Patentee after: JIANGSU ELECTRIC POWER Co.

Patentee after: INFORMATION & TELECOMMUNICATION BRANCH OF STATE GRID JIANGSU ELECTRIC POWER Co.

Address before: 100031 Xicheng District West Chang'an Avenue, No. 86, Beijing

Patentee before: State Grid Corporation of China

Patentee before: JIANGSU ELECTRIC POWER Co.

CF01 Termination of patent right due to non-payment of annual fee

Granted publication date: 20130320

CF01 Termination of patent right due to non-payment of annual fee