Background technology
Along with the explosive increase of data, the cluster of storage mass data arises at the historic moment. Because store in clusterData volume very large, what have reaches PB level, synchronization may have a large amount of data accesses. Therefore,Whether the deposit position of data is reasonable, directly has influence on the access delay of data.
Data Migration, refers to the moving process of data between different storage mediums. Under normal circumstances, numberThe number of times occurring according to migration is all fewer, and object is also how relevant with system upgrade or data backup. But dividingData Migration in level storage system, the number of times of generation is many, its objective is the position in order to adjust data,Make the access performance of all data in system reach optimum. In hierarchical stor, conventionally can be situated between according to storageThe performance of matter is by its layering, and storage hierarchy is higher, configures highlyer, and access performance is better.
Complete Data Migration, the first step will be selected the object of migration, and second step is source and the order of selecting migrationMark, final step is only real data mobile. This is the process of then carrying out of first making a plan, itsMiddle first two steps are exactly the process of making a plan, and directly concern the effect of whole Data Migration. Select migration rightElephant according to being a numerical value, this numerical value is the direct reflection to data access " temperature ", numerical value is higher,Temperature is higher. And the target of Data Migration is exactly, data high to " temperature " are retained in to higher accumulation layerInferior upper, the data that " temperature " is low are retained in lower storage hierarchy.
Migration needs consumes resources, comprises hardware resource and Internet resources, and this consumption meeting is to system bandCarry out certain negative effect, therefore the data volume of migration should be more few better. For this reason, a lot of migration strategys are just establishedDetermine threshold value, only had the temperature value of data to exceed threshold value, just can move, be no more than threshold value, can not moveMove, arrange like this, really make Data Migration amount reduce a lot, but arranging of threshold value is more difficult. CauseFor larger variation may occur the load in system, make the temperature of data that very large variation also occur,If threshold value does not make corresponding changes, be just difficult to adapt to, therefore needing has dynamic threshold value. Different threshold valuesMethod to set up inconsistent, some threshold values are relative value, for example the data temperature value on lower storage level justAs long as the temperature value of 60% data in a high storage hierarchy thereon, these data just can be moved above, this thresholdValue is a kind of dynamic threshold value really, if but the data in higher storage level are all high temperatures, butBe that its remaining space also has a lot, data below cannot be moved into. Do not make full use of the storage money of high-qualitySource. Therefore, when screening migrating objects, must consider the remaining space of storage level, could protect like thisThe reasonable utilization of card storage resources. In addition, also some migration strategy, in screening migrating objects, usesBe " rule set ", as long as migrating objects met rule, just move this " rule set "Need very professional personage to set, and for single application scenarios, be difficult to adapt to the negative of dynamic changeCarry.
Summary of the invention
The embodiment of the present invention provides a kind of method and apparatus of specified data migrating objects, is intended to solve current trueThe method of given data migrating objects does not make full use of the storage resources of high-quality, makes to also have very at remaining spaceWhen many, data below can not be moved into. Also there is the problem that is difficult to the load that adapts to dynamic change simultaneously.
For this reason, the embodiment of the present invention provides following technical scheme:
A device for specified data migrating objects, comprising:
Local screening object select module, for choosing valuation data all in each storage hierarchy, asThe object of local screening;
Local screening module, is connected with part screening object select module, and its each storage hierarchy existsThe threshold value having set, for by the temperature value of described valuation data and described threshold value comparison, carries out partScreening obtains the result of local screening;
Overall situation screening module, is connected with part screening module, for the result that described part is screened, knotThe remaining space that closes each storage hierarchy screens again, filters out the data object that does not need migration, obtainsThe object of described Data Migration;
More new module, after finishing, upgrades described threshold value according to migration results for described Data Migration.
The embodiment of the present invention also provides a kind of method of specified data migrating objects, comprises the following steps:
A: each storage hierarchy exists the threshold value having set, by the temperature value of valuation data with described inThreshold value comparison, carries out local screening and obtains local result of screening;
B: by the result of described local screening, again screen in conjunction with the remaining space of each storage hierarchy,Obtain described Data Migration object;
C: after described Data Migration finishes, upgrade described threshold value according to migration results.
Compared with prior art, embodiments of the invention tool has the following advantages:
The present invention utilizes local screening and overall situation screening to combine, first in each storage hierarchy from row filter,Again the result of screening by part is further screened, in conjunction with the remaining space of storages at different levels, determine numberAccording to the object of migration, thus can make the object of migration the least possible and can adaptive system load dynamicallyChange, can also make full use of the storage resources of high-quality.
Detailed description of the invention
In order to make object of the present invention, technical scheme and advantage clearer, below in conjunction with accompanying drawing and realityExecute example, the present invention is further elaborated. Should be appreciated that described herein is only of the present inventionPart embodiment, instead of whole embodiment. Based on the embodiment in the present invention, the common skill in this areaThe every other embodiment that art personnel obtain under the prerequisite of not making creative work, belongs to thisThe scope of bright protection.
Fig. 1 is the method flow diagram of the specified data migrating objects that provides of the embodiment of the present invention, for the ease of sayingBright, only show the part relevant to the embodiment of the present invention.
As shown in Figure 1, the method comprises the following steps:
Step 101, there is the threshold value having set in each storage hierarchy, by the temperature value of valuation dataWith described threshold value comparison, carry out local screening and obtain local result of screening.
Be understandable that, while carrying out part screening, choose valuation data all in each storage hierarchy,As the object of part screening. Concrete, in hierarchical stor, there are valuation data, each valuationData have temperature value to answer in contrast, and described temperature value can be big or small.
In the present embodiment, described hierarchical stor is divided at least three levels, is first from top to bottom successivelyLayer, the second layer, the 3rd layer, be equivalent to a pyramidal structure, and can between any two levelsCarry out Data Migration, each storage hierarchy can be toward other storage hierarchy migration datas. Wherein, firstLayer, toward the second layer or the 3rd layer of migration, moves under being called; The 3rd layer is moved toward the second layer or ground floor, is calledMove.
Be understandable that, the hierarchical stor of three levels always has the direction of 6 Data Migrations.
Concrete, described threshold value is a nonnegative real number, can be big or small.
Described threshold value comprises upper limit threshold and lower threshold, preferred, first upper limit threshold is set as to 0,Lower threshold is set as positive infinity.
Concrete, temperature value is greater than to upper limit threshold or is less than the described valuation data of lower threshold, asThe result of local screening. The data that are less than upper limit threshold by temperature value or are greater than lower threshold screen out.
Step 102, by the result of described local screening, carries out again in conjunction with the remaining space of each storage hierarchyScreening, filters out the data object that does not need migration, obtains described Data Migration object.
Concrete, the result that described local screening is obtained, arranges from big to small according to the size of its temperature value,Match with storage hierarchy, then in storage hierarchy, attempt depositing.
Concrete, if can hold described data in described storage hierarchy, deposit successfully;
Concrete, if can not hold described data, described data attempt depositing to next stage storage hierarchyPut, until described data are all deposited successfully.
In the present embodiment, it is in the situation that not considering the original memory location of data that described trial is deposited, a rootAccording to the remaining space of its temperature value and storage hierarchys at different levels, determine its deposit position because having. Deposit if originalStorage space put with because of have to deposit level consistent, described data do not need migration; If original memory location withBecause of have to deposit level inconsistent, because the deposit position that has is for moving target.
Remaining space refers to the difference between the best usage space of described storage hierarchy and actual usage space.
The ratio of the best usage space usage spaces whole with it of one-level storage is less than 1, is generally 80%,The ratio of the usage space usage spaces whole with it of all the other storage hierarchys can be adjusted according to actual conditions.
Step 103, after described Data Migration finishes, upgrades described threshold value according to migration results.
Preferably, after described Data Migration finishes, on being updated in this storage hierarchy, moves described upper limit thresholdData in minimum temperature value, the maximum temperature in the data that lower threshold moves under being updated in this storage levelValue.
Preferably, if not, move or under move generation, the threshold value that maintains this storage hierarchy is constant.
Based on identical design, the embodiment of the present invention also provides a kind of device of specified data migrating objects, asShown in Fig. 2, this device comprises:
Local screening object select module 201, for choosing valuation data all in each storage hierarchy,As the object of part screening.
Concrete, in system, there are the data of valuation, each valuation data have temperature value in contrastShould, described temperature value can be big or small.
Local screening module 202, is connected with part screening object select module 201, its each accumulation layerThe inferior threshold value having set that all exists, for by the temperature value of described valuation data and described threshold value comparison,Carry out local screening and obtain local result of screening.
Be understandable that, while carrying out part screening, choose valuation data all in each storage hierarchy,As the object of part screening. Concrete, in system, there are valuation data, each valuation data haveTemperature value is answered in contrast, and described temperature value can be big or small.
Concrete, threshold value is a nonnegative real number, can be big or small.
Described threshold value comprises upper limit threshold and lower threshold, preferred, first upper limit threshold is set as to 0,Lower threshold is set as positive infinity.
Concrete, temperature value is greater than to upper limit threshold or is less than the described valuation data of lower threshold, asThe result of local screening. The data that are less than upper limit threshold by temperature value or are greater than lower threshold screen out.
Overall screening module 203, is connected with described local screening module 202, for described part is sievedThe result of choosing, screens again in conjunction with the remaining space of each storage hierarchy, filters out the number that does not need migrationAccording to object, obtain the object of described Data Migration.
Concrete, the result that described local screening is obtained, arranges from big to small according to the size of its temperature value,Match with storage hierarchy, then in storage hierarchy, attempt depositing.
Concrete, if can hold described data in described storage hierarchy, deposit successfully;
Concrete, if can not hold described data, described data attempt depositing to next stage storage hierarchyPut, until described data are all deposited successfully.
In the present embodiment, it is in the situation that not considering the original memory location of data that described trial is deposited, a rootAccording to the remaining space of its temperature value and storage hierarchys at different levels, determine its deposit position because having. Deposit if originalStorage space put with because of have to deposit level consistent, described data do not need migration; If original memory location withBecause of have to deposit level inconsistent, because the deposit position that has is for moving target.
Remaining space refers to the difference between the best usage space of described storage hierarchy and actual usage space.
The ratio of the best usage space usage spaces whole with it of one-level storage is less than 1, is generally 80%,The ratio of the usage space usage spaces whole with it of all the other storage hierarchys can be adjusted according to actual conditions.
More new module 204, after finishing, upgrades described threshold value according to migration results for described Data Migration.
Preferably, after described Data Migration finishes, on being updated in this storage hierarchy, moves described upper limit thresholdData in minimum temperature value, the maximum temperature in the data that lower threshold moves under being updated in this storage levelValue.
Preferably, if not, move or under move generation, the threshold value that maintains this storage hierarchy is constant.
The local screening of embodiment of the present invention utilization and overall situation screening combine, first in each storage hierarchy voluntarilyScreening, then the result of screening by part is further screened, in conjunction with the remaining space of storages at different levels,The object of specified data migration, thus can make the object of migration the least possible and can adaptive system loadDynamic change, can also make full use of the storage resources of high-quality.
It will be appreciated by those skilled in the art that module in the device in embodiment can according to embodiment describe intoRow is distributed in the device of embodiment, also can carry out respective change be positioned at be different from one of the present embodiment orIn multiple devices. The module of above-described embodiment can be merged into a module, also can further split into manyIndividual submodule.
Through the above description of the embodiments, those skilled in the art can be well understood to the present inventionThe mode that can add essential general hardware platform by software realizes, and can certainly pass through hardware, but veryUnder susceptible condition, the former is better embodiment. Based on such understanding, technical scheme of the present invention in essenceThe part in other words prior art being contributed can embody with the form of software product, this computerSoftware product is stored in a storage medium, comprises that some instructions are in order to make a station terminal equipment (passableMobile phone, personal computer, server, or the network equipment etc.) carry out described in each embodiment of the present inventionMethod.
The above is only the preferred embodiment of the present invention, it should be pointed out that common for the artTechnical staff, under the premise without departing from the principles of the invention, can also make some improvements and modifications,These improvements and modifications also should be looked protection scope of the present invention.