CN105607967A - Data center-oriented energy consumption perception-based data backup method - Google Patents

Abstract

The invention relates to a data center-oriented energy consumption perception-based data backup method and belongs to the technical field of big data energy conservation backup. The method comprises the steps of creating a backup task, starting backup, and performing a repeated data deletion process for all data; recording a current repeated data deletion parameter by a backup dataset subjected to repeated data deletion; calculating the values of Preduced and Pincremental after the repeated data deletion process each time, wherein Preduced represents the energy consumption reduced due to the adoption of the repeated data deletion technology in the backup process, and Pincremental represents the energy consumption increased due to the adoption of the repeated data deletion technology in the backup process; and recording the values in dataset metadata and judging a numerical relationship of Preduced and Pincremental; if the relationship of Preduced<Pincremental occurs for continuous three times in the dataset, indicating that the dataset is no longer subjected to repeated data deletion, namely, the dataset is backed up according to a normal process; on the contrary, as long as the situation of Preduced<Pincremental does not occur for continuous three times in the dataset, continuing to perform repeated data deletion and detection; and no matter whether the repeated data deletion is carried out or not, executing the backup task as usual until the backup task is naturally ended or terminated. According to the method, the purpose of energy consumption optimization is achieved.

Description

The data back up method of a kind of data-oriented center based on energy consumption perception

Technical field

The present invention relates to a kind of backup method towards energy optimization, belong to the energy-conservation redundancy technique of large data field.

Background technology

In the data back up method of data center, the efficiency of data de-duplication mainly by data set before overweight deletingThe ratio taking up room afterwards decides, and this ratio is commonly called compression ratio or duplicate removal ratio. And repeating data is deletedThe performance of removing is mainly to evaluate by data set overall read-write speed in backup or storing process.

The backup method existing at present has the backup method of facing cloud backup services. The method has a special inspection file to repairChange situation and make the module (middleware) of respective handling, and can be embedded in existing standby system and go. And canDifferent classes of according to data centralization file, has designed an intelligent data deduplication system. This system set notWith the data partition method of dynamics, comprise single-instance (SIS), fixed block is divided (fixed-sizedpartion, FSP),Content-based division (CDC) and sliding shoe are divided (slidingblock). It can be according to the type of file, as textDocument, audio file, image file, linux sound code file etc., select division methods, to obtain best duplicate removal ratio.

In addition also has the backup policy towards throughput. This strategy will heavily be deleted than including the model of measurement index in BACKUP TIMEEnclose, but these indexs can not directly embody the energy-output ratio of backup tasks. Although one have height heavily delete than withLow time overhead, high DET is worth backup data set, necessarily also has higher energy consumption efficiency in backup procedure.

But above backup method can not accurately be distinguished the performance of these backup tasks in energy consumption. Therefore, needFormulate one from energy consumption angle, based on the backup policy of energy consumption perception.

Summary of the invention

Technology of the present invention is dealt with problems and is: overcome the deficiencies in the prior art, provide a kind of data-oriented center based on energy consumptionThe data back up method of perception, to reach the object of energy optimization.

The technology of the present invention solution: the data back up method of a kind of data-oriented center based on energy consumption perception, its feature existsIn comprising the following steps:

(1) backup tasks, once establishment, starts backup, and full backup task all needs through data de-duplication process,Obtain the backup data set of data de-duplication;

(2) backup data set to data de-duplication, records this heavy backup data set parameters of deleting, described parameterComprise size, heavily delete ratio, BACKUP TIME; Wherein size is divided into initial data size, and logic size and process repeating data are deletedExcept being stored in the size of data on medium, i.e. physical size after process; Heavily delete than being that to weigh data de-duplication effect the most directly perceivedData, i.e. the logic size of backup data set and the ratio of physical size; Above three parameters are for calculating in step (3)Intermediate variable;

(3) calculate P after each data de-duplication process_reducedAnd P_incrementalValue, and be recorded in Backup Data element of setIn data, and judge P_reducedAnd P_incrementalMagnitude relationship, wherein P_reducedRepresent that backup procedure adopts repeating data to deleteThe energy consumption reducing except technology, P_incrementalThe energy consumption that represents backup procedure to adopt data de-duplication technology and to increase;

(4) if there is P continuous 3 times in a data set_reduced＜P_incrementalRelation, this backup data set is no longerHeavily delete, backup data set backs up according to normal flow; Otherwise, as long as existing for continuous 3 times does not appear in data setP_reduced＜P_incrementalSituation, continue duplicate removal and detect;

(5) no matter heavily whether delete, backup tasks can as usual be carried out nature and finishes or be terminated.

The number of times of the data de-duplication process in described step (1) is less than 3.

P in described step (2)_incrementalComputing formula as follows:

P_incremental＝∑_nodes((P_dedup-P_standby)*T_dedup(1.1)

Wherein, T_dedupFor the time that data de-duplication expends thereon, (P_dedup-P_standby) represent, at thisIn the section time, owing to carrying out data de-duplication process, the part that server power increases, P_dedupRepresent to carry out repetitionThe power of memory device after data delete procedure, P_standbyThe memory device power under idle condition.

Described (P_dedup-P_standby) or by (P_dedup-P_standby) be considered as the mean value that server power increases, and logicalCross energy consumption model

P_active＝P_standby+P_seq+P_rand(1.2)

Resource utilization situation with data de-duplication process, calculates, and wherein Pseq order of representation IO producesEnergy consumption, Prand represents the energy consumption that random IO produces, they are the relevant energy consumption of the load bringing due to IO expense.

P in described step (1)_reducedComputing formula as follows:

$P_{reduced}=(P_{MaxSeqIO}-P_{s\tan dby})*T_{iosave}*N_{active}+(P_{s\tan dby}-P_{idle})*T_{backup}*\frac{S_{dup}}{S_{energy\_unit}}---\left(1.3\right)$

(P_MaxSeqIO-P_standby) be the poor of memory device power under transmission state and under idle condition,P_MaxSeqIORepresent the memory device power under transmission state, P_standbyRepresent the memory device under idle conditionPower; T_iosaveRepresent the transmission time reducing due to data volume, record in actual experiment process; N_activeBe illustrated in the quantity of the memory device that transmitting procedure relates to; (P_standby-P_idle) represent under standby and in low meritMemory device power poor under consumption or closed condition, P_idleThe storage of expression under low-power consumption or closed condition establishedStandby power; T_backupRepresent the execution cycle of backup tasks, elapsed time before once carrying out on backup tasks is logicalThe configuration information of crossing backup tasks obtains;Expression can be by the quantity of idle equipment, wherein S_{energy_unit}Be equal toIn the capacity of the manipulable minimum memory of storage system unit, S_dupExpression can be by the capacity of idle equipment,S_{energy_unit}Be equal to the capacity of the manipulable minimum memory of storage system unit.

The present invention's advantage is compared with prior art:

(1) can not accurately distinguish the performance of these backup tasks in energy consumption using throughput as the index of screening completely.Formulate one from energy consumption angle, the backup policy that the present invention is based on energy consumption perception can reach the object of energy optimization.The impact of comprehensive two aspects is analyzed, and finds energy consumption to increase the equalization point reducing with energy consumption, is to formulate based on energy consumption senseThe key point of the backup policy of knowing. According to the energy consumption changing value calculating, copy the backup policy towards throughput,Data set to backup tasks screens. Then, the data set that standby system only passes through screening is heavily deleted operation,To reach the object of energy optimization.

(2) Fig. 2 has shown the average power consumption performance of standby system under Different Strategies impact, owing to having avoided heavily deletingThe energy that journey consumes utilizes idle hard disk to carry out energy-conservation simultaneously, and application the present invention is towards the backup method of energy optimizationExperimental group power consumption is minimum. Approximately 8% the energy consumption reducing than backup policy towards throughput, can close more how hardThe strategy towards energy optimization of dish has reduced the power of 300W. When data set size is larger, experimental facilities is largerTime, energy-saving effect will be more obvious.

Brief description of the drawings

Fig. 1 is the inventive method realization flow figure;

Fig. 2 is the average power consumption of different pieces of information collection standby system under Different Strategies.

Detailed description of the invention

As shown in Figure 1, be the backup decision making algorithm flow chart towards energy optimization

Because data de-duplication technology has dual character to backup procedure aspect energy consumption, therefore, comprehensive two aspectsImpact is analyzed, and finds energy consumption to increase the equalization point reducing with energy consumption, is to formulate backup policy based on energy consumption perceptionKey point. So-called energy consumption equalization point (energybalancepoint) refers to, and data de-duplication process causes standbyThe energy consumption expense that part server end increases is held because space requirement reduces the energy consumption of saving, the situation that the two is equal with storageUnder, the size of backup data set, heavily delete than etc. parameter state. Find the energy consumption equalization point of data set, needing to calculate shouldData set is after through data de-duplication process, at the energy consumption of server end increase and the energy reducing at storage system endConsumption.

Energy consumption mainly the conduct oneself with dignity extra time overhead of the process of deleting and extra computational resource expense that server end increases. WithGeneral backup procedure is compared, and data de-duplication has taken more computational resource within the more time. According to serviceDevice energy consumption model, can obtain calculating the formula of the energy consumption that this part increases:

P_incremental＝∑_nodes((P_dedup-P_standby)*T_dedup(1.4)

Use P_incrementalThe energy consumption that represents backup procedure to adopt data de-duplication technology and to increase. A backup tasks canCan need multiple servers to process, need to according to circumstances calculate respectively and gather. And for every station server, repeatIt is T that data are deleted the time of expending thereon_dedup. This time can heavily deleting than handling up with server by data setRate is calculated the ratio of logic size with physics size of heavily deleting than being backup data set by formula.

(P_dedup-P_standby) represent, within this period of time, owing to carrying out data de-duplication process, serverThe part of increased power. Can be obtained by experimental result, use the data de-duplication framework proposing heavily to delete, at clothesBusiness device does not have in the situation of other computation-intensive tasks, and its resource service condition is stable. Therefore, for convenientCalculate, by (P_dedup-P_standby) be considered as the mean value that server power increases, and by energy consumption model (1.2) andThe resource utilization situation of data de-duplication process, calculates result.

The energy consumption that storage system reduces can be divided into two parts, and a part is owing to heavily deleting that little data set size causesTransport overhead reduce the Energy Intensity Reduction bringing, another part is that the reduction taking up room causes how possible memory deviceCan be in low power operation or closed condition, thus the energy consumption of saving. According to server energy consumption model, calculatedThe formula of the energy consumption that this part increases:

$P_{reduced}=(P_{MaxSeqIO}-P_{s\tan dby})*T_{iosave}*N_{active}+(P_{s\tan dby}-P_{idle})*T_{backup}*\frac{S_{dup}}{S_{energy\_unit}}---(1.5)$

Use P_reducedThe energy consumption that represents backup procedure to adopt data de-duplication technology and to reduce. Transmitting procedure reduced overheadBe actually due to the transmission working time of associated hard disk reduced and cause. (P_MaxSeqIO-P_standby) be in passingPoor with memory device power under idle condition under defeated state. These memory devices need according to concrete storage systemDetermine with data set size. In general, these have the memory device of significant change with load is mainly hard disk, Ke YitongThe energy consumption model of crossing hard disk calculates, concrete formula (1.2).

In order to simplify sight, think in hard disk in the transmitting procedure here situation in maximum order IO load. T_iosaveTableShow the transmission time reducing due to data volume, can pass through data set size, duplicate removal rate and storage system bandwidth calculation obtainArrive. N_activeBe illustrated in the quantity of the memory device that transmitting procedure relates to, refer to the quantity of the hard disk relating to here.(P_standby-P_idle) represent the poor of memory device power under standby and under low-power consumption or closed condition. Specifically adoptCalculate with which state, which the state conversion that can control these equipment according to storage system determines. T_backupTableShow the execution cycle of backup tasks, elapsed time before once carrying out on backup tasks, can be by the setting of backup tasksInformation obtains.Expression can be by the quantity of idle equipment, wherein S_{energy_unit}Being equal to storage system can operateThe capacity of minimum memory unit. Do not allow according to the setting of storage system and function, the equipment minimum unit that it can be controlled may beHard disk, the RAID group of polylith hard disk formation or a storage subsystem that comprises corresponding Control Server.

Can calculate owing to having introduced data de-duplication technology in backup procedure by formula (1.6) aboveThe change of the energy consumption producing. From energy consumption angle, can be right in actual backup procedure by this energy consumption variation predictionBackup tasks data set carries out data de-duplication, and whole to including storage system and backup server producingThe impact of standby system. The energy consumption changing value calculating according to this, copies the backup policy towards throughput, to standbyThe data set of part task screens. Then, the data set that standby system only passes through screening is heavily deleted operation, to reachTo the object of energy optimization. Here the backup wheel subthreshold N before still continuing to use, in the backup policy proposing in the present invention,For pre-check weighing delete than backup round N be appointed as 3. This is because energy consumption prediction equally need to heavily deleting than entering data setRow prediction, need to, through the process of heavily deleting of certain number of times, add up data set characteristic parameter. Set threshold value P,And according to threshold value P and difference (P_reduced-P_incremental) magnitude relationship.

In order to make there is higher energy consumption efficiency through the data set of screening in the time carrying out data de-duplication process, in concrete realityIn existing, threshold value P is set to 0. The object of doing is like this to be bound to through the data set of data de-duplication in order to ensureTherefore reduce the energy consumption of standby system. The similar backup policy towards throughput, standby by towards energy optimization of paperThe flow definition of part strategy is as follows, as shown in Figure 1:

(1) backup tasks is once establishment, and starts backup, all need to pass through data de-duplication process.

(2) through the backup data set of data de-duplication, record this heavy parameters of deleting, comprise size, heavily delete ratio,BACKUP TIME etc. And according to heavily deleting rate estimation above, calculate and heavily delete ratio, the logic size of backup data set and physics are largeLittle ratio.

(3) calculate P after each data de-duplication process_reducedAnd P_incrementalValue, and be recorded in data set metadataIn. And judge P_reducedAnd P_incrementalMagnitude relationship.

(4) if there is P continuous 3 times in a data set_reduced<P_incrementalRelation, this data set no longer carries outHeavily delete, data set backs up according to normal flow. Otherwise, as long as data set does not occur having P continuous 3 times_reduced＜P_incrementalSituation, continue duplicate removal and detect.

(5) no matter heavily whether delete, backup tasks can as usual be carried out nature and finishes or be terminated.

Provide above embodiment to be only used to describe object of the present invention, and do not really want to limit the scope of the invention. ThisBright scope is defined by the following claims. Various being equal to that does not depart from spirit of the present invention and principle and make, replaces and repaiiesChange, all should contain within the scope of the present invention.

Claims (5)

1. the data back up method of data-oriented center based on energy consumption perception, is characterized in that comprising the following steps:

(1) backup tasks, once establishment, starts backup, and full backup task all needs through data de-duplication process,Obtain the backup data set of data de-duplication;

(2) backup data set to data de-duplication, records this heavy backup data set parameters of deleting, described parameterComprise size, heavily delete ratio, BACKUP TIME; Wherein size is divided into initial data size, and logic size and process repeating data are deletedExcept being stored in the size of data on medium, i.e. physical size after process; Heavily delete than being that to weigh data de-duplication effect the most directly perceivedData, i.e. the logic size of backup data set and the ratio of physical size; Above three parameters are for calculating in step (3)Intermediate variable;

(3) calculate P after each data de-duplication process_reducedAnd P_incrementalValue, and be recorded in Backup Data element of setIn data, and judge P_reducedAnd P_incrementalMagnitude relationship, wherein P_reducedRepresent that backup procedure adopts repeating data to deleteThe energy consumption reducing except technology, P_incrementalThe energy consumption that represents backup procedure to adopt data de-duplication technology and to increase;

(4) if there is P continuous 3 times in a data set_reduced<P_incrementalRelation, this backup data set is no longerHeavily delete, backup data set backs up according to normal flow; Otherwise, as long as existing for continuous 3 times does not appear in data setP_reduced<P_incrementalSituation, continue duplicate removal and detect;

(5) no matter heavily whether delete, backup tasks can as usual be carried out nature and finishes or be terminated.

2. the data back up method of data-oriented according to claim 1 center based on energy consumption perception, its feature existsIn: the number of times of the data de-duplication process in described step (1) is less than 3.

3. the data back up method of data-oriented according to claim 1 center based on energy consumption perception, is characterized in that:P in described step (2)_incrementalComputing formula as follows:

P_incremental＝∑_nodes((P_dedup-P_standby)*T_dedup(1.1)

Wherein, T_dedupFor the time that data de-duplication expends thereon, (P_dedup-P_standby) represent, at thisIn the section time, owing to carrying out data de-duplication process, the part that server power increases, P_dedupRepresent to carry out repetitionThe power of memory device after data delete procedure, P_standbyThe memory device power under idle condition.

4. the data back up method of data-oriented according to claim 3 center based on energy consumption perception, is characterized in that:Described (P_dedup-P_standby) or by (P_dedup-P_standby) be considered as the mean value that server power increases, and pass through energyConsumption model:

P_active＝P_standby+P_seq+P_rand(1.2)

Resource utilization situation with data de-duplication process, calculates, and wherein Pseq order of representation IO producesEnergy consumption, Prand represents the energy consumption that random IO produces, they are the relevant energy consumption of the load bringing due to IO expense.

5. the data back up method of data-oriented according to claim 1 center based on energy consumption perception, is characterized in that:P in described step (1)_reducedComputing formula as follows:

$P_{reduced}=(P_{MaxSeqIO}-P_{s\tan dby})*T_{iosave}*N_{active}+(P_{s\tan dby}-P_{idle})*T_{backup}*\frac{S_{dup}}{S_{energy\_unit}}---\left(1.3\right)$

(P_MaxSeqIO-P_standby) be the poor of memory device power under transmission state and under idle condition,P_MaxSeqIORepresent the memory device power under transmission state, P_standbyRepresent the memory device under idle conditionPower; T_iosaveRepresent the transmission time reducing due to data volume, record in actual experiment process; N_activeBe illustrated in the quantity of the memory device that transmitting procedure relates to; (P_standby-P_idle) represent under standby and in low meritMemory device power poor under consumption or closed condition, P_idleThe storage of expression under low-power consumption or closed condition establishedStandby power; T_backupRepresent the execution cycle of backup tasks, elapsed time before once carrying out on backup tasks is logicalThe configuration information of crossing backup tasks obtains;Expression can be by the quantity of idle equipment, wherein S_{energy_unit}Be equal toIn the capacity of the manipulable minimum memory of storage system unit, S_dupExpression can be by the capacity of idle equipment,S_{energy_unit}Be equal to the capacity of the manipulable minimum memory of storage system unit.