CN103761167A - Method and device for achieving data center backup - Google Patents

Abstract

The invention provides a method and device for achieving data center backup. The method for achieving data center backup comprises the steps that according to the name of a table where a data block to be backed up is located and the name of a column family where the data block is located, the storage range of the data block in a target data center is determined; the data block is stored in one data node selected from the determined storage range. The method and device for achieving data center backup solve the problem that when Hbase data backup across data centers is conducted, the data blocks with the same column family name are stored in a dispersed mode, so that storage of the data blocks which are backed up to the target data center and have the same column family name is concentrated better, and therefore the reading speed is improved.

Description

A kind of method and apparatus of realizing data center's backup

Technical field

The present invention relates to large data field, relate in particular to a kind of backup method and device of the data center based on Hbase.

Background technology

The data storage of Hbase is the storage of the distributed document based on Hadoop (HDFS normally, Hadoop Distributed File System), while carrying out data storage in HDFS Yuan data center, conventionally to back up, three parts of acquiescence backups, wherein, two parts of backups are in belonging to two different back end of same machine frame, and a backup is in belonging to a back end of other frames different from above-mentioned frame in addition., when guaranteeing that data center breaks down, still can normally work meanwhile, need to back up data center.

The existing method that realizes data center's backup comprises:

Obtain data block to be backed up in former data center; In target data, select at random 1 back end to data block backup in the heart in target data in the heart, then according to existing backup method, select two other back end to back up.

Above-mentioned date storage method is due to when carrying out data center's backup, 1 back end at target data center is random selection, and the storage of the data of Hbase is stored based on row family data, in Ji Yuan data center, by belonging in form in the set of data blocks of same row family data, be stored in same back end or contiguous several back end, like this when carrying out data while reading, need to search all back end at this row Praenomen place according to the data column Praenomen reading, the back end finding is likely distributed in all back end at target data center.While making to adopt said method to carry out Hbase across data center's data backup, can not make full use of the feature of row family storage, cause in target data in the heart, the problem such as the storage of the data block of same row Praenomen disperses, discontinuous, thus cause reading speed slower.

Summary of the invention

In order to solve the problems of the technologies described above, the present invention proposes a kind of backup method and device of data center, can make full use of the feature of row family storage, the data block storage of same row Praenomen that makes to backup to destination data center is more concentrated, thereby improves reading speed.

In order to achieve the above object, the present invention proposes a kind of method that realizes data center's backup, comprising:

According to data block place table name to be backed up and row Praenomen, determine this data block memory range in the heart in target data;

In definite memory range, choose a back end storage data block.

Preferably, the described data block place table name backing up as required and row Praenomen, determine that this data block memory range in the heart in target data comprises:

According to described table name, determine the scope of the back end place frame of described data block storage;

According to described row Praenomen, determine the range of physical addresses of described back end;

Described one of them back end storage data block of choosing from memory range comprises:

From the scope of described frame, select a frame, from described range of physical addresses, select a physical address.

Preferably, describedly according to described table name, determine that the scope of the back end place frame of described data block storage comprises:

Calculate the cryptographic hash of described table name, and calculate respectively in target data the cryptographic hash of all frames in the heart;

The scope of determining the cryptographic hash of described frame is: the absolute value of the difference between the cryptographic hash of described frame and the cryptographic hash of described table name is less than or equal in target data the peaked preset ratio of the cryptographic hash of all frames in the heart;

Wherein, in former data center the maximal value of the cryptographic hash of all table names with in target data in the heart the maximal value of the cryptographic hash of all frames equate.

Preferably, described one of them back end storage data block of choosing from memory range comprises:

A frame of random selection from the scope of described frame, or the selection frame corresponding with the cryptographic hash of the absolute value minimum of the difference of the cryptographic hash of described table name.

Preferably, describedly according to described row Praenomen, determine that the range of physical addresses of described back end comprises:

Calculate the cryptographic hash of described row Praenomen, and calculate respectively the cryptographic hash of the physical address of all back end in selected frame;

The scope of determining the cryptographic hash of described physical address is: the absolute value of the difference between the cryptographic hash of described physical address and the cryptographic hash of described row Praenomen is less than or equal to the peaked preset ratio of the cryptographic hash of the physical address of all back end in selected frame; Wherein, in the table that described table name is corresponding, the maximal value of the cryptographic hash of all row Praenomens equates with the maximal value of the cryptographic hash of the physical address of all back end in selected frame.

Preferably, described one of them back end storage data block of choosing from memory range comprises:

A back end that physical address is corresponding of random selection from described range of physical addresses, or selection back end corresponding to physical address corresponding with the cryptographic hash of the absolute value minimum of the difference of the cryptographic hash of described row Praenomen.

Preferably, described maximal value is 2 π.

The invention allows for a kind of device of realizing data center's backup, at least comprise:

Determination module, for according to data block place table name to be backed up and row Praenomen, determines this data block memory range in the heart in target data;

Select module, for choose a back end storage data block at definite memory range.

Preferably, described determination module, specifically for:

According to described table name, determine the scope of the back end place frame of described data block storage; According to described row Praenomen, determine the range of physical addresses of described back end;

Described selection module, specifically for:

From the scope of described frame, select a frame, from described range of physical addresses, select a physical address.

Preferably, described determination module, specifically for:

Calculate the cryptographic hash of described table name, and calculate respectively in target data the cryptographic hash of all frames in the heart;

The scope of determining the cryptographic hash of described frame is: the absolute value of the difference between the cryptographic hash of described frame and the cryptographic hash of described table name is less than or equal in target data the peaked preset ratio of the cryptographic hash of all frames in the heart;

Wherein, in former data center the maximal value of the cryptographic hash of all table names with in target data in the heart the maximal value of the cryptographic hash of all frames equate.

Preferably, described determination module, specifically for:

Calculate the cryptographic hash of described row Praenomen, and calculate respectively the cryptographic hash of the physical address of all back end in selected frame;

The scope of determining the cryptographic hash of described physical address is: the absolute value of the difference between the cryptographic hash of described physical address and the cryptographic hash of described row Praenomen is less than or equal to the peaked preset ratio of the cryptographic hash of the physical address of all back end in selected frame; Wherein, in the table that described table name is corresponding, the maximal value of the cryptographic hash of all row Praenomens equates with the maximal value of the cryptographic hash of the physical address of all back end in selected frame.

Compared with prior art, the present invention includes: according to data block place table name to be backed up and row Praenomen, determine this data block memory range in the heart in target data; In definite memory range, choose a back end storage data block.By technical scheme of the present invention, take full advantage of the feature of row family storage, the problem that while having solved Hbase across data center's data backup, the storage of the data block of same row Praenomen disperses, the data block storage of same row Praenomen that makes to backup to target data center is more concentrated, thereby has improved reading speed.

Accompanying drawing explanation

Below the accompanying drawing in the embodiment of the present invention is described, the accompanying drawing in embodiment is for a further understanding of the present invention, is used from explanation the present invention with instructions one, does not form limiting the scope of the invention.

Fig. 1 is the process flow diagram of realizing the method for data center's backup of the present invention;

Fig. 2 is the process flow diagram of the embodiment of the method that realizes data center backup of the present invention;

Fig. 3 is that the structure that realizes the device of data center's backup of the present invention forms schematic diagram.

Embodiment

For the ease of those skilled in the art's understanding, below in conjunction with accompanying drawing, the invention will be further described, can not be used for limiting the scope of the invention.

Referring to Fig. 1, the present invention proposes a kind of method that realizes data center's backup, comprising:

Step 100, according to data block place table name to be backed up and row Praenomen, determine this data block memory range in the heart in target data.

In this step, can be according to the scope of the back end place frame of table name specified data piece storage; According to the range of physical addresses of row Praenomen specified data node.Wherein,

According to the scope of the back end place frame of table name specified data piece storage, comprise:

Calculate the cryptographic hash of table name, and calculate respectively in target data the cryptographic hash of all frames in the heart; The scope of determining the cryptographic hash of frame is: the absolute value of the difference between the cryptographic hash of frame and the cryptographic hash of table name is less than or equal in target data the peaked preset ratio of the cryptographic hash of all frames in the heart; Wherein, in former data center the maximal value of the cryptographic hash of all table names with in target data in the heart the maximal value of the cryptographic hash of all frames equate.

According to the range of physical addresses of row Praenomen specified data node, comprise:

The cryptographic hash of calculated column Praenomen, and calculate respectively the cryptographic hash of the physical address of all back end in selected frame; The scope of determining the cryptographic hash of physical address is: the absolute value of the difference between the cryptographic hash of physical address and the cryptographic hash of row Praenomen is less than or equal to the peaked preset ratio of the cryptographic hash of the physical address of all back end in selected frame; Wherein, in the table that table name is corresponding, the maximal value of the cryptographic hash of all row Praenomens equates with the maximal value of the cryptographic hash of the physical address of all back end in selected frame.

Wherein, it is 2 π that maximal value can be, but not limited to, and in table corresponding to the maximal value of the cryptographic hash of all table names and table name, the maximal value of the cryptographic hash of all row Praenomens can equate, also can be unequal.

Step 101, in definite memory range, choose back end storage data block.

In this step, a frame can be selected from the scope of frame at random, a physical address of random selection from range of physical addresses.The back end that selected frame and physical address are corresponding is the memory location of data block, after data block storage, then selects two outer two back end to back up according to existing backup method.

In this step, also can from the scope of frame, select the frame corresponding with the cryptographic hash of the absolute value minimum of the difference of the cryptographic hash of table name, from range of physical addresses, select back end corresponding to physical address corresponding with the cryptographic hash of the absolute value minimum of the difference of the cryptographic hash of row Praenomen.

The present invention is stored in the data block of same table name and same row Praenomen in identical memory range, makes the database storage of same row Praenomen comparatively concentrated, thereby has improved reading speed.

Below by a specific embodiment explanation How to choose back end, back up.

Step 200, obtain data block place table name to be backed up and column Praenomen.

In this step, how to obtain database place table name to be backed up and column Praenomen and belong to prior art, can not be used for limiting the scope of the invention.For example, when data block is upgraded, frame can record more table name and the row Praenomen of new data block, when backing up, just can directly obtain table name and the row Praenomen of data block to be backed up.

Step 201, set up spherical coordinate system, the data block place table name of acquisition and column Praenomen are mapped on the sphere of radii fixus in spherical coordinate system.

In this step, a bit representing with two coordinate figures on sphere: the projection of line in XoY plane and the angle of X-axis at the point on the point on sphere and the line at coordinate center and the angle theta of Z axis, sphere and coordinate center

In this step, the point on employing sphere represents the position in data block Yuan data center, gets place table name and is

getting column Praenomen is θ.

Wherein,

can be the cryptographic hash of table name, θ can be the cryptographic hash of row Praenomen.How to calculate each cryptographic hash and can adopt existing method to realize, such as, adopt existing hash algorithm to calculate, the protection domain that its specific implementation is not intended to limit the present invention, repeats no more here.

Step 202, take that to be mapped to point on sphere be axle to the line at coordinate center, got coordinate center and became the straight line of predetermined angle with axle, the sphere intercepting for a week that pivots is put the storage area of corresponding data block for this.

In this step, suppose to take that the some P of data on sphere and the line OP of coordinate center O are axle, got O point and with the angle of the OP straight line that is γ around the rotate a circle area of the sphere that the conical region that obtains intercepts of OP, be the corresponding (cryptographic hash of table name, the cryptographic hash of row Praenomen) corresponding storage area, by conical region projection in YoZ plane, the scope that obtains θ ' is θ ' ∈ [θ-γ, θ+γ]; By conical region projection in XoY plane, obtain formula (1):

$\left\{\begin{array}{c}\tan \mathrm{\γ}=\frac{b}{r}\\ \tan {\mathrm{\γ}}^{\′}=\frac{b}{r\sin \mathrm{\θ}}\end{array}\right.---\left(1\right)$

Obtain

so obtain

scope as shown in formula (2):

Wherein, b be O point and with the intersection point of the angle of the OP straight line that is γ and the sphere distance to OP, the length that r is OP, γ ' for got O point and with the angle of the angle of the OP straight line projection in XoY plane XoY plane upslide movie queen and OP that is γ.

The ratio of the area of the sphere that conical region intercepts and the total area of sphere is as shown in Equation (3):

Obtain the value of γ ' as shown in Equation (4):

${\mathrm{\γ}}^{\′}=\frac{\mathrm{\πRatio}}{2\sin \mathrm{\θ}\sin \mathrm{\γ}}---\left(4\right)$

Meet θ ' scope and

the region of scope be the storage area of data block.

Step 203, by the physical address map of the frame that in target's center, all back end are corresponding and back end to the sphere of radii fixus in spherical coordinate system.

In this step, adopt point on sphere to represent back end position in the heart in target data, get frame and be

getting physical address is θ.

Wherein, can be the cryptographic hash of frame, θ can be the cryptographic hash of physical address.How to calculate each cryptographic hash and belong to prior art, can not be used for limiting the scope of the invention.

Step 204, from the storage area of data block, select one of them to put corresponding back end as the memory location of data block.

In this step, back end corresponding to the nearest point of can chosen distance data block corresponding point is as the memory location of data block.

Referring to Fig. 3, the invention allows for a kind of device of realizing data center's backup, at least comprise:

Determination module, for according to data block place table name to be backed up and row Praenomen, determines this data block memory range in the heart in target data;

Select module, for choose a back end storage data block at definite memory range.

In device of the present invention, determination module, specifically for:

According to the scope of the back end place frame of table name specified data piece storage; According to the range of physical addresses of row Praenomen specified data node;

Select module, specifically for:

From the scope of frame, select a frame, from range of physical addresses, select a physical address.

In device of the present invention, determination module, specifically for:

Calculate the cryptographic hash of described table name, and calculate respectively in target data the cryptographic hash of all frames in the heart;

The scope of determining the cryptographic hash of frame is: the absolute value of the difference between the cryptographic hash of frame and the cryptographic hash of table name is less than or equal in target data the peaked preset ratio of the cryptographic hash of all frames in the heart;

Wherein, in former data center the maximal value of the cryptographic hash of all table names with in target data in the heart the maximal value of the cryptographic hash of all frames equate.

In back-up device of the present invention, determination module, specifically for:

Calculate the cryptographic hash of described row Praenomen, and calculate respectively the cryptographic hash of the physical address of all back end in selected frame;

The scope of determining the cryptographic hash of physical address is: the absolute value of the difference between the cryptographic hash of physical address and the cryptographic hash of row Praenomen is less than or equal to the peaked preset ratio of the cryptographic hash of the physical address of all back end in selected frame; Wherein, in the table that table name is corresponding, the maximal value of the cryptographic hash of all row Praenomens equates with the maximal value of the cryptographic hash of the physical address of all back end in selected frame.

It should be noted that; above-described embodiment understands for the ease of those skilled in the art; be not limited to protection scope of the present invention; do not departing under the prerequisite of inventive concept of the present invention, any apparent replacement that those skilled in the art make the present invention and improvement etc. are all within protection scope of the present invention.

Claims (11)

1. a method that realizes data center's backup, is characterized in that, comprising:

According to data block place table name to be backed up and row Praenomen, determine this data block memory range in the heart in target data;

In definite memory range, choose a back end storage data block.

2. method according to claim 1, is characterized in that, the described data block place table name backing up as required and row Praenomen determine that this data block memory range in the heart in target data comprises:

According to described table name, determine the scope of the back end place frame of described data block storage;

According to described row Praenomen, determine the range of physical addresses of described back end;

Described one of them back end storage data block of choosing from memory range comprises:

From the scope of described frame, select a frame, from described range of physical addresses, select a physical address.

3. method according to claim 2, is characterized in that, describedly according to described table name, determines that the scope of the back end place frame of described data block storage comprises:

Calculate the cryptographic hash of described table name, and calculate respectively in target data the cryptographic hash of all frames in the heart;

The scope of determining the cryptographic hash of described frame is: the absolute value of the difference between the cryptographic hash of described frame and the cryptographic hash of described table name is less than or equal in target data the peaked preset ratio of the cryptographic hash of all frames in the heart;

Wherein, in former data center the maximal value of the cryptographic hash of all table names with in target data in the heart the maximal value of the cryptographic hash of all frames equate.

4. method according to claim 3, is characterized in that, described one of them back end storage data block of choosing from memory range comprises:

A frame of random selection from the scope of described frame, or the selection frame corresponding with the cryptographic hash of the absolute value minimum of the difference of the cryptographic hash of described table name.

5. method according to claim 3, is characterized in that, describedly according to described row Praenomen, determines that the range of physical addresses of described back end comprises:

Calculate the cryptographic hash of described row Praenomen, and calculate respectively the cryptographic hash of the physical address of all back end in selected frame;

The scope of determining the cryptographic hash of described physical address is: the absolute value of the difference between the cryptographic hash of described physical address and the cryptographic hash of described row Praenomen is less than or equal to the peaked preset ratio of the cryptographic hash of the physical address of all back end in selected frame; Wherein, in the table that described table name is corresponding, the maximal value of the cryptographic hash of all row Praenomens equates with the maximal value of the cryptographic hash of the physical address of all back end in selected frame.

6. method according to claim 5, is characterized in that, described one of them back end storage data block of choosing from memory range comprises:

A back end that physical address is corresponding of random selection from described range of physical addresses, or selection back end corresponding to physical address corresponding with the cryptographic hash of the absolute value minimum of the difference of the cryptographic hash of described row Praenomen.

7. according to the method described in claim 3～6 any one, described maximal value is 2 π.

8. a device of realizing data center's backup, is characterized in that, at least comprises:

Determination module, for according to data block place table name to be backed up and row Praenomen, determines this data block memory range in the heart in target data;

Select module, for choose a back end storage data block at definite memory range.

9. device according to claim 8, is characterized in that, described determination module, specifically for:

According to described table name, determine the scope of the back end place frame of described data block storage; According to described row Praenomen, determine the range of physical addresses of described back end;

Described selection module, specifically for:

From the scope of described frame, select a frame, from described range of physical addresses, select a physical address.

10. device according to claim 9, is characterized in that, described determination module, specifically for:

Calculate the cryptographic hash of described table name, and calculate respectively in target data the cryptographic hash of all frames in the heart;

The scope of determining the cryptographic hash of described frame is: the absolute value of the difference between the cryptographic hash of described frame and the cryptographic hash of described table name is less than or equal in target data the peaked preset ratio of the cryptographic hash of all frames in the heart;

Wherein, in former data center the maximal value of the cryptographic hash of all table names with in target data in the heart the maximal value of the cryptographic hash of all frames equate.

11. devices according to claim 10, is characterized in that, described determination module, specifically for:

Calculate the cryptographic hash of described row Praenomen, and calculate respectively the cryptographic hash of the physical address of all back end in selected frame;

The scope of determining the cryptographic hash of described physical address is: the absolute value of the difference between the cryptographic hash of described physical address and the cryptographic hash of described row Praenomen is less than or equal to the peaked preset ratio of the cryptographic hash of the physical address of all back end in selected frame; Wherein, in the table that described table name is corresponding, the maximal value of the cryptographic hash of all row Praenomens equates with the maximal value of the cryptographic hash of the physical address of all back end in selected frame.

Images