CN103716384A - Method and device for realizing cloud storage data synchronization in cross-data-center manner - Google Patents
Method and device for realizing cloud storage data synchronization in cross-data-center manner Download PDFInfo
- Publication number
- CN103716384A CN103716384A CN201310693744.1A CN201310693744A CN103716384A CN 103716384 A CN103716384 A CN 103716384A CN 201310693744 A CN201310693744 A CN 201310693744A CN 103716384 A CN103716384 A CN 103716384A
- Authority
- CN
- China
- Prior art keywords
- unit
- cloud storage
- data
- storage system
- metadata node
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application provides a method and device for realizing cloud storage data synchronization in a cross-data-center manner. In the invention, a cloud storage system is established in a source data center and a target data center respectively, and the maximum number of per data unit is acquired from a metadata node local of a source cloud storage system and the metadata node local of a target cloud storage system respectively. The maximum numbers of per data unit acquired from the metadata node local of the source cloud storage system and the metadata node local of the target cloud storage system are compared, and when the former is greater than the latter, per data unit meeting the following conditions is acquired from the metadata node local of the source cloud storage system: when the number is greater than the one acquired from the metadata node local of the target cloud storage system, data synchronization is performed on the acquired per data unit between the source data center and the target data center. Therefore, cloud storage data synchronization in the cross-data-center manner is realized and high reliability of cloud storage is guaranteed.
Description
Technical field
The application relates to computer memory technical, particularly across data center, realizes the synchronous method and apparatus of cloud storage data.
Background technology
Cloud is stored in cloud computing (Cloud Computing) conceptive extension and is developed out.Cloud computing is the development of distributed treatment (Distributed Computing), parallel processing (Parallel Computing) and grid computing (Grid Computing), be to see through network huge computing program is split into numerous less subprogram automatically, then transfer to bulky systems that multi-section server forms result to be returned to user after analyzing as calculated.By cloud computing technology, Internet Service Provider can be within the several seconds, processes number in hundred million information necessarily even, reach and " supercomputer " same powerful network service.
Concept and the cloud computing of cloud storage are similar, it refers to by functions such as cluster application, grid or distributed file systems, a large amount of various dissimilar memory devices in network are gathered to collaborative work by application software, the common system that data storage and Operational Visit function are externally provided, guarantee the fail safe of data, and save storage sky.
At present, cloud is stored a kind of trend that has become storage development in future, and still, the development trend of following cloud storage, mainly still develops from fail safe, reliability, practicality equal angles.It is synchronous that Er Kua data center realizes cloud storage data, met the requirement that cloud is stored in high reliability, and it breaks down such as power down is even in the situation of the disaster such as massive blackout Yi data center, still can recover from another data center.
Therefore, across data center, realizing the synchronous method of cloud storage data is current technical problem urgently to be resolved hurrily.
Summary of the invention
The application provides Liao Kua data center to realize the synchronous method and apparatus of cloud storage data, and it is synchronous that Yi Kua data center realizes cloud storage data, guarantees the high reliability of cloud storage.
The technical scheme that the application provides comprises:
Across data center, realize the synchronous method of cloud storage data, comprising:
Respectively at source data center, cloud storage system is set up at destination data center, described source data center, two of offices, destination data center diverse location, at source data center, the cloud storage system that destination data center is set up all comprises metadata node, the cloud storage system of setting up at described source data center, be called source cloud storage system, for cloud stores service is externally provided, when receiving Data Update request, related data corresponding to described Data Update request is organized into a unit data unit, the mode increasing progressively according to number order is that described unit data unit distributes numbering, and described unit data unit and the numbering that is assigned with are stored in to this locality, the cloud storage system of setting up at described destination data center, is called object cloud storage system, for backup source cloud storage system,
From metadata node metadata node this locality local and described object cloud storage system of described source cloud storage system, obtain respectively the maximum numbering of unit data unit, wherein, if metadata node this locality of object cloud storage system is storage cell data cell not, the maximum of giving tacit consent to the local unit data of the metadata node unit of object cloud storage system is numbered set point;
Relatively from metadata node metadata node this locality local and described object cloud storage system of described source cloud storage system, obtain the maximum numbering of unit data unit,
If the former equals the latter, wait for setting-up time, return to afterwards respectively the operation that the maximum of obtaining unit data unit from metadata node metadata node this locality local and described object cloud storage system of described source cloud storage system is numbered;
If the former is less than the latter, send alarm prompt;
If the former is greater than the latter, from metadata node this locality of described source cloud storage system, obtain the unit data unit meeting the following conditions: numbering is greater than the numbering of obtaining from metadata node this locality of described object cloud storage system, and the unit data unit executing data between described source data center, destination data center, this being obtained is synchronous.
A kind ofly across data center, realize the synchronous device of cloud storage data, this device is used at source data center, between destination data center, carry out data synchronous, wherein, described source data center, two of offices, destination data center diverse location, and all set up cloud storage system, at source data center, the cloud storage system that destination data center is set up all comprises metadata node, the cloud storage system of setting up at described source data center, be called source cloud storage system, for cloud stores service is externally provided, when receiving Data Update request, related data corresponding to described Data Update request is organized into a unit data unit, the mode increasing progressively according to number order is that described unit data unit distributes numbering, and described unit data unit and the numbering that is assigned with are stored in to this locality, the cloud storage system of setting up at described destination data center, is called object cloud storage system, for backup source cloud storage system,
This device comprises:
Acquiring unit, for obtain respectively the maximum numbering of unit data unit from metadata node metadata node this locality local and described object cloud storage system of described source cloud storage system, wherein, if metadata node this locality of object cloud storage system is storage cell data cell not, the maximum of giving tacit consent to the local unit data of the metadata node unit of object cloud storage system is numbered set point;
Comparing unit, for relatively obtain the maximum numbering of unit data unit from metadata node metadata node this locality local and described object cloud storage system of described source cloud storage system,
If the former equals the latter, wait for setting-up time, return to afterwards respectively the operation that the maximum of obtaining unit data unit from metadata node metadata node this locality local and described object cloud storage system of described source cloud storage system is numbered;
If the former is less than the latter, send alarm prompt;
If the former is greater than the latter, send notice to data synchronisation unit;
Described data synchronisation unit, for obtaining from metadata node this locality of described source cloud storage system the unit data unit meeting the following conditions: numbering is greater than the numbering of obtaining from metadata node this locality of described object cloud storage system, and the unit data unit executing data between described source data center, destination data center, this being obtained is synchronous.
As can be seen from the above technical solutions, in the present invention, by respectively at source data center, cloud storage system is set up at destination data center, from metadata node metadata node this locality local and described object cloud storage system of described source cloud storage system, obtain respectively the maximum numbering of unit data unit, relatively from metadata node metadata node this locality local and described object cloud storage system of described source cloud storage system, obtain the maximum numbering of unit data unit, when the former is greater than the latter, from metadata node this locality of described source cloud storage system, obtain the unit data unit meeting the following conditions: numbering is greater than the numbering of obtaining from metadata node this locality of described object cloud storage system, at described source data center, the unit data unit executing data between destination data center, this being obtained is synchronous.This realizes Liao Kua data center, and to realize cloud storage data synchronous, guarantees the high reliability of cloud storage.
Accompanying drawing explanation
The method flow diagram that Fig. 1 provides for the embodiment of the present invention;
The step 104 realization flow figure that Fig. 2 provides for the embodiment of the present invention;
The synchronous flow chart of unit data unit executing data between described source data center, destination data center, this being obtained that Fig. 3 provides for the embodiment of the present invention;
The structure drawing of device that Fig. 4 provides for the embodiment of the present invention.
Embodiment
In order to make the object, technical solutions and advantages of the present invention clearer, below in conjunction with the drawings and specific embodiments, describe the present invention.
The invention provides across data center and realize the synchronous method of cloud storage data, below by Fig. 1, be described:
Referring to Fig. 1, the method flow diagram that Fig. 1 provides for the embodiment of the present invention.As shown in Figure 1, this flow process can comprise the following steps:
Here, source data center, two of offices, destination data center diverse location, two places are apart from possibility hundreds of kilometer, even thousands of kilometers.
In the present invention, at source data center, destination data center sets up cloud storage system, its structure is similar, mainly comprise: metadata node and Chunk node, wherein, metadata node is used for managing cloud storage system, mainly stored the index information of some files, and Chunk node has been stored the particular content of file.
In the present invention, for ease of describing, by the cloud storage system of setting up at described source data center, be called source cloud storage system, this source cloud storage system externally provides cloud stores service under normal circumstances, be specially: when receiving Data Update request, related data corresponding to described Data Update request is organized into a unit data unit, the mode increasing progressively according to number order is that described unit data unit distributes numbering, and described unit data unit and the numbering that is assigned with are stored in to this locality; Here, described Data Update request generally comprises: write request and/or removal request.Such as, when the metadata node of source cloud storage system receives write request, related data corresponding to this write request is organized into a unit data unit (being called metalog), the mode increasing progressively according to number order is that described unit data unit distributes numbering, such as, before not receiving this write request, the maximum that local unit data unit has distributed is numbered M, and when receiving write request, related data corresponding to this write request is organized into a unit data unit, for distributing numbering M+1 in this unit data unit.Afterwards, described unit data unit and the numbering that is assigned with are stored in to this locality.
As one embodiment of the present of invention, related data corresponding to described Data Update request comprises but is not limited to:
Data Identification, described Data Identification is: the sign of file or file;
Action type; Wherein, described action type comprises: revise, delete or create.
Wherein, during the sign that is file at Data Identification, related data corresponding to described Data Update request also comprises the routing information that the content in file is stored at source cloud storage service system.
In the present invention, for ease of describing, the cloud storage system by setting up at described destination data center, is called object cloud storage system, for backup source cloud storage system.
As one embodiment of the present of invention, the maximum numbering that in step 102, metadata node this locality local from the metadata node of described source cloud storage system and described object cloud storage system obtains unit data unit is respectively that at destination data center, to set up object cloud storage system be to carry out setting-up time foundation after source cloud storage system is set up at source data center and described source cloud storage system complete Data Update according to Data Update request.Here, described setting-up time is that source cloud storage system and object cloud storage system are set up the time of being separated by.
And if the object cloud storage system that destination data center is set up is set up after exceeding described setting-up time; can not perform step 102; directly source cloud storage system is shut down, and synchronous according to the mode of batch synchronization executing data between source data center, destination data center.Reason is: the object cloud storage system that destination data center is set up is set up after exceeding described setting-up time; illustrate that source cloud storage system is by may having stored a large amount of unit datas unit to current; this can cause last between described source data center, destination data center the synchronous unit data unit of executing data many; for saving lock in time; source cloud storage system is shut down, and synchronous according to the mode of batch synchronization executing data between source data center, destination data center.
As one embodiment of the present of invention, preferably, in this step 104, described metadata node this locality from described source cloud storage system obtains numbering that metadata node metadata node this locality local and described object cloud storage system of Shi Cong source, the unit data unit cloud storage system satisfying condition obtains and differs and carry out the prerequisite within setting range.
And if the numbering that metadata node this locality of and described object cloud storage system local from the metadata node of source cloud storage system obtains differs over setting range; do not perform step 104; but source cloud storage system is shut down, and synchronous according to the mode of batch synchronization executing data between source data center, destination data center.Reason is: if the numbering that metadata node this locality of and described object cloud storage system local from the metadata node of source cloud storage system obtains differs over setting range; explanation last between described source data center, destination data center the synchronous unit data unit of executing data many; for saving lock in time; source cloud storage system is shut down, and synchronous according to the mode of batch synchronization executing data between source data center, destination data center.
So far, complete flow process shown in Fig. 1.Preferably, in flow process shown in Fig. 1 step 104 complete data synchronous after, can also further this be participated in to the number record of unit data unit of synchronous maximum numbering at object cloud storage server, so that the numbering of the maximum in real time inspection destination server.
In flow process shown in Fig. 1, as a preferred embodiment of the present invention, metadata node this locality from source cloud storage system described in step 104 obtains the unit data unit meeting the following conditions: numbering is greater than the numbering of obtaining from metadata node this locality of described object cloud storage system, and the unit data unit executing data between described source data center, destination data center, this being obtained synchronously can pass through shown in Fig. 2:
Referring to Fig. 2, the step 104 realization flow figure that Fig. 2 provides for the embodiment of the present invention.As shown in Figure 2, this flow process can comprise the following steps:
Here, N can be according to actual conditions setting, such as being set to 50.
This step 204 is to carry out under the prerequisite of the unit data element number that identifies at this batch unit N that is greater than setting.Take N as 50 being example, in this step 204, if that obtains from metadata node this locality of described object cloud storage system is numbered 50, that from metadata node this locality of described source cloud storage system, obtains is numbered 200, first from metadata node this locality of source cloud storage system, obtain the unit data unit that is numbered 51 to 100, the unit data unit executing data between described source data center, destination data center, this being obtained is synchronous; Afterwards, from metadata node this locality of source cloud storage system, obtain the unit data unit that is numbered 101 to 150 again, the unit data unit executing data between described source data center, destination data center, this being obtained is synchronous, finally, from metadata node this locality of source cloud storage system, obtain the unit data unit that is numbered 151 to 200, the unit data unit executing data between described source data center, destination data center, this being obtained is synchronous.By this in batches synchronous, can avoid when data are synchronous, occurring between source data center, destination data center network congestion.
So far, complete flow process shown in Fig. 2.
Wherein, in above-mentioned steps 203, step 204, the unit data unit executing data between described source data center, destination data center, this being obtained synchronously can be realized by flow process shown in Fig. 3:
Referring to Fig. 3, the synchronous flow chart of unit data unit executing data between described source data center, destination data center, this being obtained that Fig. 3 provides for the embodiment of the present invention.As shown in Figure 3, this flow process can comprise the following steps:
So far, complete flow process shown in Fig. 3.By flow process shown in Fig. 3, can realize that across data center, to realize cloud storage data synchronous, guarantee the high reliability of cloud storage.
It should be noted that, in the present invention, before the unit data unit executing data between source data center, destination data center, this being obtained is synchronous, the method further comprises:
Dedicated bandwidth based on setting is set up network and is connected between source data center, destination data center,
If network connects, be not successfully established, attempt set point number, after attempting set point number, still the unsuccessful network of successfully setting up between source data center, destination data center connects, and sends alarm prompt.
Above method provided by the invention is described, below device provided by the invention is described:
Referring to Fig. 4, the structure drawing of device that Fig. 4 provides for the embodiment of the present invention.This device is used at source data center, between destination data center, carry out data synchronous, wherein, described source data center, two of offices, destination data center diverse location, and all set up cloud storage system, at source data center, the cloud storage system that destination data center is set up all comprises metadata node, the cloud storage system of setting up at described source data center, be called source cloud storage system, for cloud stores service is externally provided, when receiving Data Update request, related data corresponding to described Data Update request is organized into a unit data unit, the mode increasing progressively according to number order is that described unit data unit distributes numbering, and described unit data unit and the numbering that is assigned with are stored in to this locality, the cloud storage system of setting up at described destination data center, is called object cloud storage system, for backup source cloud storage system,
As shown in Figure 4, this device comprises:
Acquiring unit, for obtain respectively the maximum numbering of unit data unit from metadata node metadata node this locality local and described object cloud storage system of described source cloud storage system, wherein, if metadata node this locality of object cloud storage system is storage cell data cell not, the maximum of giving tacit consent to the local unit data of the metadata node unit of object cloud storage system is numbered set point;
Comparing unit, for relatively obtain the maximum numbering of unit data unit from metadata node metadata node this locality local and described object cloud storage system of described source cloud storage system,
If the former equals the latter, wait for setting-up time, return to afterwards respectively the operation that the maximum of obtaining unit data unit from metadata node metadata node this locality local and described object cloud storage system of described source cloud storage system is numbered;
If the former is less than the latter, send alarm prompt;
If the former is greater than the latter, send notice to data synchronisation unit;
Described data synchronisation unit, for obtaining from metadata node this locality of described source cloud storage system the unit data unit meeting the following conditions: numbering is greater than the numbering of obtaining from metadata node this locality of described object cloud storage system, and the unit data unit executing data between described source data center, destination data center, this being obtained is synchronous.
Preferably, in the present invention, related data corresponding to described Data Update request comprises but is not limited to:
Data Identification, described Data Identification is: the sign of file or file;
During the sign that is file at Data Identification, the content in file is at the routing information of source cloud storage service system storage; During the sign that is file at Data Identification, file is at the routing information of source cloud storage service system storage;
Action type; Wherein, described action type comprises: revise, delete or create;
There is the timestamp of operation.
Preferably, in the present invention, described acquiring unit obtains from metadata node this locality of source cloud storage system the unit data unit meeting the following conditions: numbering is greater than the numbering of obtaining from metadata node this locality of described object cloud storage system, and the unit data unit executing data between described source data center, destination data center, this being obtained synchronously comprises:
Identify the local unit data element number meeting the following conditions existing of metadata node of described source cloud storage system: numbering is greater than the numbering of obtaining from metadata node this locality of described object cloud storage system;
The batch unit N of this unit data element number identifying and setting relatively;
If the former is less than or equal to the latter, directly from metadata node this locality of source cloud storage system, obtain the unit data unit meeting the following conditions: numbering is greater than the numbering of obtaining from metadata node this locality of described object cloud storage system, and the unit data unit executing data between described source data center, destination data center, this being obtained is synchronous;
If the former is greater than the latter, first from metadata node this locality of source cloud storage system, obtain N the unit data unit meeting the following conditions: numbering is greater than the numbering of obtaining from metadata node this locality of described object cloud storage system, and the unit data unit executing data between described source data center, destination data center, this being obtained is synchronous; Afterwards the maximum numbering in the unit data unit obtaining is numbered as candidate, identify the local unit data element number meeting the following conditions existing of metadata node of described source cloud storage system: numbering is greater than candidate's numbering, return to the relatively operation of the batch unit N of this unit data element number identifying and setting.
Preferably, in the present invention, the unit data unit executing data that described data synchronisation unit is obtained this between source data center, destination data center synchronously comprises:
For the per unit data cell of obtaining, if the sign that the Data Identification that this unit data unit comprises is file, all Organization of Datas that directly this unit data unit comprised become a message to send to destination data center; If the sign that the Data Identification that this unit data unit comprises is file, the all Organization of Datas that this unit data unit comprised become a message to send to destination data center, and the action type comprising in this unit data unit is when deleting, the routing information comprising according to this unit data unit obtains file content, and the file content obtaining is write to destination data center.
Preferably, in the present invention, the described acquiring unit maximum numbering that metadata node this locality local from the metadata node of described source cloud storage system and described object cloud storage system obtains unit data unit is respectively that at destination data center, to set up object cloud storage system be to carry out setting-up time foundation after source cloud storage system is set up at source data center and described source cloud storage system complete Data Update according to Data Update request;
Described data synchronisation unit is obtained numbering that metadata node metadata node this locality local and described object cloud storage system of Shi Cong source, the unit data unit cloud storage system satisfying condition obtains and is differed and carry out the prerequisite within setting range from metadata node this locality of described source cloud storage system;
The object cloud storage system that described data synchronisation unit is also set up at destination data center is set up after exceeding setting-up time, or, when the numbering that metadata node this locality local from the metadata node of source cloud storage system and described object cloud storage system obtains differs over setting range, further source cloud storage system is shut down, and synchronous according to the mode of batch synchronization executing data between source data center, destination data center; Described setting-up time is that source cloud storage system and object cloud storage system are set up the time of being separated by.
So far, complete the unit describe shown in Fig. 4.
As can be seen from the above technical solutions, in the present invention, by respectively at source data center, cloud storage system is set up at destination data center, from metadata node metadata node this locality local and described object cloud storage system of described source cloud storage system, obtain respectively the maximum numbering of unit data unit, relatively from metadata node metadata node this locality local and described object cloud storage system of described source cloud storage system, obtain the maximum numbering of unit data unit, when the former is greater than the latter, from metadata node this locality of described source cloud storage system, obtain the unit data unit meeting the following conditions: numbering is greater than the numbering of obtaining from metadata node this locality of described object cloud storage system, at described source data center, the unit data unit executing data between destination data center, this being obtained is synchronous.This realizes Liao Kua data center, and to realize cloud storage data synchronous, guarantees the high reliability of cloud storage.
The foregoing is only preferred embodiment of the present invention, in order to limit the present invention, within the spirit and principles in the present invention not all, any modification of making, be equal to replacement, improvement etc., within all should being included in the scope of protection of the invention.
Claims (13)
1.Yi Zhongkua data center realizes the synchronous method of cloud storage data, it is characterized in that, the method comprises:
Respectively at source data center, cloud storage system is set up at destination data center, described source data center, two of offices, destination data center diverse location, at source data center, the cloud storage system that destination data center is set up all comprises metadata node, the cloud storage system of setting up at described source data center, be called source cloud storage system, for cloud stores service is externally provided, when receiving Data Update request, related data corresponding to described Data Update request is organized into a unit data unit, the mode increasing progressively according to number order is that described unit data unit distributes numbering, and described unit data unit and the numbering that is assigned with are stored in to this locality, the cloud storage system of setting up at described destination data center, is called object cloud storage system, for backup source cloud storage system,
From metadata node metadata node this locality local and described object cloud storage system of described source cloud storage system, obtain respectively the maximum numbering of unit data unit, wherein, if metadata node this locality of object cloud storage system is storage cell data cell not, the maximum of giving tacit consent to the local unit data of the metadata node unit of object cloud storage system is numbered set point;
Relatively from metadata node metadata node this locality local and described object cloud storage system of described source cloud storage system, obtain the maximum numbering of unit data unit,
If the former equals the latter, wait for setting-up time, return to afterwards respectively the operation that the maximum of obtaining unit data unit from metadata node metadata node this locality local and described object cloud storage system of described source cloud storage system is numbered;
If the former is less than the latter, send alarm prompt;
If the former is greater than the latter, from metadata node this locality of described source cloud storage system, obtain the unit data unit meeting the following conditions: numbering is greater than the numbering of obtaining from metadata node this locality of described object cloud storage system, and the unit data unit executing data between described source data center, destination data center, this being obtained is synchronous.
2. method according to claim 1, is characterized in that, related data corresponding to described Data Update request comprises but be not limited to:
Data Identification, described Data Identification is: the sign of file or file;
During the sign that is file at Data Identification, the content in file is at the routing information of source cloud storage service system storage; During the sign that is file at Data Identification, file is at the routing information of source cloud storage service system storage;
Action type; Wherein, described action type comprises: revise, delete or create;
There is the timestamp of operation.
3. method according to claim 1 and 2, it is characterized in that, described metadata node this locality from source cloud storage system obtains the unit data unit meeting the following conditions: numbering is greater than the numbering of obtaining from metadata node this locality of described object cloud storage system, and the unit data unit executing data between described source data center, destination data center, this being obtained synchronously comprises:
Identify the local unit data element number meeting the following conditions existing of metadata node of described source cloud storage system: numbering is greater than the numbering of obtaining from metadata node this locality of described object cloud storage system;
The batch unit N of this unit data element number identifying and setting relatively;
If the former is less than or equal to the latter, directly from metadata node this locality of source cloud storage system, obtain the unit data unit meeting the following conditions: numbering is greater than the numbering of obtaining from metadata node this locality of described object cloud storage system, and the unit data unit executing data between described source data center, destination data center, this being obtained is synchronous;
If the former is greater than the latter, first from metadata node this locality of source cloud storage system, obtain N the unit data unit meeting the following conditions: numbering is greater than the numbering of obtaining from metadata node this locality of described object cloud storage system, and the unit data unit executing data between described source data center, destination data center, this being obtained is synchronous; Afterwards the maximum numbering in the unit data unit obtaining is numbered as candidate, identify the local unit data element number meeting the following conditions existing of metadata node of described source cloud storage system: numbering is greater than candidate's numbering, return to the relatively operation of the batch unit N of this unit data element number identifying and setting.
4. method according to claim 3, is characterized in that, the described unit data unit executing data between source data center, destination data center, this being obtained synchronously comprises:
For the per unit data cell of obtaining, if the sign that the Data Identification that this unit data unit comprises is file, all Organization of Datas that directly this unit data unit comprised become a message to send to destination data center; If the sign that the Data Identification that this unit data unit comprises is file, the all Organization of Datas that this unit data unit comprised become a message to send to destination data center, and the action type comprising in this unit data unit is when deleting, the routing information comprising according to this unit data unit obtains file content, and the file content obtaining is write to destination data center.
5. method according to claim 1, it is characterized in that, the maximum numbering that described metadata node this locality local from the metadata node of described source cloud storage system and described object cloud storage system respectively obtains unit data unit is that at destination data center, to set up object cloud storage system be to carry out setting-up time foundation after source cloud storage system is set up at source data center and described source cloud storage system complete Data Update according to Data Update request.
6. method according to claim 1, it is characterized in that, described metadata node this locality from described source cloud storage system obtains numbering that metadata node metadata node this locality local and described object cloud storage system of Shi Cong source, the unit data unit cloud storage system satisfying condition obtains and differs and carry out the prerequisite within setting range.
7. according to the method described in claim 5 or 6, it is characterized in that, if the object cloud storage system that destination data center is set up is set up after exceeding setting-up time, or, the numbering that metadata node this locality local from the metadata node of source cloud storage system and described object cloud storage system obtains differs over setting range, and the method further comprises:
Source cloud storage system is shut down, and synchronous according to the mode of batch synchronization executing data between source data center, destination data center;
Described setting-up time is that source cloud storage system and object cloud storage system are set up the time of being separated by.
8. method according to claim 1, is characterized in that, before the unit data unit executing data between described source data center, destination data center, this being obtained is synchronous, the method further comprises:
Dedicated bandwidth based on setting is set up network and is connected between source data center, destination data center,
If network connects, be not successfully established, attempt set point number, after attempting set point number, still the unsuccessful network of successfully setting up between source data center, destination data center connects, and sends alarm prompt.
9.Yi Zhongkua data center realizes the synchronous device of cloud storage data, it is characterized in that, this device is used at source data center, between destination data center, carry out data synchronous, wherein, described source data center, two of offices, destination data center diverse location, and all set up cloud storage system, at source data center, the cloud storage system that destination data center is set up all comprises metadata node, the cloud storage system of setting up at described source data center, be called source cloud storage system, for cloud stores service is externally provided, when receiving Data Update request, related data corresponding to described Data Update request is organized into a unit data unit, the mode increasing progressively according to number order is that described unit data unit distributes numbering, and described unit data unit and the numbering that is assigned with are stored in to this locality, the cloud storage system of setting up at described destination data center, is called object cloud storage system, for backup source cloud storage system,
This device comprises:
Acquiring unit, for obtain respectively the maximum numbering of unit data unit from metadata node metadata node this locality local and described object cloud storage system of described source cloud storage system, wherein, if metadata node this locality of object cloud storage system is storage cell data cell not, the maximum of giving tacit consent to the local unit data of the metadata node unit of object cloud storage system is numbered set point;
Comparing unit, for relatively obtain the maximum numbering of unit data unit from metadata node metadata node this locality local and described object cloud storage system of described source cloud storage system,
If the former equals the latter, wait for setting-up time, return to afterwards respectively the operation that the maximum of obtaining unit data unit from metadata node metadata node this locality local and described object cloud storage system of described source cloud storage system is numbered;
If the former is less than the latter, send alarm prompt;
If the former is greater than the latter, send notice to data synchronisation unit;
Described data synchronisation unit, for obtaining from metadata node this locality of described source cloud storage system the unit data unit meeting the following conditions: numbering is greater than the numbering of obtaining from metadata node this locality of described object cloud storage system, and the unit data unit executing data between described source data center, destination data center, this being obtained is synchronous.
10. device according to claim 9, is characterized in that, related data corresponding to described Data Update request comprises but be not limited to:
Data Identification, described Data Identification is: the sign of file or file;
During the sign that is file at Data Identification, the content in file is at the routing information of source cloud storage service system storage; During the sign that is file at Data Identification, file is at the routing information of source cloud storage service system storage;
Action type; Wherein, described action type comprises: revise, delete or create;
There is the timestamp of operation.
11. according to the device described in claim 9 or 10, it is characterized in that, described acquiring unit obtains from metadata node this locality of source cloud storage system the unit data unit meeting the following conditions: numbering is greater than the numbering of obtaining from metadata node this locality of described object cloud storage system, and the unit data unit executing data between described source data center, destination data center, this being obtained synchronously comprises:
Identify the local unit data element number meeting the following conditions existing of metadata node of described source cloud storage system: numbering is greater than the numbering of obtaining from metadata node this locality of described object cloud storage system;
The batch unit N of this unit data element number identifying and setting relatively;
If the former is less than or equal to the latter, directly from metadata node this locality of source cloud storage system, obtain the unit data unit meeting the following conditions: numbering is greater than the numbering of obtaining from metadata node this locality of described object cloud storage system, and the unit data unit executing data between described source data center, destination data center, this being obtained is synchronous;
If the former is greater than the latter, first from metadata node this locality of source cloud storage system, obtain N the unit data unit meeting the following conditions: numbering is greater than the numbering of obtaining from metadata node this locality of described object cloud storage system, and the unit data unit executing data between described source data center, destination data center, this being obtained is synchronous; Afterwards the maximum numbering in the unit data unit obtaining is numbered as candidate, identify the local unit data element number meeting the following conditions existing of metadata node of described source cloud storage system: numbering is greater than candidate's numbering, return to the relatively operation of the batch unit N of this unit data element number identifying and setting.
12. devices according to claim 11, is characterized in that, the unit data unit executing data that described data synchronisation unit is obtained this between source data center, destination data center synchronously comprises:
For the per unit data cell of obtaining, if the sign that the Data Identification that this unit data unit comprises is file, all Organization of Datas that directly this unit data unit comprised become a message to send to destination data center; If the sign that the Data Identification that this unit data unit comprises is file, the all Organization of Datas that this unit data unit comprised become a message to send to destination data center, and the action type comprising in this unit data unit is when deleting, the routing information comprising according to this unit data unit obtains file content, and the file content obtaining is write to destination data center.
13. devices according to claim 9, it is characterized in that, the described acquiring unit maximum numbering that metadata node this locality local from the metadata node of described source cloud storage system and described object cloud storage system obtains unit data unit is respectively that at destination data center, to set up object cloud storage system be to carry out setting-up time foundation after source cloud storage system is set up at source data center and described source cloud storage system complete Data Update according to Data Update request;
Described data synchronisation unit is obtained numbering that metadata node metadata node this locality local and described object cloud storage system of Shi Cong source, the unit data unit cloud storage system satisfying condition obtains and is differed and carry out the prerequisite within setting range from metadata node this locality of described source cloud storage system;
The object cloud storage system that described data synchronisation unit is also set up at destination data center is set up after exceeding setting-up time, or, when the numbering that metadata node this locality local from the metadata node of source cloud storage system and described object cloud storage system obtains differs over setting range, further source cloud storage system is shut down, and synchronous according to the mode of batch synchronization executing data between source data center, destination data center; Described setting-up time is that source cloud storage system and object cloud storage system are set up the time of being separated by.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310693744.1A CN103716384A (en) | 2013-12-17 | 2013-12-17 | Method and device for realizing cloud storage data synchronization in cross-data-center manner |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310693744.1A CN103716384A (en) | 2013-12-17 | 2013-12-17 | Method and device for realizing cloud storage data synchronization in cross-data-center manner |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN103716384A true CN103716384A (en) | 2014-04-09 |
Family
ID=50408954
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201310693744.1A Pending CN103716384A (en) | 2013-12-17 | 2013-12-17 | Method and device for realizing cloud storage data synchronization in cross-data-center manner |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103716384A (en) |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104168335A (en) * | 2014-09-02 | 2014-11-26 | 东软熙康健康科技有限公司 | Data synchronization method and device |
| CN105554135A (en) * | 2015-12-23 | 2016-05-04 | 北京奇虎科技有限公司 | Method and device for synchronizing meta information among digital nodes |
| CN106921703A (en) * | 2015-12-25 | 2017-07-04 | 阿里巴巴集团控股有限公司 | The method of cross-border data syn-chronization, system, and domestic and overseas data center |
| CN108121804A (en) * | 2017-12-22 | 2018-06-05 | 百度在线网络技术（北京）有限公司 | Method, apparatus, terminal and the storage medium of cross-region distributed storage data |
| CN108259543A (en) * | 2016-12-29 | 2018-07-06 | 广东中科遥感技术有限公司 | Distributed cloud storage database and its be deployed in the method for multiple data centers |
| CN108475254A (en) * | 2016-12-16 | 2018-08-31 | 华为技术有限公司 | Object clone method, device and object storage device |
| CN109726250A (en) * | 2018-12-27 | 2019-05-07 | 星环信息科技（上海）有限公司 | Data-storage system, metadatabase synchronization and data cross-domain calculation method |
| WO2022078243A1 (en) * | 2020-10-16 | 2022-04-21 | 北京沃东天骏信息技术有限公司 | Information synchronization method and apparatus |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1489052A (en) * | 2002-10-11 | 2004-04-14 | 鸿富锦精密工业（深圳）有限公司 | Multi-node file syn chronizing system and method |
| CN1984461A (en) * | 2006-04-07 | 2007-06-20 | 华为技术有限公司 | Method for restoring user information by access position register |
| CN101582920A (en) * | 2009-06-09 | 2009-11-18 | 中兴通讯股份有限公司 | Method and device for verifying and synchronizing data blocks in distributed file system |
| CN101808137A (en) * | 2010-03-29 | 2010-08-18 | 成都市华为赛门铁克科技有限公司 | Data transmission method, device and system |
| CN102073494A (en) * | 2010-12-30 | 2011-05-25 | 用友软件股份有限公司 | Method and device for managing cache data |
| CN102693168A (en) * | 2011-03-22 | 2012-09-26 | 中兴通讯股份有限公司 | A method, a system and a service node for data backup recovery |
| US20130218837A1 (en) * | 2012-02-16 | 2013-08-22 | Verizon Patent And Licensing Inc. | Cloud data synchronization with local data |
-
2013
- 2013-12-17 CN CN201310693744.1A patent/CN103716384A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1489052A (en) * | 2002-10-11 | 2004-04-14 | 鸿富锦精密工业（深圳）有限公司 | Multi-node file syn chronizing system and method |
| CN1984461A (en) * | 2006-04-07 | 2007-06-20 | 华为技术有限公司 | Method for restoring user information by access position register |
| CN101582920A (en) * | 2009-06-09 | 2009-11-18 | 中兴通讯股份有限公司 | Method and device for verifying and synchronizing data blocks in distributed file system |
| CN101808137A (en) * | 2010-03-29 | 2010-08-18 | 成都市华为赛门铁克科技有限公司 | Data transmission method, device and system |
| CN102073494A (en) * | 2010-12-30 | 2011-05-25 | 用友软件股份有限公司 | Method and device for managing cache data |
| CN102693168A (en) * | 2011-03-22 | 2012-09-26 | 中兴通讯股份有限公司 | A method, a system and a service node for data backup recovery |
| US20130218837A1 (en) * | 2012-02-16 | 2013-08-22 | Verizon Patent And Licensing Inc. | Cloud data synchronization with local data |
Cited By (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104168335A (en) * | 2014-09-02 | 2014-11-26 | 东软熙康健康科技有限公司 | Data synchronization method and device |
| CN104168335B (en) * | 2014-09-02 | 2018-08-03 | 宁波（东软熙康）智慧健康研究院有限公司 | A kind of method of data synchronization and device |
| CN105554135A (en) * | 2015-12-23 | 2016-05-04 | 北京奇虎科技有限公司 | Method and device for synchronizing meta information among digital nodes |
| CN105554135B (en) * | 2015-12-23 | 2019-03-05 | 北京奇虎科技有限公司 | The method and device of synchronous metamessage between back end |
| CN106921703A (en) * | 2015-12-25 | 2017-07-04 | 阿里巴巴集团控股有限公司 | The method of cross-border data syn-chronization, system, and domestic and overseas data center |
| CN108475254A (en) * | 2016-12-16 | 2018-08-31 | 华为技术有限公司 | Object clone method, device and object storage device |
| CN108259543B (en) * | 2016-12-29 | 2021-07-06 | 广东中科遥感技术有限公司 | Distributed cloud storage database and method for deploying same in multiple data centers |
| CN108259543A (en) * | 2016-12-29 | 2018-07-06 | 广东中科遥感技术有限公司 | Distributed cloud storage database and its be deployed in the method for multiple data centers |
| CN108121804A (en) * | 2017-12-22 | 2018-06-05 | 百度在线网络技术（北京）有限公司 | Method, apparatus, terminal and the storage medium of cross-region distributed storage data |
| CN108121804B (en) * | 2017-12-22 | 2020-06-05 | 百度在线网络技术（北京）有限公司 | Cross-region distributed data storage method, device, terminal and storage medium |
| CN109726250B (en) * | 2018-12-27 | 2020-01-17 | 星环信息科技（上海）有限公司 | Data storage system, metadata database synchronization method and data cross-domain calculation method |
| CN109726250A (en) * | 2018-12-27 | 2019-05-07 | 星环信息科技（上海）有限公司 | Data-storage system, metadatabase synchronization and data cross-domain calculation method |
| WO2022078243A1 (en) * | 2020-10-16 | 2022-04-21 | 北京沃东天骏信息技术有限公司 | Information synchronization method and apparatus |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11614990B2 (en) | Automatic correlation of dynamic system events within computing devices | |
| CN109034993B (en) | Account checking method, account checking equipment, account checking system and computer readable storage medium | |
| CN103716384A (en) | Method and device for realizing cloud storage data synchronization in cross-data-center manner | |
| CN108234170B (en) | Monitoring method and device for server cluster | |
| US20180189367A1 (en) | Data stream ingestion and persistence techniques | |
| US20170357703A1 (en) | Dynamic partitioning techniques for data streams | |
| US20190042659A1 (en) | Data writing and reading and apparatus and cloud storage system | |
| CN103473277B (en) | The Snapshot Method and device of file system | |
| CN102611745B (en) | On-line file moving method, device and system | |
| CN108108127B (en) | File reading method and system | |
| US20150135255A1 (en) | Client-configurable security options for data streams | |
| US8375200B2 (en) | Embedded device and file change notification method of the embedded device | |
| CN111309550A (en) | Data acquisition method, system, equipment and storage medium of application program | |
| US8635224B2 (en) | Clustering streaming graphs | |
| AU2014209697A1 (en) | Method and system for using a recursive event listener on a node in hierarchical data structure | |
| WO2016029744A1 (en) | Metadata recovery method and relevant device | |
| CN111338834B (en) | Data storage method and device | |
| CN103186418B (en) | A kind of task distribution method and system | |
| CN115934414A (en) | Data backup method, data recovery method, device, equipment and storage medium | |
| CN109542841B (en) | Method for creating data snapshot in cluster and terminal equipment | |
| CN113609091B (en) | Log management method, device, equipment and storage medium | |
| CN113704203A (en) | Log file processing method and device | |
| CN113760469A (en) | Distributed computing method and device | |
| CN112600918B (en) | Industrial control edge big data efficient processing method and system based on BS architecture | |
| CN112711608B (en) | Data display method, device, computer readable storage medium and computer equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20140409 |
|
| RJ01 | Rejection of invention patent application after publication |