WO2018045758A1

WO2018045758A1 - Data storage method and data centre

Info

Publication number: WO2018045758A1
Application number: PCT/CN2017/081340
Authority: WO
Inventors: 白平昌
Original assignee: 华为技术有限公司
Priority date: 2016-09-08
Filing date: 2017-04-21
Publication date: 2018-03-15
Also published as: CN106484321A

Abstract

A data storage method and a data centre, the data storage method comprising: a data centre reads a second version history queue composed of version numbers stored by an object in the present terminal (S101); determining whether a version number of data of the object presently stored in a first version history queue is greater than the version number in the second version history queue (S102); if said version number is greater than the version number in the second version history queue, then the data centre determines that an optimistic locking conflict is not produced, and the present storage of the data of the object is successful (S103). Hence, when the version number of data of an object presently being stored is greater than a version number already stored by the object, the present method can successfully store the data of the object presently being stored, greatly reducing network overheads compared to methods requiring data centres to reach a resolution for storage.

Description

Data storage method and data center

Technical field

The present invention relates to the field of communications technologies, and in particular, to a data storage method and a data center.

Background technique

At present, cloud storage is a cloud computing system with data storage and management as its core. Users can connect to the cloud to access data through networkable terminals at any time and any place. Among them, object storage is the core of cloud storage, also known as object-based storage. The biggest feature of the object storage system is that the object name is a domain name address. Once the domain name is set to be public, everyone can access the object. In order to solve the problems of disaster recovery and availability of the object storage system, a multi-data center solution can be adopted.

In the multi-data center object storage system, each data center can process the client's business request, that is, the object can be uploaded to any data center, or can be downloaded to the same object from any data center. After the object is uploaded to any data center, it is necessary to ensure that the data of the object finally stored in each data center is the same data, so that the same data can be downloaded when the object is downloaded from any data center. Therefore, when there is data of the uploaded object in the object storage system, a round of resolution process is started. If accepted by most data centers, each data center saves the data of the object uploaded this time, thereby making any one of the data When the data center downloads the object, it can download the same data. However, reaching a round of resolutions requires multiple network interactions in various data centers, resulting in large network overhead.

Summary of the invention

The embodiment of the invention provides a data storage method and a data center, which can reduce network overhead.

A first aspect of the embodiments of the present invention provides a data storage method, which is applied to an object storage system including a plurality of data centers, and includes: when the data center stores the data of the object, the first version of the version number of the object in the local end is read. The second version of the history queue; the data center determines whether the version number of the data of the object stored in the first version history queue is greater than the version number in the second version history queue; if the version number in the second version history queue is greater than the version number, the data is The center determines that no optimistic lock conflict has occurred, and the data storage of the object stored this time is successful.

The version number includes a receiving time and a universal unique identifier UUID of the storage request for receiving data for storing the object sent by the client; the version number after the receiving time is greater than the version number of the receiving time, and the receiving time is the same. The version number of the UUID is larger than the version number of the UUID. The different storage requests of the object correspond to different data of the object, and the data of different objects correspond to different version numbers. Therefore, the data of the object corresponding to each storage request can be distinguished by the version number, and different storage requests can be determined according to the version number. The old and new relationship of the different data of the corresponding object. The UUID is a Universally Unique Identifier (UUID), which is unique in a distributed system, that is, in each data center, and is identified by the current date, time, clock sequence, and globally unique IEEE machine. Number composition. The first part of the UUID is related to time. For example, after a UUID is generated by the same data center, a UUID is generated after a few seconds, and the first part is different, and the others are the same. Even if different data centers receive the same two storage requests for the data of the object, the universal unique identifier is different. Therefore, the receiving time in the version number may be the local time of the data center that receives the storage request, and need not be Full The precise timing of the atomic clock determined by the precise synchronization of the ball time. This embodiment reduces the difficulty of implementing the data storage method and saves the cost of data storage because the data center does not strictly require the atomic clocks of the data centers to be synchronized with the global time to ensure the same time.

It can be seen that, in the embodiment of the present invention, when the version number of the data of the object stored in the current time is greater than the version number of the object already existing, the data of the object stored in the current storage is successfully stored, and a round of resolution is required to be stored. Compared to the method, the network overhead is greatly reduced. In addition, each storage request corresponds to a version number, and the version number can be used to distinguish the old and new relationship of the data of the object corresponding to different storage requests, and the version number of the data of the object stored in each data center in the object storage system is greater than The existing version number of the object can be stored successfully, so that the data of the object stored in each data center is the latest data of the object received by the object storage system, and the data of the object in each data center is consistent.

In a first possible implementation manner of the first aspect, before the data center reads the second version history queue formed by the version number of the object existing in the local end, the data center may receive the copy request sent by the other data center; The request extracts the data of the object and the first version history queue of the object; in response to the copy request, the data of the object is stored. This embodiment can also synchronize the data of the objects received by other data centers into the data center.

In conjunction with the implementation of the first aspect, in a second possible implementation manner of the first aspect, the data center receives the storage request of the object before reading the second version history queue formed by the version number of the object existing in the local end; The data center generates a version number of the data of the object to be stored for the storage request; the data center adds the version number to the tail of the version history queue of the object, constitutes the first version history queue of the object; the data center responds to the storage request, and stores The object's data, and sends a copy request to the other data center, the copy request carries the first version of the object's historical queue. This embodiment can not only store the data of the object in the storage request received by the data center, but also send a copy request to other data centers, so that other data can also synchronously store the data of the object according to the copy request.

With reference to the implementation of the first aspect, or the first or second possible implementation manner of the first aspect, in a third possible implementation manner of the first aspect, the data center is determined in the first version history queue The version number of the data of the stored object is not greater than the version number in the history queue of the second version, and it can be determined that an optimistic lock conflict occurs, and the data storage of the object stored this time fails; then, the data center can compare the first version of the historical queue. The third version history queue is merged with the version number in the second version history queue; the third version history queue is stored to update the version history queue of the local object. The data center may be in an optimistic lock conflict, and the data of the object to be stored is determined to be expired data. However, in order to ensure that the data center updates the version history queue of the object in time, the first version of the historical queue and the local end may still be used. The second version history queue formed by the existing version number of the object is merged to obtain a third version history queue to update the version history queue of the object in the local end.

Correspondingly, whether the storage request triggers the data center storage data or the replication request triggers the data center to store the data, if the data center determines that no optimistic lock conflict occurs, the first version history queue is the current latest version history queue, and the data center can The first version history queue is stored as a version history queue for the object to be used when the data of the object is updated next time.

In conjunction with the third possible implementation of the first aspect, in a fourth possible implementation of the first aspect, The data center compares the version number of the first version of the historical queue with the second version of the historical queue, and aligns the third version of the historical queue, which may be specifically: the order of the data center from the tail to the head, and determining the first version of the historical queue and the first Whether the first version number is the same in the second version history queue; if there is the first version number, the data center will match the first version history queue with the first version number in the second version history queue. All version numbers of the tail are sorted, and the third version history queue is merged; if the first version number is not present, the data center sorts the first version history queue and all version numbers in the second version history queue. The third version history queue is merged.

A second aspect of the embodiments of the present invention provides a data center, where the data center has a behavior function for implementing the method provided by the foregoing first aspect, and the function may be implemented by using hardware or by executing corresponding software by hardware. The hardware or software includes one or more modules corresponding to the functions described above.

DRAWINGS

The drawings used in the embodiments are briefly described. It is obvious that the drawings in the following description are only some embodiments of the present invention, and those skilled in the art, without any creative work, Other drawings can also be obtained from these figures.

1 is a schematic diagram of an object storage system according to an embodiment of the present invention;

2 is a schematic flowchart of a data storage method according to an embodiment of the present invention;

3 is a schematic structural diagram of a data center disclosed in an embodiment of the present invention;

FIG. 4 is another schematic structural diagram of a data center according to an embodiment of the present invention.

detailed description

In order to facilitate the understanding of the embodiments of the present invention, the application scenarios of the embodiments of the present invention are described below. Please refer to FIG. 1. FIG. 1 is a schematic diagram of an object storage system according to an embodiment of the present invention. As shown in FIG. 1 , in the embodiment of the present invention, the object storage system may include multiple data centers. In the embodiment of the present invention, two data centers DC1 and DC2 are taken as an example, and each data center may include a load balancing module (Load Balancing, LB), an object storage subsystem, and a KV (Key-Value) subsystem, wherein the load balancing subsystem is configured to allocate storage requests of client Client A and Client B to respective object storage subsystems to avoid storage of an object. The load on the subsystem is too large; the object storage subsystem is responsible for assigning the version number to the object's data and sorting the multiple version numbers of the object; the KV subsystem is responsible for storing or writing the object's data to disk. As shown in FIG. 1 , each data center can receive a storage request uploaded by a client, and the data storage method provided by the embodiment of the present invention can be used in each data center of the object storage system, regardless of which data center is uploaded from the data storage system. The object's data and reduces network overhead. The details are described below separately.

Referring to FIG. 2, FIG. 2 is a data storage method according to an embodiment of the present invention. As shown in FIG. 2, the data storage method includes the following steps:

S101. The data center stores the data of the object, and reads a second version history queue formed by the version number of the object existing in the local end;

S102. The data center determines whether the version number of the data of the object stored in the first version history queue is greater than The version number in the second version of the historical queue; if it is greater than the version number in the second version history queue, step S103 is performed; if it is not greater than the version number in the second version history queue, the process ends.

S103. The data center determines that an optimistic lock conflict is not generated, and the data storage of the object stored this time is successful.

In the embodiment of the present invention, the execution body of the steps S101 to S103 may also be an object storage subsystem of the data center or a server of the data center, etc., which is not limited by the embodiment of the present invention.

In the embodiment of the present invention, the version number includes a receiving time and a universal unique identification code UUID of the storage request for receiving the data of the storage object sent by the client; the version number after the receiving time is greater than the version number of the receiving time first. For the version number with the same receiving time, the version number with a large UUID is larger than the version number with a smaller UUID. The different storage requests of the object correspond to different data of the object, and the different data of the object corresponds to different version numbers. Therefore, the data of the object corresponding to each storage request can be distinguished by the version number, and different storage can be determined according to the version number. Request new and old relationships for different data of the corresponding object. The UUID is a Universally Unique Identifier (UUID), which is unique in a distributed system, that is, in each data center, and is identified by the current date, time, clock sequence, and globally unique IEEE machine. Number composition. The first part of the UUID is related to time. For example, after a UUID is generated by the same data center, a UUID is generated after a few seconds, and the first part is different, and the others are the same. Even if different data centers receive the same two storage requests for the data of the object, the universal unique identifier is different. Therefore, the receiving time in the version number may be the local time of the data center that receives the storage request, and need not be Precise timing with precise time synchronization of atomic clocks around the world. This embodiment reduces the difficulty of implementing the data storage method and saves the cost of data storage because the data center does not strictly require the atomic clocks of the data centers to be synchronized with the global time to ensure the same time.

Optionally, before the data center reads the second version history queue formed by the version number of the object existing in the local end, the method may further include the following steps:

11) The data center receives the copy request sent by other data centers;

12) the data center extracts the data of the object from the copy request and the first version history queue of the object;

13) The data center responds to the copy request and stores the data of the object.

In this embodiment, the stored data comes from data synchronized by other data centers, thereby ensuring data consistency in each data center.

Optionally, the data of the data center storage object may also be the data of the object corresponding to the storage request sent by the client to the data center, and may include the following steps:

21) the data center receives the storage request of the object;

22) the data center generates a version number of the object for the storage request;

23) The data center adds the version number to the end of the object's version history queue, forming the first version of the object. This historical queue;

24) The data center responds to the storage request, stores the data of the object, and sends a copy request to the other data center.

The copy request is used to synchronously write data of an object corresponding to the storage request to another data center, where the copy request carries a first version history queue of the object. The version number includes a receiving time and a UUID of the storage request corresponding to the version number; the receiving time is a receiving time of the storage request corresponding to the data of the object to be stored in the data center.

In the embodiment of the present invention, when the data center stores the data of the object, if the object does not exist, the version history queue of the object is empty, NULL; if the object already exists, the version history queue of the object is not empty. For example, the version number generated by the data center for the storage request is V2. If the version history queue of the data center before the storage request is NULL, the first version history queue is NULL->V2; if the data center is in the storage The version history queue before the request is NULL->V1, then the first version history queue is NULL->V1->V2. In addition, the version number in the version history queue located near the head is smaller than the version number located near the end of the queue, and the newly generated version number can be directly added to the tail of the version history pair column to form the first version history queue.

In the embodiment of the present invention, when the data center receives the storage request, the data center generates a version number for the object to be stored, and adds the version number to the data center that has been stored or already exists. The end of the version history queue of the object is formed; the second version history queue is a version history queue of the object stored or existing in the data center. For example, when receiving the storage request, the data center DC1 may generate a version number of the data of the object to be stored for the storage request, and add the version number to the tail of the version history queue of the object in DC1 to form the first version. The history queue; when the data center DC2 receives the copy request sent by the DC1, the first version history queue can be extracted, and the second version history queue of the version number of the object already existing in the DC2 is read, and the DC2 is based on the first The version history queue and the second version history queue store data of the object corresponding to the storage request.

In the embodiment of the present invention, if the data center determines that the version number of the data of the object stored in the first version of the history queue is not greater than the version number in the history queue of the second version, the data storage method may further include the following steps:

31) the data center determines to generate an optimistic lock conflict, and the data storage of the object stored in the current storage fails;

32) The data center compares the version numbers of the first version history queue with the second version history queue, and merges and lists the third version history queue;

33) The data center stores the third version history queue to update the version history queue of the object in the local end.

In this implementation manner, the data center stores the third version history queue, so that when the data of the object is updated next time, the third version history queue is used as the version history queue of the object that already exists in the data center.

For example, scenario 1 is: the data center DC1 receives the storage request of the object, and the generated version number is V1; if the version history queue of the object that already exists in DC1 is NULL, the first version history queue is NULL->V1.

For scenario 1, the second version history queue read by DC1 is NULL, and DC1 determines the history of the first version in the column. The version number V1 is greater than the NULL in the second version history queue. Therefore, DC1 determines that no optimistic lock conflict is generated, so DC1 can successfully store the data of the object corresponding to the version number V1.

For example, the scenario 2 is: the data center DC1 receives the copy request sent by the data center DC2, and the copy request is used to synchronously write the data of the object corresponding to the storage request received by the DC2 to the DC1, where the copy request carries the object. The first version history queue is NULL->V1, and the version history queue of the object already existing by DC1 is NULL, wherein the version number V1 is generated by DC2 for the data of the object corresponding to the storage request it receives.

For scene 2, the second version history queue read by DC1 is NULL. DC1 determines that the version number V1 in the first version history pair column is greater than the NULL value in the second version history queue. Therefore, DC1 determines that no optimistic lock conflict occurs. The data of the object corresponding to the version number V1 is successfully stored.

For example, the scenario 3 is: the data center DC1 receives two storage requests of the same object, and the version numbers generated for the two storage requests are respectively V1 and V2, wherein the storage request receiving time corresponding to the version number V1 is received. Prior to the receiving time of the storage request corresponding to the version number V2, the version number V1 is smaller than the version number V2, and the version history queue of the object already existing by DC1 is NULL.

For the scenario 3, DC1 first stores the data of the object corresponding to the version number V1, and determines that the first version history queue is NULL->V1, and the read second version history queue is NULL, and DC1 determines the first version history pair column. The medium version number V1 is greater than the NULL in the second version history queue. Therefore, the DC1 determines that the optimistic lock conflict is not generated, and the DC1 can successfully store the data of the object corresponding to the version number V1. Second, the DC1 stores the object corresponding to the version number V2. The data determines that the first version history queue is NULL->V1->V2, and the read second version history queue is NULL->V1, and DC1 determines that the version number V2 in the first version history is greater than the second version history. The version number V1 in the queue, therefore, DC1 determines that no optimistic lock conflict is generated, and DC1 can successfully store the data of the object corresponding to the version number V2.

For example, in scenario 4, the data center DC1 receives the storage request of the object, and the generated version number is V3. The version history queue of the object that already exists in DC1 is NULL->V1->V2, and the first version history queue is NULL->V1->V2->V3.

For scenario 4, the second version history queue read by DC1 is NULL->V1->V2, and DC1 determines that the version number V3 in the first version history pair column is greater than each version number in the second version history queue, therefore, DC1 determines that no optimistic lock conflict is generated, and DC1 can successfully store the data of the object corresponding to version number V3.

For example, the scenario 5 is: the data center DC1 receives the copy request sent by the data center DC2, and the copy request is used to synchronously write the data of the object corresponding to the storage request received by the DC2 to the DC1, where the copy request carries the object. The first version history queue is NULL->V1->V2, and the version history queue of the object already existing by DC1 is NULL->V1, wherein the version number V2 is generated by DC2 for the data of the object corresponding to the storage request it receives. .

For scenario 5, the second version history queue read by DC1 is NULL->V1, and DC1 determines that the version number V2 of the tail of the first version history column is greater than the V1 of the tail of the second version history queue, therefore, DC1 is determined. The optimistic lock conflict is not generated, and DC1 can successfully store the data of the object corresponding to the version number V2.

For example, scenario 6 is: the data center DC1 receives the storage request of the object, and the generated version number is V1; meanwhile, the data center DC2 receives the storage request of the same object, and the generated version number is V2, and the DC1 and DC2 are already The existing version history queue of the object is NULL, and the version number V1 is greater than the version number V2 (ie, the data center DC1 and the data center DC2 receive the storage request at the same time, but the UUID is different in the version number generated by the two, V1 UUID is greater than the UUID of V2).

For the scenario 6 above, first, when DC1 stores the data of the object corresponding to the version number V1, the first version history queue is NULL->V1, and the second version history queue is NULL, and the first version is determined according to the version number comparison rule. The V1 in the history queue is larger than the NULL in the history queue of the second version, and it is determined that the optimistic lock conflict is not generated. DC1 stores the data of the object corresponding to the version number V1 successfully. Secondly, the DC1 receives the copy request sent by the DC2, and stores the version number V2. The data of the corresponding object is the NULL->V2 of the first version. The history queue of the second version is NULL->V1 (that is, the data of the object corresponding to the successful version number V1 is stored by DC1). The queue is the first version history queue NULL->V1 when storing the data of the object corresponding to the version number V1. Since V2 in the first version history queue is smaller than V1 in the second version history queue, it is determined that an optimistic lock conflict occurs, DC1 The data of the object corresponding to the version number V2 fails to be stored, and the data corresponding to the version number V2 is expired data. Therefore, the order of the DC1 from the tail to the head determines the first version of the historical queue and the second. The same version number does not exist in the history queue; all the version numbers in the first version history queue and the second version history queue are sorted, that is, the version number V2 and the version number V1 are sorted; the version number V1 is greater than the version number V2, Therefore, the third version history queue is NULL->V2->V1, and DC1 stores the third version history queue to update the version history queue of the object to NULL->V2->V1.

For example, scenario 7 is: the data center DC1 receives the storage request of the object, and the generated version number is V1; meanwhile, the data center DC2 receives the storage request of the same object, and the generated version number is V2, and the DC1 and DC2 have already been generated. The existing version history queue of the object is NULL, and the version number V1 is smaller than the version number V2 (that is, although the data center DC1 and the data center DC2 receive the storage request at the same time, the UUID in the version number generated by the two is different, V1 UUID is less than the UUID of V2).

For the scenario 7 above, first, when DC1 stores the data of the object corresponding to the version number V1, the first version history queue is NULL->V1, the second version history queue is NULL, and the first version history queue has V1 greater than the second version. NULL in the history queue determines that no optimistic lock conflict is generated. DC1 stores the data of the object corresponding to version number V1 successfully. Secondly, DC1 receives the copy request sent by DC2, and stores the data of the object corresponding to version number V2. The version history queue of the version is NULL->V2. In this case, the history queue of the second version is NULL->V1 (that is, when the data of the object corresponding to the successful version number V1 is stored by DC1, the version history queue of the object is the storage version number V1. The first version history queue of the corresponding object data is NULL->V1); V2 in the first version history queue is larger than V1 in the second version history queue, and it is determined that no optimistic lock conflict is generated, and DC1 will be the object corresponding to the version number V2. The data was stored successfully.

It should be noted that whether the DC1 storage data triggered by the storage request or the DC1 storage data triggered by the replication request, if DC1 determines that no optimistic lock conflict is generated, it indicates that the first version history queue is the current latest version history queue, and DC1 can The first version history queue is stored as a version history queue for the object to be used when the data of the object is updated next time.

The data storage method shown in FIG. 2 can read the existing version of the object in the local end when storing the data of the object. The second version of the history queue is configured by the data center; the data center determines whether the version number of the data stored in the first version of the historical queue is greater than the version number in the history queue of the second version; The version number, the data center determines that the optimistic lock conflict is not generated, and the stored data of the object is successfully stored. It can be seen that the version number of the data of the object stored in the present embodiment is greater than the version of the object that already exists. When the number is stored, the data of the object stored in this storage is successfully stored, and the network overhead is greatly reduced compared with the method that needs to achieve a round of resolution for storage. In addition, each storage request corresponds to a version number, and the version number can be used to distinguish the old and new relationship of the data of the object corresponding to different storage requests, and the version number of the data of the object stored in each data center in the object storage system is greater than The existing version number of the object can be stored successfully, so that the data of the object stored in each data center is the latest data of the object received by the object storage system, and the data of the object in each data center is consistent.

The embodiment of the present invention takes the object storage system in FIG. 1 as an example, and describes the data storage method in detail, which may include the following steps:

The client Client A sends a storage request to the data center DC1, and the storage request is used to store the data 1 of the object Obj;

DC1 receives the storage request, and the load balancing module 1 in DC1 allocates the storage request to the object storage subsystem 11 in DC1 for processing;

The object storage subsystem 11 generates a version number V1 of the data 1 of the object Obj for the storage request;

The object storage subsystem 11 reads the version history queue of the object Obj in DC1 to be NULL;

The object storage subsystem 11 adds the version number V1 to the tail of the version history queue of the object Obj, and the first version history queue constituting the object Obj is NULL->V1;

The object storage subsystem 11 calls the data 1 of the KV subsystem 1 storage object Obj in DC1, and sends a copy request to the data center DC2, the copy request carrying the first version history queue NULL->V1 and the data 1 of the object Obj;

When the object storage subsystem 11 calls the data 1 of the KV subsystem 1 storage object Obj in DC1, the optimistic lock conflict returned by the KV subsystem 1 is not received, and the data 1 of the object Obj is determined to be successfully stored, and at this time, the object Obj of the DC1 is The version history queue is NULL->V1;

The data center DC2 receives the copy request sent by the object storage subsystem 11, and the load balancing module 1 in DC2 allocates the copy request to the object storage subsystem 21 in DC2 for processing;

The object storage subsystem 21 extracts the data 1 of the object Obj from the copy request and the first version history queue of the object Obj is NULL->V1;

The object storage subsystem 21 calls the KV subsystem 2 in DC2 to respond to the copy request, and stores the data 1 of the object Obj;

When the KV subsystem 2 stores the data 1 of the object Obj, the second version history queue formed by reading the version number of the object Obj already existing in the DC2 is NULL;

The KV subsystem 2 determines that the version number V1 of the data 1 of the object Obj in the first version history queue is greater than the empty second version history queue, and determines that no optimistic lock conflict is generated;

The object storage subsystem 21 does not receive the optimistic lock conflict returned by the KV subsystem 2, and determines that the data 1 of the object Obj is successfully stored, and at this time, the version history queue of the object Obj in DC2 is NULL->V1;

At this time, the client Client B sends a storage request to the data center DC2 for storing the data 2 of the object Obj; meanwhile, the client Client A sends a storage request to the data center DC1 for storing the object Obj. Data 3;

DC2 receives the storage request, and the load balancing module 2 in DC2 allocates the storage request to the object storage subsystem 21 in DC2 for processing; DC1 receives the storage request, and the load balancing module 1 in DC1 allocates the storage request to the object storage in DC1. System 11 processing;

The object storage subsystem 21 generates a version number V2 of the data 2 of the object Obj for the storage request; the object storage subsystem 11 generates a version number V3 of the data 3 of the object Obj for the storage request; wherein the reception time in the version number V2 is The receiving time in the version number V3 is the same, but the UUID in the version number V2 is smaller than the UUID in the version number V3;

The object storage subsystem 21 reads the version history queue of the object Obj in DC2 as NULL->V1; the object storage subsystem 21 adds the version number V2 to the end of the version history queue of the object Obj, constituting the first version history of the object Obj The queue is NULL->V1->V2; the object storage subsystem 11 reads the version history queue of the object Obj in DC1 to be NULL->V1; the object storage subsystem 11 adds the version number V3 to the team of the version history queue of the object Obj. At the end, the first version of the historical queue constituting the object Obj is NULL->V1->V3;

The object storage subsystem 21 calls the data 2 of the KV subsystem 2 in the DC2 storage object Obj, and sends a copy request to the data center DC1, which carries the data of the first version history queue NULL->V1->V2 and the object Obj. 2; the object storage subsystem 11 calls the data 3 of the KV subsystem 1 storage object Obj in DC1, and sends a copy request to the data center DC2, the copy request carrying the first version history queue NULL->V1->V3 and the object Obj Data 3;

When the object storage subsystem 21 calls the data 2 of the object Vb subsystem 2 in the DC2 storage object Obj, the optimistic lock conflict returned by the KV subsystem 2 is not received, and the data 2 of the object Obj is determined to be successfully stored, and at this time, the object Obj of the DC2 is The version history queue is NULL->V1->V2; when the object storage subsystem 11 calls the data 3 of the KV subsystem 1 storage object Obj in DC1, the optimistic lock conflict returned by the KV subsystem 1 is not received, and the data of the object Obj is determined. 3 The storage is successful, and the version history queue of the object Obj in DC1 is NULL->V1->V3;

The data center DC2 receives the copy request sent by the object storage subsystem 11, the load balancing module 1 in DC2 assigns the copy request to the object storage subsystem 21 in DC2; the data center DC1 receives the copy request sent by the object storage subsystem 21. , the load balancing module 1 in DC1 assigns the copy request to the object storage subsystem in DC1 11 processing;

The object storage subsystem 21 extracts the data 3 of the object Obj from the copy request and the first version history queue of the object Obj is NULL->V1->V3; the object storage subsystem 11 extracts the data of the object Obj from the copy request 2 And the first version history queue of the object Obj is NULL->V1->V2;

The object storage subsystem 21 calls the KV subsystem 2 in DC2 to respond to the copy request, and stores the data 3 of the object Obj; the object storage subsystem 11 calls the KV subsystem 1 in DC2 to respond to the copy request, and stores the data 2 of the object Obj;

When the KV subsystem 2 stores the data 3 of the object Obj, the second version history queue formed by reading the version number of the object Obj already existing in DC2 is NULL->V1->V2; when the KV subsystem 1 stores the data 2 of the object Obj Reading the second version history queue formed by the version number of the object Obj existing in DC1 is NULL->V1->V3;

The KV subsystem 2 determines that the version number V3 of the data 3 of the object Obj in the first version history queue is greater than the version number of the second version history queue NULL->V1->V2, and determines that no optimistic lock conflict is generated; the KV subsystem 1 determining that the version number V2 of the data 2 of the object Obj in the first version history queue is not greater than V3 of the second version history queue ULL->V1->V3, determining that an optimistic lock conflict is generated;

The object storage subsystem 21 does not receive the optimistic lock conflict returned by the KV subsystem 2, and determines that the data 3 of the object Obj is successfully stored, and at this time, the version history queue of the object Obj in DC2 is NULL->V1->V2->V3; The object storage subsystem 11 receives the optimistic lock conflict returned by the KV subsystem 1, and determines that the data 2 storage of the object Obj fails. At this time, the object storage subsystem 11 compares the first version history queue NULL->V1->V2 with the second. The version number in the version history queue NULL->V1->V3 merges the third version history queue NULL->V1->V2->V3; the object storage subsystem 11 stores the third version history queue as the object Obj Version history queue in DC1.

It can be seen that the data of the object Obj finally stored in DC1 and DC2 is data 3; the version history queue of the object Obj finally stored is NULL->V1->V2->V3. The data storage method does not need to perform network resolution between the data centers DC1 and DC2, and each data center can achieve consistent data of the stored objects, thereby reducing network overhead. In addition, in this embodiment, the receiving time of the storage request in the version number is the local time of the data center that receives the storage request, and it is not necessary to accurately time the atomic clock of the global time precision synchronization through GPS, thereby reducing the difficulty of implementing the solution. And cost.

Please refer to FIG. 3. FIG. 3 is a schematic structural diagram of a data center according to an embodiment of the present invention. The data center may include the following modules:

The reading module 310 is configured to: when storing the data of the object, read the second version history queue formed by the version number of the object already existing in the local end;

The determining module 320 is configured to determine whether the version number of the data of the object stored in the first version history queue is greater than the version number in the second version history queue;

The determining module 330 is configured to determine, when the determining module 320 determines that the version number of the data of the object stored in the first version historical queue is greater than the version number in the historical queue of the second version, determining that no optimistic lock conflict occurs. The stored data of the object is successfully stored;

In the embodiment of the present invention, the version number includes a receiving time and a universal unique identification code UUID of the storage request for receiving the data of the storage object sent by the client; the version number after the receiving time is greater than the version number of the receiving time first. For the version number with the same receiving time, the version number with a large UUID is larger than the version number with a smaller UUID. The different storage requests of the object correspond to different data of the object, and different storage requests also correspond to different version numbers. Therefore, the data of the object corresponding to each storage request can be distinguished by the version number, and different storage can be determined according to the version number. Request new and old relationships for different data of the corresponding object. The UUID is a Universally Unique Identifier (UUID), which is unique in a distributed system, that is, in each data center, and is identified by the current date, time, clock sequence, and globally unique IEEE machine. Number composition. The first part of the UUID is related to time. For example, after a UUID is generated by the same data center, a UUID is generated after a few seconds, and the first part is different, and the others are the same. Even if different data centers receive the same two storage requests for the data of the object, the universal unique identifier is different. Therefore, the receiving time in the version number may be the local time of the data center that receives the storage request, and need not be Precise timing with precise time synchronization of atomic clocks around the world. This embodiment reduces the difficulty of implementing the data storage method and saves the cost of data storage because the data center does not strictly require the atomic clocks of the data centers to be synchronized with the global time to ensure the same time.

In the embodiment of the present invention, the data center shown in FIG. 3 further includes:

The first receiving module 340 is configured to receive a copy request sent by another data center before the reading module 310 reads the second version history queue formed by the version number of the object existing in the local end;

The extracting module 350 is configured to extract data of the object and a first version history queue of the object from the copy request;

The first storage module 360 is configured to store data of the object in response to the copy request.

The second receiving module 370 is configured to receive a storage request of the object before the reading module 310 reads the second version history queue formed by the version number of the object existing in the local end;

a generating module 380, configured to generate, according to the storage request, a version number of the data of the object to be stored this time;

The adding module 390 is configured to add a version number to the tail of the version history queue of the object, and constitute a first version history queue of the object;

The second storage module 390a is configured to store the data of the object in response to the storage request, and send a copy request to the other data center, where the copy request carries the first version history queue of the object.

In the embodiment of the present invention, the first receiving module 340 and the second receiving module 370 may be the same receiving module and perform different operations.

In the embodiment of the present invention, the determining module 330 is further configured to: when the determining module 320 determines that the version number of the data of the object stored in the first version history queue is not greater than the version number in the history queue of the second version, Determining that an optimistic lock conflict is generated, and the data storage of the object stored this time fails;

Furthermore, the data center shown in Figure 3 further includes:

The comparison module 390b is configured to compare the version numbers of the first version history queue and the second version history queue, and merge and arrange the third version history queue;

The third storage module 390c is further configured to store the third version history queue to update the version history queue of the object in the local end.

In the embodiment of the present invention, the first storage module 360, the second storage module 390a, and the third storage module 390c may be the same storage module, and respectively perform different operations.

In the embodiment of the present invention, the comparison module 390b may include the following units:

a determining unit, configured to determine, from the tail of the team to the order of the head, whether the first version number is the same in the first version history queue and the second version history queue;

a merging unit, configured to: when the determining unit determines that the first version of the historical queue and the second version of the historical queue have the same version number, the first version of the historical queue is the same as the first version of the second version of the historical queue All the version numbers of the number to the end of the team are sorted, and the third version history queue is merged; the merging unit is further configured to determine, in the determining unit, that the first version of the historical queue does not have the first version number in the second version history queue. The first version history queue is sorted with all version numbers in the second version history queue, and the third version history queue is merged.

It should be noted that whether the data center storage data triggered by the storage request or the data center storage data triggered by the replication request, if the data center determines that no optimistic lock conflict occurs, the first version history queue is the current latest version history queue. The data center may store the first version history queue as the version history queue of the object, so as to be used when the data of the object is updated next time.

In the data center shown in FIG. 3, the reading module can read the second version history queue formed by the version number of the object existing in the local end when storing the data of the object; the determining module can determine the first version history queue. Whether the version number of the data stored in the object is greater than the version number in the second version history queue; the determining module is configured to determine, in the determining module, the version of the data of the object stored in the first version history queue When the number is greater than the version number in the history queue of the second version, it is determined that the optimistic lock conflict is not generated, and the data storage of the object stored in this time is successful; it can be seen that the version number of the data of the object that can be stored in the embodiment of the present invention can be If the version number of the object is greater than the version number of the object, the data stored in the object is successfully stored. Compared with the method that needs to achieve a round of resolution for storage, the network overhead is greatly reduced. In addition, each storage request corresponds to a version number, and the version number can be used to distinguish the old and new relationship of the data of the object corresponding to different storage requests, and the version number of the data of the object stored in each data center in the object storage system is greater than The existing version number of the object can be stored successfully, so that the data of the object stored in each data center is the latest data of the object received by the object storage system, thereby ensuring the data of the object in each data center. To.

The embodiments of the present invention and the method embodiments are based on the same concept, and the technical effects brought by the embodiments are also the same. For the specific process, reference may be made to the description of the foregoing method embodiments.

Referring to FIG. 5, FIG. 5 is another schematic structural diagram of a data center according to an embodiment of the present invention. In the embodiment of the present invention, the data center includes a processor 410, a memory 420, and a transceiver 430. The transceiver 430 is configured to transmit and receive data with and from an external device. The number of processors 410 in the data center can be one or more. In some embodiments of the invention, processor 410, memory 420, and transceiver 430 may be coupled by a bus system or other means. The data center can be used to perform the method shown in Figure 2. For the meanings and examples of the terms involved in the embodiment, reference may be made to the corresponding method embodiments described above. I will not repeat them here.

The program code is stored in the memory 420, and the processor 410 is configured to call the program code stored in the memory 420 for performing the following operations:

When storing the data of the object, reading the second version history queue formed by the version number of the object existing in the local end;

Determining whether a version number of the data of the object stored in the first version history queue is greater than a version number in the second version history queue;

If it is greater than the version number in the second version history queue, it is determined that an optimistic lock conflict is not generated, and the data storage of the object stored in the current storage is successful;

The version number includes a receiving time and a universal unique identifier UUID of the storage request for receiving data for storing the object sent by the client; the version number after the receiving time is greater than the version number of the receiving time, and the receiving time is the same The version number of the UUID is larger than the version number of the UUID.

In the embodiment of the present invention, the processor 410 is configured to call the program code stored in the memory 420, and before the data center reads the second version history queue formed by the version number of the object existing in the local end, the following operations may be performed:

Receiving a copy request sent by another data center through the transceiver 430;

Extracting data of the object and the first version history queue of the object from the copy request;

The data of the object is stored in response to the copy request.

In the embodiment of the present invention, the processor 410 is configured to call the program code stored in the memory 420, and before performing the second version history queue formed by the version number of the object existing in the local end, the following operations may be performed:

Receiving a storage request of the object;

Generating a version number of the data of the object to be stored this time for the storage request;

Adding the version number to the tail of the version history queue of the object to form a first version history queue of the object;

In response to the storage request, data of the object is stored and a copy request is sent to other data centers via the transceiver 430, the copy request carrying a first version history queue of the object.

In the embodiment of the present invention, the processor 410 is configured to invoke the program code stored in the memory 420, and determine that the version number of the data of the object stored in the first version history queue is not greater than the second version history queue. The version number in , you can also do the following:

Determining that an optimistic lock conflict is generated, and the data storage of the object stored this time fails;

Comparing the version numbers of the first version history queue and the second version history queue, and arranging the third version history queues;

The third version history queue is stored to update the version history queue of the object at the local end.

In the embodiment of the present invention, the processor 410 is configured to invoke the program code stored in the memory 420, compare the version numbers in the first version history queue and the second version history queue, and combine and arrange the third version history queue. Can be specifically:

Determining, from the tail of the team to the order of the head, whether the first version number is the same in the first version history queue and the second version history queue;

If there is a first identical version number, sorting the first version history queue with the first version number of the first version in the second version history queue to all version numbers at the end of the queue, and merging the same Three version history queue;

If the first version number is not the same, the first version history queue and all the version numbers in the second version history queue are sorted to merge the third version history queue.

The data storage method and the data center disclosed in the embodiments of the present invention are described in detail above. The principles and implementation manners of the present invention are described in the following. The description of the above embodiments is only used to help understand the present invention. The method and its core idea; at the same time, those skilled in the art, according to the idea of the present invention, there will be changes in the specific implementation and application scope. In summary, the content of the specification should not be understood as Limitations of the invention.

Claims

A data storage method is applied to an object storage system including a plurality of data centers, and includes:

When the data center stores the data of the object, the second version history queue formed by the version number of the object existing in the local end is read;

Determining, by the data center, whether a version number of the data of the object stored in the first version history queue is greater than a version number in the second version history queue;

If the version number is greater than the version number in the second version history queue, the data center determines that an optimistic lock conflict is not generated, and the data storage of the object stored in the current storage is successful;

The version number includes a receiving time and a universal unique identifier UUID of the storage request for receiving data for storing the object sent by the client; the version number after the receiving time is greater than the version number of the receiving time, and the receiving time is the same The version number of the UUID is larger than the version number of the UUID.
The method according to claim 1, wherein the method further comprises: before the data center reads the second version history queue formed by the version number of the object that exists in the local end, the method further includes:

The data center receives a copy request sent by another data center;

Extracting data of the object and the first version history queue of the object from the copy request;

The data of the object is stored in response to the copy request.
The method according to claim 1, wherein the method further comprises: before the data center reads the second version history queue formed by the version number of the object that exists in the local end, the method further includes:

The data center receives a storage request of the object;

Generating, by the data center, a version number of the data of the object to be stored this time for the storage request;

The data center adds the version number to the tail of the version history queue of the object, and constitutes a first version history queue of the object;

The data center stores the data of the object in response to the storage request, and sends a copy request to another data center, the copy request carrying a first version history queue of the object.
The method according to any one of claims 1 to 3, wherein if the data center determines that the version number of the data of the object stored in the first version history queue is not greater than the second version The version number in the history queue, the method further includes:

The data center determines that an optimistic lock conflict is generated, and the data storage of the object that is stored this time fails;

Comparing, by the data center, the version numbers in the first version history queue and the second version history queue, and arranging the third version history queues;

The data center stores the third version history queue to update a version history queue of the object at the local end.
The method according to claim 4, wherein the data center compares the version numbers in the first version history queue and the second version history queue, and aligns the third version history queues, including:

Determining, in the order of the data center from the tail to the head, whether the first version number is the same in the first version history queue and the second version history queue;

If there is a first identical version number, the data center sorts the first version history queue and the first version number in the second version history queue to all version numbers at the end of the queue. , merge the third version of the historical queue;

If the first version number is not the same, the data center sorts the first version history queue and all the version numbers in the second version history queue to merge the third version history queue.
A data center characterized by comprising:

a reading module, configured to read a second version history queue formed by a version number of the object existing in the local end when storing data of the object;

a judging module, configured to determine whether a version number of the data of the object stored in the first version history queue is greater than a version number in the second version history queue;

a determining module, configured to determine, when the determining module determines that the version number of the data of the object stored in the first version history queue is greater than the version number in the second version history queue, determining that no optimistic lock conflict occurs The data storage of the object stored in the current storage is successful;

The version number includes a receiving time and a universal unique identifier UUID of the storage request for receiving data for storing the object sent by the client; the version number after the receiving time is greater than the version number of the receiving time, and the receiving time is the same The version number of the UUID is larger than the version number of the UUID.
The data center according to claim 6, further comprising:

a first receiving module, configured to receive a copy request sent by another data center before the reading module reads the second version history queue formed by the version number of the object that exists in the local end;

An extracting module, configured to extract data of the object and the first version history queue of the object from the copy request;

And a first storage module, configured to store data of the object in response to the copy request.
The data center according to claim 6, further comprising:

a second receiving module, configured to receive a storage request of the object before the reading module reads the second version history queue formed by the version number of the object that exists in the local end;

a generating module, configured to generate, for the storage request, a version number of data of the object to be stored this time;

Adding a module, configured to add the version number to a tail of a version history queue of the object, and constitute a first version history queue of the object;

And a second storage module, configured to store data of the object in response to the storage request, and send a copy request to another data center, where the copy request carries a first version history queue of the object.
A data center according to any one of claims 6 to 8, characterized in that

The determining module is further configured to: when the determining module determines that the version number of the data of the object stored in the first version history queue is not greater than the version number in the second version history queue, Optimistic lock conflict, the data storage of the object stored this time fails;

The data center also includes:

a comparison module, configured to compare the version numbers of the first version history queue and the second version history queue, and align the third version history queues;

And a third storage module, configured to store the third version history queue to update a version history queue of the object at the local end.
The data center of claim 9 wherein said comparing module comprises:

a determining unit, configured to determine, from an end of the queue to the head, whether the first version number is the same in the first version history queue and the second version history queue;

a merging unit, configured to: when the determining unit determines that the first version history queue and the second version history queue have the same version number, the first version history queue and the second The first identical version number in the version history queue is sorted to all version numbers at the end of the queue, and the third version history queue is merged;

The merging unit is further configured to: determine, by the determining unit, that the first version history queue and the second version history queue do not have a first version number, and the first version history queue and the All version numbers in the second version history queue are sorted, and the third version history queue is merged.