CN110515543B

CN110515543B - Object bucket-based snapshot method, device and system

Info

Publication number: CN110515543B
Application number: CN201910713323.8A
Authority: CN
Inventors: 张天旭; 鲁加福; 王豪迈; 胥昕
Original assignee: Xsky Beijing Data Technology Corp ltd
Current assignee: Beijing Xingchen Tianhe Technology Co ltd
Priority date: 2019-08-02
Filing date: 2019-08-02
Publication date: 2021-02-19
Anticipated expiration: 2039-08-02
Also published as: CN110515543A

Abstract

The invention discloses a snapshot method, a snapshot device and a snapshot system based on an object bucket. Wherein, the method comprises the following steps: after receiving an object to be written, determining a target snapshot, wherein the target snapshot is a snapshot generated at last; adding version information of the target snapshot to the object to obtain the object carrying the version information; writing an object carrying version information into an object storage system, and generating metadata of the object, wherein the metadata of the object indicates that the object data belongs to the target snapshot. The invention solves the technical problem that no high-performance snapshot scheme exists in the field of object storage in the prior art.

Description

Object bucket-based snapshot method, device and system

Technical Field

The invention relates to the field of data storage, in particular to a snapshot method, a snapshot device and a snapshot system based on an object bucket.

Background

A snapshot is used to refer to a fully available copy of a given data set that includes an image of the corresponding data at some point in time (the point in time at which the copy began). Fig. 1a is a schematic diagram of a snapshot technique in the prior art, and in conjunction with fig. 1, along the direction of a time axis, a brief snapshot operation may include: data writes, create snapshot T1, modify data, roll back to T1, and so on.

The current implementation modes of the snapshot mainly include two types, one is copy-On-write (COW), and the other is redirect-On-write (ROW). Fig. 1b is a schematic diagram of COW, and referring to fig. 1b, after a snapshot is created, in a write operation process, if data of a source volume changes, a snapshot system will copy original data to a corresponding data block on the snapshot volume (see step 1 in fig. 1 b), and then rewrite the source volume (see step 2 in fig. 1 b). During this operation, at least a 2-fold write amplification of the write operation is caused. During the read operation, if the data to be read is in the snapshot, the data is directly read from the snapshot (see step 3 of fig. 1 b). Fig. 1c is a schematic diagram of how the ROW reads data, fig. 1d is a schematic diagram of how the ROW deletes data, and fig. 1c and 1d are assembled, in the ROW model, the incremental file is written into the snapshot. When reading, the incremental part is read from the snapshot, the unmodified part is read from the source, and the data segments (volumes) of the two areas are spliced into the complete target data.

In the deleting process of the ROW model, since the snapshot contains the incremental information and the incremental data has uniqueness, data refilling is required when the snapshot is deleted (as shown in fig. 1d, data B in the snapshot is refilled into the source volume), which results in a secondary write operation.

Therefore, both copy-on-write (COW) and redirect-on-write (ROW) have their disadvantages, but they cannot avoid copying, for example, a COW method needs to copy data when creating a snapshot, and a ROW method needs to copy data when deleting, so that the writing speed, reading speed, or deleting speed is low, and the snapshot has low performance.

Aiming at the problem that the snapshot has low performance due to the fact that the snapshot scheme in the prior art involves data migration and the writing speed, the reading speed or the deleting speed are low, an effective solution is not provided at present.

Disclosure of Invention

The embodiment of the invention provides a snapshot method, a snapshot device and a snapshot system based on an object bucket, which at least solve the technical problem that no high-performance snapshot scheme exists in the field of object storage in the prior art.

According to an aspect of the embodiments of the present invention, there is provided a snapshot method based on an object bucket, where the snapshot includes: the object snapshot method based on the object bucket comprises the following steps that metadata and object data are used, wherein the metadata are used for recording the link relation between snapshots, the link relation comprises the dependency relation of a child snapshot on a parent snapshot, the object data are used for recording objects belonging to the snapshots, and the object bucket-based snapshot method comprises the following steps: after receiving an object to be written, determining a target snapshot, wherein the target snapshot is a snapshot generated at last; adding version information of the target snapshot to the object to obtain the object carrying the version information; and writing the object carrying the version information into an object storage system, and generating metadata of the object, wherein the metadata of the object indicates that the object data belongs to the target snapshot.

Further, if the object data corresponding to the object does not exist in the target snapshot, the object carrying the version information is directly written into the object storage system, and the metadata of the object is generated; and if the object data corresponding to the object exists in the target snapshot, replacing the existing object in the object storage system with the object carrying the version information, and generating the metadata of the object.

Further, the method further comprises: receiving a first deleting instruction, and determining a snapshot to be deleted according to the first deleting instruction, wherein the first deleting instruction comprises identification information of the snapshot to be deleted; judging whether the snapshot to be deleted is referred by other snapshots; if the snapshot to be deleted is referred by other snapshots, deleting the metadata of the snapshot to be deleted, and keeping the object data of the snapshot to be deleted; if the snapshot to be deleted is referenced by other snapshots, the metadata of the snapshot to be deleted is marked as deleted.

Further, if the snapshot to be deleted is referenced by other snapshots, the parent snapshot of the child snapshot of the snapshot to be deleted is changed to the parent snapshot of the snapshot to be deleted after the metadata of the snapshot to be deleted is marked as deleted.

Further, the method further comprises: receiving a second deleting instruction, wherein the second deleting instruction is used for deleting an object to be deleted belonging to any snapshot; judging whether the original snapshot to which the object to be deleted belongs is referred by other snapshots; if the original snapshot is referred by other snapshots, adding the original snapshot in an invisible object list of the object to be deleted; and if the original snapshot is not referred by other snapshots, deleting the object data of the object to be deleted in the original snapshot.

Further, the method further comprises: receiving a rollback instruction, wherein the rollback instruction comprises an identifier of a target snapshot to be rolled back; and creating a rollback snapshot according to the rollback instruction, and determining the rollback snapshot as a sub-snapshot of the target snapshot.

Further, after a new snapshot is created according to the rollback instruction and the created new snapshot is determined as a sub-snapshot of the target snapshot, a creation instruction for creating the snapshot is received; detecting whether an object exists in the rollback snapshot; if the rollback snapshot has no object, deleting the rollback snapshot, and creating a sub-snapshot by taking the newly created non-rollback snapshot as a parent snapshot; and if the object exists in the rollback snapshot, the rollback snapshot is taken as the parent snapshot to create the child snapshot.

Further, the method further comprises: detecting creation times of two sub-snapshots having the same parent snapshot; if the creation time of the two sub-snapshots is less than a preset time threshold, merging the two sub-snapshots, and marking the two sub-snapshots as deleted, wherein the link relation between any one of the sub-snapshots and the parent snapshot is disconnected, and the sub-snapshot with the link relation with the parent snapshot disconnected is connected to the other sub-snapshot

According to an aspect of the embodiments of the present invention, there is provided an object bucket-based snapshot apparatus, where the snapshot includes: the device comprises metadata and object data, wherein the metadata is used for recording the link relation between snapshots, the link relation comprises the dependency relation of a child snapshot on a parent snapshot, the object data is used for recording objects belonging to the snapshots, and the object bucket-based snapshot device comprises: the determining module is used for determining a target snapshot after receiving the object to be written, wherein the target snapshot is the latest generated snapshot; the adding module is used for adding the version information of the target snapshot to the object to obtain the object carrying the version information; and the writing module is used for writing the object carrying the version information into the object storage system and generating the metadata of the object, wherein the metadata of the object indicates that the object data belongs to the target snapshot.

According to an aspect of an embodiment of the present invention, there is provided an object bucket-based snapshot system, the system including an object storage system for storing objects and at least one snapshot, the at least one snapshot including: metadata for recording a link relationship between the snapshots, the link relationship including a dependency relationship of the child snapshot on the parent snapshot; and the object data is used for recording the objects belonging to the snapshot.

Further, the metadata includes: a parent snapshot identification, a child snapshot identification, and an identification of a deleted parent snapshot.

According to an aspect of the embodiments of the present invention, there is provided a storage medium, where the storage medium includes a stored program, and when the program runs, a device on which the storage medium is located is controlled to execute the above-mentioned object bucket-based snapshot method.

According to an aspect of the embodiments of the present invention, there is provided a processor for executing a program, wherein the program executes the object bucket-based snapshot method as described in the right.

According to the embodiment of the invention, on the basis of ROW, the version information is added to the object names of all the uploaded objects under the snapshot to correlate the object data with the snapshot, so that all the operations after the snapshot is created can be performed on one object carrying the version information. That is, under each snapshot, all data is a separate copy, all associated with the snapshot version. According to the scheme, the data structures of the parent snapshot and the child snapshot are added to the snapshot information, so that when some snapshot information is read, splicing threading is carried out according to the link relation between the snapshots, each latest object in a link can be extracted from the snapshot information, and the zero-copy bucket snapshot capability in the field of object storage is realized. By the capacity, the space utilization of the system is maximized, no real data copy exists, the same data can be stored only one copy, and the data does not need to be migrated back and forth, so that the technical problem that the object storage field in the prior art does not have a high-performance snapshot scheme is solved.

Drawings

The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the invention and together with the description serve to explain the invention without limiting the invention. In the drawings:

FIG. 1a is a schematic diagram of a prior art snapshot technique;

FIG. 1b is a schematic illustration of COW;

FIG. 1c is a schematic diagram of ROW reading data;

FIG. 1d is a schematic diagram of ROW deleting data;

FIG. 2 is a flow diagram of a method for object bucket based snapshot in accordance with an embodiment of the present invention;

FIG. 3 is a schematic diagram of a memory structure according to an embodiment of the present invention;

FIG. 4 is a schematic diagram of deleting a snapshot according to an embodiment of the present invention;

FIG. 5 is a diagram of a delete object according to an embodiment of the present invention;

FIG. 6 is a diagram of a sub-snapshot count that is not zero, according to an embodiment of the invention;

FIG. 7 is a schematic illustration of a visual state of an object OBJ, in accordance with an embodiment of the present invention;

FIG. 8 is a schematic diagram of a tree relationship according to an embodiment of the present invention;

FIG. 9 is a schematic diagram of creating a snapshot and uploading an object, according to an embodiment of the present invention;

FIG. 10 is a schematic diagram rolled back to T2 on the basis of FIG. 9;

FIG. 11 is a schematic illustration of uploading object data in a rollback snapshot;

FIG. 12 is a schematic diagram of creating a snapshot after rolling back the snapshot, in accordance with an embodiment of the present invention;

FIG. 13 is a schematic diagram of another snapshot creation after rollback of a snapshot in accordance with an embodiment of the present invention;

FIG. 14 is a diagram of a first type of object list, according to an embodiment of the present invention;

fig. 15 is a schematic diagram of a first object bucket-based snapshot apparatus according to an embodiment of the present invention.

In the following, for the sake of understanding, the terms appearing in the present application are explained as follows:

and (4) storing the snapshot: with respect to a fully available copy of a given data set, the copy includes an image of the corresponding data at some point in time (the point in time at which the copy begins). The snapshot may be a copy of the data it represents or may be a replica of the data.

Bucket snapshot: bucket Snapshot, where a Bucket is intended to logically isolate objects in an object storage system, and a Bucket Snapshot is a fully available copy for generating all data and index sets of a target Bucket at a certain time, and the capacity of the target Bucket for corresponding all data at a certain time point (a time point of Snapshot creation) can be restored through the copy.

And (4) snapshot bucket: snapshot Bucket, an object Bucket (container) that enables Bucket Snapshot functionality.

And (3) snapshot rollback: snapshot Rollback, a Snapshot Rollback, is often applied to a scenario requiring data Rollback, such as data exception, error operation, business requirement, and the like. All metadata and object data after rollback will be updated to the state of the snapshot creation point in time, the purpose of this group of operations being to provide data backtracking capability. In the state, any uploading, deleting and other operations can be carried out on any object, and a new snapshot can be created to save the current state and be rolled back to any snapshot version.

And (4) visualization: visual, for any object created under the snapshot bucket, whether the object can be shown under a snapshot is represented by whether the object is visible or not. The visual state characterizes whether and how an object exists in a snapshot state.

Object version: object Version, for objects uploaded into the bucket, distinguishes homonym objects created/updated at different times with different versions. The uploading of the object under each snapshot obtains an object version matching the snapshot version. Object versions are distinguished by prefix/infix/suffix. Without creating a new snapshot, the object update operation will update with a new object containing the same version identification rather than generating a new version of the object. In other words, all objects under each snapshot have a dedicated version, such as T1_ object A, T2_ object A, T3_ object a.

And (3) read-only snapshot: the logic of read-only Snapshot falls back to a target Snapshot version. In this state, the downloaded object will have the object content of the target snapshot version.

Bucket Snapshot object List: and obtaining a visible Object List under the target Snapshot by using the Bucket Snapshot Object List.

Detailed Description

In order to make the technical solutions of the present invention better understood, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

It should be noted that the terms "first," "second," and the like in the description and claims of the present invention and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the invention described herein are capable of operation in sequences other than those illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.

Example 1

In accordance with an embodiment of the present invention, there is provided an embodiment of an object bucket based snapshot method, it should be noted that the steps illustrated in the flowchart of the accompanying drawings may be performed in a computer system such as a set of computer executable instructions, and that although a logical order is illustrated in the flowchart, in some cases the steps illustrated or described may be performed in an order different than here.

Fig. 2 is a flowchart of a snapshot method based on object buckets according to an embodiment of the present invention, where the snapshot includes: metadata and object data, wherein the metadata is used for recording a link relationship between snapshots, the link relationship includes a dependency relationship of a child snapshot on a parent snapshot, and the object data is used for recording objects belonging to the snapshots, as shown in fig. 2, the method includes the following steps:

step S202, after receiving the object to be written, determining target snapshots, wherein each snapshot is created as a sub-snapshot of the last snapshot, and the target snapshot is the last generated snapshot.

First, the above-described memory structure of the present application will be explained. In the storage structure of the application, the snapshot is a barrel snapshot, and the snapshot forms a chain structure with a parent-child relationship according to the generated time. Fig. 3 is a schematic diagram of a memory structure according to an embodiment of the present invention, and as shown in fig. 3, T1, T2, T3, and T4 are sequentially generated in time, so that a chain relationship of the structure is T1- - > T2- - > T3- - > T4. Besides the object data, the snapshot further includes metadata for describing the link relationship, and as shown in fig. 3, the metadata may exist in the form of a pointer, and includes the following information: parent snapshot, child snapshot, and deleted attachment snapshot. Because the complete snapshots are formed by splicing pointers of the metadata, each snapshot only stores the changed part under the snapshot, and low redundancy of the metadata is realized.

The last snapshot of each snapshot is the snapshot generated before each snapshot, and as shown in connection with fig. 3, the last snapshot of T2 is T1, the last snapshot of T3 is T2, and so on.

In the storage structure, after receiving the object to be written, it is determined that the snapshot generated last is the target snapshot, as shown in fig. 3, the current target snapshot is snapshot T5, and if the snapshot T4 is the target snapshot before the snapshot T5 is generated.

And step S104, adding the version information of the target snapshot to the object to obtain the object carrying the version information.

Specifically, the version information of the target snapshot may be a snapshot identifier of the target snapshot. Still referring to fig. 3, "T1", "T2", "T3" and "T4" are identifications of snapshots.

Adding version information to the object may be adding a prefix, a suffix, or a infix to the name of the object, or adding attribute information, that is, version information representing the target snapshot, to the object without changing the name of the object.

In an alternative embodiment, still referring to fig. 3, when an object OBJ is uploaded in snapshot T1, a prefix T1 may be added to the object to obtain object T1-OBJ. Similarly, when the object OBJ is uploaded to the snapshot T2, a prefix T2 is added to the object to obtain an object T2-OBJ; when the object OBJ is uploaded to the snapshot T3, a prefix T3 is added to the object, resulting in an object T3-OBJ. The above-mentioned T1, T2 and T3 are version information, and T1-OBJ, T2-OBJ and T3-OBJ are objects carrying version information.

Step S106, writing the object carrying the version information into an object storage system, and generating metadata of the object, wherein the metadata of the object indicates that the object data belongs to the target snapshot.

Still in the above embodiment of fig. 3, when an object OBJ is uploaded in the snapshot T1, a prefix T1 may be added to the object to obtain an object T1-OBJ, and then the object T1-OBJ is uploaded to the object storage system, and metadata of the object T1-OBJ is generated, where the metadata of the object T1-OBJ is used to indicate that the object T1-OBJ belongs to the snapshot T1. Similarly, when the object OBJ is uploaded to the snapshot T2, a prefix T2 is added to the object to obtain an object T2-OBJ, and then the object T2-OBJ is uploaded to the object storage system to generate metadata of the object T2-OBJ, wherein the metadata of the object T2-OBJ is used for indicating that the object T2-OBJ belongs to the snapshot T2; when the object OBJ is uploaded to the snapshot T3, a prefix T3 is added to the object to obtain an object T3-OBJ, then the object T3-OBJ is uploaded to an object storage system, metadata of the object T3-OBJ is generated, and the metadata of the object T3-OBJ is used for indicating that the object T3-OBJ belongs to the snapshot T3. The above-mentioned T1, T2 and T3 are version information, and T1-OBJ, T2-OBJ and T3-OBJ are objects carrying version information.

By the above method of uploading objects in snapshots, each object in each snapshot only generates one version, the latest version of the object is stored in the last snapshot, and the same object may generate multiple versions of data in multiple snapshots. Under the storage structure, when the object is read, the object is searched from back to front only according to the link relation between the snapshots. For example, an object OBJ is searched in a link relation T1- - > T2- - > T3- - > T4, where T4 does not include the object OBJ, so that the object OBJ is searched from a parent snapshot T3 of T4, and T3 includes T3-OBJ, so that data to be searched can be obtained by obtaining T3-OBJ from T3.

In the embodiment of the present application, on the basis of the ROW, version information is added to object names of all uploaded objects under a snapshot to associate object data with the snapshot, so that all operations after the snapshot is created can be performed on one object carrying the version information. That is, under each snapshot, all data is a separate copy, all associated with the snapshot version. According to the scheme, the data structures of the parent snapshot and the child snapshot are added to the snapshot information, so that when some snapshot information is read, splicing threading is carried out according to the link relation between the snapshots, each latest object in a link can be extracted from the snapshot information, and the zero-copy bucket snapshot capability in the field of object storage is realized. By the capacity, the space utilization of the system is maximized, no real data copy exists, the same data can be stored only one copy, and the data does not need to be migrated back and forth, so that the technical problem that the object storage field in the prior art does not have a high-performance snapshot scheme is solved.

As an alternative embodiment, writing the object carrying the version information into an object storage system, and generating metadata of the object includes: if the object data corresponding to the object does not exist in the target snapshot, directly writing the object carrying the version information into an object storage system, and generating metadata of the object; and if the object data corresponding to the object exists in the target snapshot, replacing the existing object in the object storage system with the object carrying the version information, and generating the metadata of the object.

An object may be uploaded multiple times before the next snapshot is generated, thus maintaining a snapshot that contains only one version of an object by means of overlay. Still referring to FIG. 3, for example, before T4 is generated, object OBJ needs to be uploaded, before version information T3 is added to OBJ, T3-OBJ is obtained, T3-OBJ is uploaded to the object storage system, and T3-OBJ belongs to T3. At this time, the snapshot T3 already contains the T3 version of the OBJ, and if the OBJ is continuously uploaded, the version information of the OBJ is still T3-OBJ, and the existing T3-OBJ is covered by using the new T3-OBJ, so that the latest version of the object is stored in the snapshot.

As an alternative embodiment, the method further includes: receiving a first deleting instruction, and determining a snapshot to be deleted according to the first deleting instruction, wherein the first deleting instruction comprises identification information of the snapshot to be deleted; judging whether the snapshot to be deleted is referred by other snapshots; if the snapshot to be deleted is referred by other snapshots, deleting metadata of the snapshot to be deleted, and keeping object data of the snapshot to be deleted, wherein the metadata is used for recording a parent snapshot and a child snapshot of the snapshot; if the snapshot to be deleted is not referenced by other snapshots, the metadata of the snapshot to be deleted is marked as deleted.

Specifically, the first deletion instruction is used to delete a snapshot, and whether the snapshot is referenced or not may be determined according to whether the snapshot has a sub-snapshot or not, and if the snapshot includes a sub-snapshot that depends on the snapshot, the snapshot is referenced by other snapshots, and if the snapshot does not include a sub-snapshot that depends on the snapshot, the snapshot is not referenced by other snapshots. Whether or not it is relied upon by other snapshots can be determined based on the link relationship.

If the snapshot needing to be deleted is not referenced by other snapshots, the object data in the snapshot can be deleted without influencing the data search.

In an alternative embodiment, FIG. 4 is a diagram illustrating a delete snapshot, according to an embodiment of the present invention. As shown in connection with FIG. 4, snapshot T3 contains sub-snapshot T4, which is not an isolated snapshot, so that if snapshot T3 needs to be deleted, only the metadata of T3 can be logically deleted. If it is snapshot T4 that needs to be deleted, the metadata and object data of snapshot T4 may be deleted because snapshot T4 has no child snapshots, i.e., snapshot T4 is not referenced by other snapshots.

Therefore, according to the scheme, data does not need to be migrated in the process of deleting the snapshot. The snapshot related information is only based on metadata, and the metadata is removed in a mode of picking a certain node like a linked list. For example: for example, the snapshots T1-T2-T3-T4 only need to reconstruct the index T1-T2- (T3) T4 after removing the T3, so that even if the snapshot is deleted, the data can still be read/writable immediately, and no long tail operation is caused by real IO.

As an alternative embodiment, if the snapshot to be deleted is referred to by other snapshots, after the metadata of the snapshot to be deleted is marked as deleted, the method further includes: and changing the parent node data of the child snapshot of the snapshot to be deleted into the parent node of the data to be deleted.

For the deleted snapshot, the child snapshot can not reference the deleted snapshot, so that the reference relationship of the child snapshot is changed, the child snapshot references the parent snapshot, and the link relationship is closed again.

Still referring to FIG. 3, since snapshot T3 was deleted, the corresponding modification in the metadata of T4 changed its parent snapshot to the parent snapshot T2 of T3, and the record in the deleted parent snapshot was T3. Before T3 is deleted, if an object OBJ needs to be retrieved, it is first searched from T4, and since there is no T4-OBJ of the latest version for OBJ in T4, it is necessary to read T3-OBJ in parent snapshot T3 of T4 when reading OBJ. After T3 delete, the parent snapshot and deleted parent snapshot are sorted: t2- - > T3 recursively generates a reverse linked list T1- - > T2- - > T3- - > T4, for an object which does not exist under T4, because T3 is deleted, a version of 'T2-xxx' is searched from T2, and if the T2 does not exist, the version is searched from T1 according to the linked list.

As an alternative embodiment, the method further includes: receiving a second deleting instruction, wherein the second deleting instruction is used for deleting an object to be deleted belonging to any snapshot; judging whether the original snapshot to which the object to be deleted belongs is referred by other snapshots; if the snapshot to which the object to be deleted belongs is referred by other snapshots, adding an original snapshot in an invisible object list of the object to be deleted; and if the snapshot to which the object to be deleted belongs is not referenced by other snapshots, deleting the object data of the object to be deleted in the original snapshot.

Specifically, the second deletion instruction is used to delete an object belonging to the snapshot, the original snapshot is used to represent the snapshot to which the object indicated in the second deletion instruction belongs, the invisible List is used to record the object whose visible state is invisible, and the invisible List may be a List, a set, a map, or other data structure, and only the snapshot set to the deletion state needs to be represented.

When the object is deleted, whether the original snapshot is referred to or not still needs to be determined, and if the original snapshot is not referred to, the object indicated by the second deletion instruction is isolated and can be deleted; if the original snapshot is referred to, the object indicated by the second deletion instruction is not isolated, so that the object can only be logically deleted, and the original snapshot is added to the invisible list of the deleted object.

Fig. 5 is a schematic diagram of an object deletion according to an embodiment of the present invention, and in conjunction with fig. 5, if the deleted object is T3, the sub-snapshot count of T3 is not 0 because T3 is referenced by T4, and if the deleted object is T4, the sub-snapshot count of T4 is 0 because T4 has no sub-snapshot.

Fig. 6 is a diagram illustrating that the sub-snapshot count is not zero, and in conjunction with fig. 6, T4 is a rollback snapshot of T3, and T6 is a rollback snapshot of T3. The data link of snapshot T5 is T5< -T4 < -T3 < -T2 < -T1. Taking deletion of T2-OBJ as an example, and T4 is a sub-snapshot of T3, so the invisible list of OBJ includes T4, and thus the visible state of the object OBJ shown in fig. 7 is obtained, i.e., T1, T2, and T3 are visible, deletion in T4, none in T5, i.e., snapshot T5 prohibits access to the OBJ. The data link of snapshot T6 is T6< -T3 < -T2 < -T1. T2-OBJ is exposed because any node in the link for T6 is not included in the invisible list of object OBJ.

All snapshot metadata is established to be tree-like based on the link relationship between the snapshots (as shown in fig. 8), that is, each snapshot may have at most one parent snapshot and several child snapshots.

As an alternative embodiment, the method further includes: receiving a rollback instruction, wherein the rollback instruction comprises an identifier of a target snapshot to be rolled back; and creating a rollback snapshot according to the rollback instruction, and determining the rollback snapshot as a sub-snapshot of the target snapshot.

Specifically, the above-mentioned rollback snapshot is a new snapshot created and arranged on the time axis after the previous snapshot, but the difference from the normal snapshot is that the parent snapshot of the rollback snapshot is the destination snapshot of the rollback, and not the previous snapshot on the time axis.

Fig. 9 is a schematic diagram of creating snapshots and uploading objects according to an embodiment of the present invention, and in conjunction with fig. 9, creating snapshots T1, T2, T3, and T4, uploading the same object OBJ under each snapshot, and carrying version information in a manner of adding a prefix, which are respectively named as: t1_ OBJ, T2_ OBJ, T3_ OBJ, T4_ OBJ. Fig. 10 is a schematic diagram of rolling back to T2 on the basis of fig. 9, and in conjunction with fig. 10, if rolling back from T4 to T2 is required, a rolling back snapshot T5 is automatically created, and T5 refers to snapshot T2, that is, T5 is a sub-snapshot of T2.

Fig. 11 is a schematic diagram of uploading object data in a rollback snapshot, and in connection with fig. 11, if an object OBJ is uploaded to the rollback snapshot, the naming may be T5-OBJ-a, and the suffix "a" may be used to indicate that objects in the snapshot are all objects that have been rolled back.

As an alternative embodiment, after creating a new snapshot according to the rollback instruction and determining the created new snapshot as a sub-snapshot of the destination snapshot, the method further includes: receiving a creation instruction for creating a snapshot; detecting whether an object exists in the rollback snapshot; if the rollback snapshot has no object, deleting the rollback snapshot and determining the created snapshot as a sub-snapshot of the recently created non-rollback snapshot; and if the object exists in the rollback snapshot, determining that the created snapshot is a sub-snapshot of the rollback snapshot.

In the above steps, the creation of the new snapshot is continued after the snapshot is rolled back, and the parent snapshot of the created new snapshot needs to be determined before the new snapshot is created. If the rollback snapshot has an object, the rollback snapshot is established as a parent snapshot, if the rollback snapshot has no object, the rollback snapshot can be deleted, and a non-rollback snapshot which is established recently is determined as the parent snapshot.

In an alternative embodiment, fig. 12 is a schematic diagram of creating a snapshot after the rollback snapshot according to an embodiment of the present invention, and in conjunction with fig. 12, T5 is the rollback snapshot rolled back to T2, and T5-OBJ is uploaded in the rollback snapshot T5, that is, there is an object in the rollback snapshot, so when a new snapshot is created and T5 is used as a parent snapshot, snapshot T6 is obtained. Fig. 13 is another schematic diagram of creating a snapshot after the snapshot is rolled back, and referring to fig. 13, after T6, rolling back to T3, i.e., a rolled back snapshot T7 can be created, the parent snapshot of which is T3, no object exists in T7, and if the snapshot is created after T7, T7 can be deleted, created directly, and T3 is taken as the snapshot of the parent snapshot.

Still referring to fig. 13, when reading an object, an association list is generated: t1- > T2- > T3- > T7, wherein, T3-T7 has no object, T2-T3 has visible object T2_ OBJ, T1-T2 has visible object T1_ OBJ, T1_ OBJ is an old version of OBJ, and T2_ OBJ is a new version of OBJ, so the visible object after rollback is T2_ OBJ. Its object list may be as shown in fig. 14, the same object takes the latest version when read, so the object shown is OBJ (version T3), i.e. the T3 snapshot creates the uploaded version T2_ OBJ.

As an alternative embodiment, after receiving a creation instruction for creating a snapshot, the method further includes: detecting creation times of two sub-snapshots having the same parent snapshot; and if the creation time of the two sub-snapshots is less than a preset time threshold, merging the two sub-snapshots, and marking the two sub-snapshots as deleted, wherein the link relation between any one of the sub-snapshots and the parent snapshot is disconnected, and the sub-snapshot with the link relation with the parent snapshot disconnected is connected to the other sub-snapshot.

Specifically, the preset time threshold may be a default time threshold, or may be set by the user according to actual needs. When two child snapshots with the parent are taken, the merging of the snapshots can be performed not only based on the judgment of time, but also under the trigger of a user.

In the above steps, the merging of the snapshots is implemented based on data binary merging, and is used to merge two or more snapshots with the snapshot of the parent snapshot to obtain an aggregated snapshot, and perform logical deletion on two child snapshots, after the logical deletion, the actual addressing mode is unchanged, but the logically deleted objects are no longer displayed to the user. When the two snapshots are merged, any two snapshots with a common parent snapshot are disconnected, and one snapshot is linked to the other snapshot to serve as a child version of the other snapshot. After the two snapshots are merged, pairwise merging can be continuously performed by using the method until the two snapshots are merged into one snapshot. Through the merging of snapshots, the aim of facilitating metadata management is fulfilled.

Example 2

According to an embodiment of the present invention, an embodiment of a snapshot apparatus based on an object bucket is provided, and fig. 15 is a schematic diagram of a first snapshot apparatus based on an object bucket according to an embodiment of the present invention, where a snapshot includes: as shown in fig. 15, the apparatus includes:

the determining module 150 is configured to determine a target snapshot after receiving the object to be written, where the target snapshot is a snapshot that is generated last.

The adding module 152 is configured to add version information of the target snapshot to the object to obtain the object with the version information.

A writing module 154, configured to write an object carrying version information into an object storage system, and generate metadata of the object, where the metadata of the object indicates that the object data belongs to the target snapshot.

As an alternative embodiment, the writing module includes: the writing sub-module is used for directly writing the object carrying the version information into an object storage system and generating metadata of the object if the object data corresponding to the object does not exist in the target snapshot; and the replacing sub-module is used for replacing the existing object in the object storage system by using the object carrying the version information if the object data corresponding to the object exists in the target snapshot and generating the metadata of the object.

As an alternative embodiment, the apparatus further comprises: the first receiving module is used for receiving a first deleting instruction and determining the snapshot to be deleted according to the first deleting instruction, wherein the first deleting instruction comprises identification information of the snapshot to be deleted; the first judgment module is used for judging whether the snapshot to be deleted is quoted by other snapshots; the first deleting module is used for marking the metadata of the snapshot to be deleted as deleted if the snapshot to be deleted is referred by other snapshots; and the second deleting module is used for deleting the metadata and the object data of the snapshot to be deleted if the snapshot to be deleted is not referred by other snapshots.

As an alternative embodiment, the apparatus further comprises: and the changing module is used for changing the parent snapshot of the child snapshot of the snapshot to be deleted into the parent snapshot of the snapshot to be deleted after the metadata of the snapshot to be deleted is marked as deleted if the snapshot to be deleted is referred by other snapshots.

As an alternative embodiment, the apparatus further comprises: the second receiving module is used for receiving a second deleting instruction, wherein the second deleting instruction is used for deleting an object to be deleted belonging to any snapshot; the second judgment module is used for judging whether the original snapshot to which the object to be deleted belongs is referred by other snapshots; the third deleting module is used for adding the original snapshot into the invisible object list of the object to be deleted if the original snapshot is referred by other snapshots; and the fourth deleting module is used for deleting the object data of the object to be deleted in the original snapshot if the original snapshot is not referred by other snapshots.

As an alternative embodiment, the apparatus further comprises: a third receiving module, configured to receive a rollback instruction, where the rollback instruction includes an identifier of a target snapshot to be rolled back; and the determining module is used for creating the rollback snapshot according to the rollback instruction and determining the rollback snapshot as a sub-snapshot of the target snapshot.

As an alternative embodiment, the apparatus further comprises: after a new snapshot is created according to the rollback instruction and the created new snapshot is determined as a sub-snapshot of the target snapshot, a fourth receiving module is used for receiving a creation instruction for creating the snapshot; the first detection module is used for detecting whether an object exists in the rollback snapshot; the first establishing module is used for deleting the rollback snapshot if no object exists in the rollback snapshot and establishing a child snapshot by taking the newly established non-rollback snapshot as a parent snapshot; and the second creating module is used for creating the child snapshot by taking the rollback snapshot as the parent snapshot if the rollback snapshot has the object.

As an alternative embodiment, the apparatus further comprises: the second detection module is used for detecting the creation time of two sub-snapshots with the same parent snapshot; and the merging module is used for merging the two sub-snapshots and marking the two sub-snapshots as deleted if the creation time of the two sub-snapshots is less than a preset time threshold, wherein the link relation between any one of the sub-snapshots and the parent snapshot is disconnected, and the sub-snapshot with the link relation with the parent snapshot disconnected is connected to the other sub-snapshot.

Example 3

According to an embodiment of the present invention, there is provided an embodiment of an object bucket-based snapshot system, the system including an object storage system for storing objects and at least one snapshot, the at least one snapshot including: metadata for recording a link relationship between the snapshots, the link relationship including a dependency relationship of the child snapshot on the parent snapshot; and the object data is used for recording the objects belonging to the snapshot.

As an alternative embodiment, the metadata includes: a parent snapshot identification, a child snapshot identification, and an identification of a deleted parent snapshot.

The snapshot system based on the object bucket in this embodiment may include the snapshot in embodiment 1, and specific embodiments are described in embodiment 1, and are not described here again.

The snapshot system based on the object bucket in this embodiment of the application may perform the steps in embodiment 1, which is not described herein again.

Example 4

According to an embodiment of the present invention, a storage medium is provided, where the storage medium includes a stored program, and when the program runs, a device in which the storage medium is located is controlled to execute the object bucket-based snapshot method according to any one of embodiments 1.

Example 5

According to an embodiment of the present invention, there is provided a processor configured to execute a program, where the program executes the object bucket-based snapshot method according to any one of embodiments 1 during runtime.

The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.

In the above embodiments of the present invention, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.

In the embodiments provided in the present application, it should be understood that the disclosed technology can be implemented in other ways. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units may be a logical division, and in actual implementation, there may be another division, for example, multiple units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, units or modules, and may be in an electrical or other form.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.

The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a removable hard disk, a magnetic or optical disk, and other various media capable of storing program codes.

The foregoing is only a preferred embodiment of the present invention, and it should be noted that, for those skilled in the art, various modifications and decorations can be made without departing from the principle of the present invention, and these modifications and decorations should also be regarded as the protection scope of the present invention.

Claims

1. An object bucket based snapshot method, wherein the snapshot comprises: the object bucket-based snapshot method comprises the following steps that metadata and object data are obtained, wherein the metadata are used for recording link relations among snapshots, the link relations comprise dependency relations of sub-snapshots on parent snapshots, the object data are used for recording objects belonging to the snapshots, and the object bucket-based snapshot method comprises the following steps:

after receiving an object to be written, determining a target snapshot, wherein the target snapshot is a snapshot generated at last;

adding version information of the target snapshot to the object to obtain an object carrying the version information;

writing the object carrying the version information into an object storage system, and generating metadata of the object, wherein the metadata of the object indicates that the object data belongs to the target snapshot;

writing the object carrying the version information into an object storage system, and generating metadata of the object, including: if the object data corresponding to the object does not exist in the target snapshot, directly writing the object carrying the version information into an object storage system, and generating metadata of the object; if the object data corresponding to the object exists in the target snapshot, replacing the existing object in the object storage system with the object carrying the version information, and generating metadata of the object;

the method further comprises the following steps: receiving a rollback instruction, wherein the rollback instruction comprises an identifier of a target snapshot to be rolled back; creating a rollback snapshot according to the rollback instruction, and determining the rollback snapshot to be a sub-snapshot of the target snapshot;

after creating a new snapshot according to the rollback instruction and determining that the created new snapshot is a sub-snapshot of the destination snapshot, the method further includes: receiving a creation instruction for creating a snapshot; detecting whether an object exists in the rollback snapshot; if the rollback snapshot has no object, deleting the rollback snapshot, and creating a child snapshot by taking the newly created non-rollback snapshot as a parent snapshot; and if the rollback snapshot has an object, taking the rollback snapshot as a parent snapshot to create a child snapshot.

2. The method of claim 1, further comprising:

receiving a first deleting instruction, and determining a snapshot to be deleted according to the first deleting instruction, wherein the first deleting instruction comprises identification information of the snapshot to be deleted;

judging whether the snapshot to be deleted is referred by other snapshots;

if the snapshot to be deleted is referred to by other snapshots, marking the metadata of the snapshot to be deleted as deleted;

and if the snapshot to be deleted is not referred by other snapshots, deleting the metadata and the object data of the snapshot to be deleted.

3. The method of claim 2, wherein if the snapshot to be deleted is referenced by other snapshots, after marking metadata of the snapshot to be deleted as deleted, the method further comprises:

and changing the parent snapshot of the child snapshot of the snapshot to be deleted into the parent snapshot of the snapshot to be deleted.

4. The method of claim 1, further comprising:

receiving a second deleting instruction, wherein the second deleting instruction is used for deleting an object to be deleted belonging to any snapshot;

judging whether the original snapshot to which the object to be deleted belongs is referred by other snapshots;

if the original snapshot is referred by other snapshots, adding the original snapshot to an invisible object list of the object to be deleted;

and if the original snapshot is not referred by other snapshots, deleting the object data of the object to be deleted in the original snapshot.

5. The method of claim 1, further comprising:

detecting creation times of two sub-snapshots having the same parent snapshot;

and if the creation time of the two sub-snapshots is less than a preset time threshold, merging the two sub-snapshots, and marking the two sub-snapshots as deleted, wherein the link relation between any one of the two sub-snapshots and the parent snapshot is disconnected, and the sub-snapshot with the link relation with the parent snapshot is connected to the other sub-snapshot.

6. An object bucket based snapshot apparatus, wherein the snapshot comprises: the device comprises metadata and object data, wherein the metadata is used for recording a link relation between snapshots, the link relation comprises a dependency relation of a child snapshot on a parent snapshot, the object data is used for recording objects belonging to the snapshots, and the object bucket-based snapshot device comprises:

the device comprises a determining module, a judging module and a sending module, wherein the determining module is used for determining a target snapshot after receiving an object to be written, and the target snapshot is a snapshot generated finally;

the adding module is used for adding the version information of the target snapshot to the object to obtain the object carrying the version information;

a writing module, configured to write an object carrying the version information into an object storage system, and generate metadata of the object, where the metadata of the object indicates that the object data belongs to the target snapshot;

further comprising: receiving a rollback instruction, wherein the rollback instruction comprises an identifier of a target snapshot to be rolled back; creating a rollback snapshot according to the rollback instruction, and determining the rollback snapshot to be a sub-snapshot of the target snapshot;

7. An object bucket based snapshot system, the system comprising an object storage system for storing objects and at least one snapshot, the at least one snapshot comprising:

metadata for recording a link relationship between the snapshots, wherein the link relationship comprises a dependency relationship of the child snapshot on the parent snapshot;

object data for recording objects belonging to the snapshot;

8. The system of claim 7, wherein the metadata comprises: a parent snapshot identification, a child snapshot identification, and an identification of a deleted parent snapshot.

9. A storage medium comprising a stored program, wherein the program, when executed, controls a device on which the storage medium is located to perform the object bucket-based snapshot method of any one of claims 1 to 5.

10. A processor, configured to execute a program, wherein the program runtime performs the object bucket based snapshot method of any one of claims 1 to 5.