CN115686356A - Clone splitting method and cluster of storage volume - Google Patents
Clone splitting method and cluster of storage volume Download PDFInfo
- Publication number
- CN115686356A CN115686356A CN202211173256.3A CN202211173256A CN115686356A CN 115686356 A CN115686356 A CN 115686356A CN 202211173256 A CN202211173256 A CN 202211173256A CN 115686356 A CN115686356 A CN 115686356A
- Authority
- CN
- China
- Prior art keywords
- clone
- volume
- metadata
- split
- log
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Abstract
The method comprises the steps that a device node where a source volume is located responds to a clone splitting request aiming at a clone volume, the clone splitting request is determined and stored in a distributed log system shared by the bottom layer of a cluster, metadata to be split which is depended on by the clone volume and is contained in the source volume are controlled by the distributed log system to generate a split log object containing the metadata to be split, an object identifier of the split log object is sent to the device node where the clone volume is located, the content of the split log object does not need to be copied and transmitted to the clone volume, the clone splitting efficiency and performance are greatly improved, network resources consumed by data transmission are saved, the device where the clone volume is located writes the received object identifier into the bottom layer of a multilayer data structure of the device where the clone volume is located, the device does not have mutual exclusion with a service IO, blocking of the service IO is thoroughly avoided, and the service processing efficiency and reliability are improved.
Description
Technical Field
The present application relates generally to the field of computer technologies, and more particularly, to a method and a cluster for clone splitting of a storage volume.
Background
The link cloning technology has been widely applied to various clusters, and as shown in the schematic diagram of the link cloning technology shown in fig. 1, a clone volume may be created in a device node of a cluster in a snapshot manner, so that the clone volume may share at least part of data and metadata of a corresponding source volume, thereby saving storage space. However, since the clone volume depends on the source volume, the reliability and availability of the data in the clone volume are limited by the source volume, and in the service data reading application, the read request may need to be forwarded to the source volume, which reduces the service data reading performance.
On the contrary, in order to improve data reliability or in a scenario where a source volume needs to be deleted, a clone splitting operation manner as shown in fig. 2 is proposed, in which metadata and data shared by clone volumes in the source volume are read and written into the clone volumes to become metadata and data of the clone volumes, so that the metadata and data between the clone volumes and the source volume are independent, but this clone splitting manner takes a lot of time and may also block data read-write services.
Disclosure of Invention
In order to solve the above problem, in one aspect, the present application provides a method for clone splitting of a storage volume, the method comprising:
obtaining a clone splitting request for a clone volume;
responding the clone splitting request, and determining metadata to be split; the metadata to be split refers to metadata which is depended by the clone volume and is possessed by a source volume, and the metadata is stored in a distributed log system;
controlling the distributed log system to generate a split log object aiming at the clone volume, and obtaining an object identifier of the split log object; the split log object contains the metadata to be split;
and sending the object identification to a device node where the clone volume is located, and writing the object identification into the bottom layer of a multilayer data structure of the clone volume by the device node.
Optionally, each layer of metadata of the multi-layer data structure of the source volume is stored in a corresponding metadata log object of the distributed log system;
the step of responding the clone splitting request and determining the metadata to be split comprises the following steps:
determining metadata to be split according to the dependency relationship between the source volume and the clone volume; and the metadata to be split is stored in a metadata log object corresponding to a depended layer of the source volume.
Optionally, the controlling the distributed log system to generate a split log object includes:
reading the determined metadata to be split;
and sending the metadata to be split to the distributed log system, and generating a log object to be split containing the metadata to be split by the distributed log system.
Optionally, the controlling the distributed log system to generate a split log object further includes:
determining that the metadata to be split meets an integration condition, and performing layered integration processing on the metadata to be split;
sending the to-be-split metadata to the distributed logging system comprises:
and sending the integrated metadata to be split to the distributed log system.
Optionally, the obtaining the object identifier of the split log object includes:
receiving an object identifier fed back after the distributed log system is generated; the object identification is used to identify the split log object.
Optionally, the obtaining a clone split request for a clone volume includes:
receiving a clone splitting request sent by a device node where a clone volume is located; alternatively, the first and second liquid crystal display panels may be,
determining that a source volume is depended on by at least one clone volume, and obtaining a clone split request for the selected clone volume.
In another aspect, the present application further provides a method for clone splitting of a storage volume, the method comprising:
obtaining an object identification for a split log object of a clone volume; the split log object is generated by a distributed log system controlled by a device node where a source volume is located in response to a clone split request aiming at the clone volume, and comprises metadata which is depended by the clone volume and is possessed by the source volume, and the metadata is stored in the distributed log system;
writing the object identification to a bottom layer of a multi-layered data structure of the clone volume.
Optionally, in the process of responding to the clone split request, the method further includes:
obtaining a data write request for the clone volume;
sending the requested metadata to be written to the distributed journal system, and writing the received metadata to a metadata journal object corresponding to the top layer of the multi-layer data structure of the clone volume by the distributed journal system.
Optionally, the method further includes:
obtaining a data access request for the clone volume;
according to the sequence of the multilayer data structure of the clone volume from the top layer to the bottom layer, sending a data query request containing an object identifier recorded by a corresponding layer to the distributed log system, and querying whether metadata to be accessed exists in a metadata log object with the object identifier by the distributed log system;
and receiving metadata to be accessed fed back by the distributed log system.
In yet another aspect, the present application further provides a cluster, including: a plurality of appliance nodes and a distributed logging system to which the plurality of appliance nodes are connected, the appliance nodes having at least one storage volume deployed therein, wherein:
the equipment node is used for realizing the clone splitting method of the storage volume.
In still another aspect, the present application further provides a computer-readable storage medium, on which a computer program is stored, where the computer program is loaded and executed by a processor to implement the above-mentioned clone splitting method for a storage volume.
Therefore, the method and the cluster for the clone splitting of the storage volume are provided, a device node where a source volume is located responds to a clone splitting request aiming at the clone volume, the device node where the source volume is located determines and stores the data in a distributed log system shared by the bottom layer of the cluster, the source volume has to-be-split metadata depended on by the clone volume, the distributed log system is controlled to generate a split log object containing the to-be-split metadata, an object identifier of the split log object is sent to a device node where the clone volume is located, the content of the split log object does not need to be copied and transmitted to the clone volume, the clone splitting efficiency and performance are greatly improved, network resources consumed by data transmission are saved, the device where the clone volume is located writes the received object identifier into the bottom layer of a multilayer data structure of the device node where the clone volume is located, the device node does not mutually exclusive with service IO, blocking of the service IO is thoroughly avoided, and the service processing efficiency and reliability are improved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the provided drawings without creative efforts.
FIG. 1 is a schematic diagram of a scenario in which a clone volume depends on a source volume;
FIG. 2 is a schematic flow diagram of an alternative method of clonal disruption;
FIG. 3 is a schematic flow chart diagram of an alternative example of a method for clone splitting a storage volume as set forth herein;
FIG. 4 is a schematic flow chart diagram of yet another alternative example of a method for clone splitting of storage volumes as presented herein;
FIG. 5 is a schematic diagram illustrating an application scenario of a source volume and a clone volume suitable for the clone splitting method for a storage volume proposed in the present application;
FIG. 6 is a schematic flow chart diagram of yet another alternative example of a method for clone splitting of storage volumes as presented herein;
FIG. 7 is a schematic flow chart diagram of yet another alternative example of a method for clone splitting of storage volumes as presented herein;
FIG. 8 is a schematic diagram of an alternative example of a clone splitting apparatus for storage volumes as disclosed herein;
FIG. 9 is a schematic structural diagram of yet another alternative example of a clone splitting apparatus for storage volumes as set forth in the present application;
FIG. 10 is a schematic diagram of a system architecture of an alternative example of a cluster suitable for the clone splitting method for storage volumes set forth herein.
Detailed Description
The clone splitting scheme described in the background section is that after being dependent on metadata and data of a source volume, a clone volume is copied into the clone volume to become data and metadata of the clone volume, so that the clone volume and the source volume are completely split, and the clone volume has independent metadata and data and is not dependent on the source volume. However, such a way of copying metadata and data is time-consuming, especially in distributed cluster application, in consideration of load and capacity balancing issues, a clone volume is usually created on another device node different from the device node where the source volume is located, so that when the clone splitting operation is performed, data and metadata of the source volume need to be copied across the device nodes, which greatly increases the clone splitting time.
In order to improve the above problem, it is proposed to write data of each storage volume in a device node to a shared storage space so that the data meets reliability and availability requirements, so that when a clone is split, the data does not need to be copied, but the metadata still needs to be copied. When copying metadata, in order to avoid overwriting new data written by the service, it is necessary to check whether the new data has been written by the service of the clone volume, if not, the copied metadata is saved, otherwise, the corresponding part of metadata is discarded. Therefore, the clone splitting method still takes time and repeatedly and temporarily blocks data read-write service.
In order to further improve the above problem, the metadata of each storage volume may be managed by using a multi-layer data structure, such as a data structure of a Log Structured Merge Tree (LSM Tree), and the metadata of each storage volume is stored in the bottom layer by using a distributed Log system, so that when performing a clone splitting operation, only the original metadata of the clone volume, that is, the metadata that is dependent on the clone volume and is possessed by the source volume, is the metadata to be split, and then a new metadata Log object is generated in the distributed Log system to store the read metadata to be classified, so that the metadata independent from the source volume and the metadata independent from the clone volume are stored in the distributed Log system, and then, the device node where the source volume is located only needs to send the object identifier of the newly generated split Log object containing the metadata to be split to the clone volume, so as to quickly complete the clone splitting, thereby avoiding the data and metadata of the storage volume from being copied across nodes, and greatly improving the clone splitting efficiency and performance.
Moreover, the device node where the clone volume is located writes the received object identifier into the bottom layer of the multilayer data structure as the old data stored by the clone volume, so that the business processing process of writing/reading data into/from the top layer of the multilayer data structure of the clone volume is not interfered, and zero blocking of business IO is realized.
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Referring to fig. 3, a schematic flowchart of an optional example of the method for clone splitting a storage volume provided in the present application may be applied to an equipment node where a source volume is located, and in practical applications, a clone volume may be a new storage volume created for any storage volume or multiple storage volumes (which may be referred to as a source volume depended by the clone volume) in the equipment node of a cluster. The clone volume and the source volume can be deployed in the same equipment node, load and capacity balance is considered, the clone volume and the source volume can also be deployed in different equipment nodes, the clone volume and the source volume can be determined according to a cluster application scene, and the equipment deployment relation of the clone volume and the source volume is not limited.
Based on the above analysis, as shown in fig. 3, the method for clone splitting of a storage volume according to the embodiment of the present application may include, but is not limited to:
step S31, obtaining a clone splitting request aiming at a clone volume;
in practical applications, in order to improve data reliability or delete a source volume, the source volume and the clone volume dependent on the source volume in the cluster need to be separated, so that the source volume and the clone volume both have their own independent data and metadata, and a clone splitting operation may be performed on the device node where the source volume is located, for example, one or more clone volumes dependent on the source volume are selected to perform a splitting operation, so that the device node where the source volume is located makes a clone splitting request for the selected clone volume. As can be seen, step S31 may include determining that the source volume is depended on by at least one clone volume, obtaining a clone split request for the selected clone volume.
In still other embodiments, in a scenario that the clone volume and the source number are deployed in different device nodes, the clone volume that needs to be split may also be directly selected, a splitting operation is performed on the clone volume, and a generated clone split request for the clone volume is sent to a device node where the source volume that the clone volume depends on is located, so that step S31 may be: the device node where the source volume is located receives the clone split request sent by the device node where the clone volume is located, but is not limited to the several ways of obtaining the clone split request described in this application.
It should be noted that, the generation triggering manner of the clone split request is not limited in the present application, that is, the splitting operation may be a user who uses the clone volume or the equipment node where the source volume is located, selects a clone volume to be split or a source volume that depends on the clone volume from storage volumes deployed by the equipment node displayed on the storage volume operation interface, and triggers the split button to generate the clone split request for the selected clone volume, but is not limited to this operation implementation manner.
Optionally, the management device may also be configured to send the clone splitting request to the device node, for example, the clone splitting request is generated according to an operation of a manager or generated based on a service processing requirement, and the implementation process of step S31 may be determined according to an actual situation, which is not described in detail herein.
Step S32, responding to the clone splitting request, and determining metadata to be split;
in conjunction with the above description related to the technical solution of the present application, metadata for each storage volume in each device node may be stored in the underlying distributed log system. The distributed Log system can be a distributed Log storage system (recorded as LSS) sharing Log structured, any equipment node writes data and metadata of the LSS, other equipment nodes can access the data and the metadata, the metadata of each storage volume can be managed by different nodes according to a certain rule, and the data storage mode of the distributed Log system is not limited by the application.
Based on this, the device node where the source volume is located may determine which clone volume and the source volume on which the clone volume depends the clone volume that is requested to be split, and then may determine the metadata to be split that the source volume has, that is, the metadata that the source volume has and is depended on by the clone volume according to the dependency relationship between the clone volume and the source volume, and as described above, the metadata is stored in the distributed log system LSS. The determination implementation process of the metadata to be split is not described in detail. It will be appreciated that in actual practice, the source volume and clone volume currently share the metadata to be split.
Step S33, controlling the distributed log system to generate a split log object aiming at the clone volume, and obtaining an object identifier of the split log object;
according to the method and the device, the clone volume needs to have independent metadata, under the condition that the processing service of the source volume is not influenced, the device node where the source volume is located can read the determined metadata to be split from the distributed log system, a new metadata log object (log object) can be generated in the distributed log system, the read metadata to be split is written into the new metadata log object, and a corresponding split log object is obtained, namely the split log object newly generated by the distributed log system contains the determined metadata to be split.
It should be understood that after the splitting operation of the metadata depended on by the cloned volume in the source volume is completed in the above manner, the metadata to be split depended on by the cloned volume is still included in the metadata log object stored in the distributed log system and corresponding to each layer depended on by the cloned volume in the source volume, so as to meet the service processing requirement of the source volume. That is, the clone splitting is successful, the metadata and the data of the clone volume will not depend on the source volume, that is, the source volume and the clone volume both have their own independent data and metadata, and the splitting process of the data can be referred to the description of the corresponding parts of the above embodiments, and the embodiments of the present application can describe the metadata splitting process.
In order to facilitate identification of each metadata log object stored in the distributed log system, when the metadata log object is generated, one-to-one corresponding object identifier, such as an object ID, may be generated.
And step S34, sending the object identification to the equipment node where the clone volume is located, and writing the object identification into the bottom layer of the multilayer data structure of the clone volume by the equipment node.
After the device node where the source volume is located obtains the object identifier of the split log object according to the method, the object identifier can be pushed to the device node where the clone volume is located, and the clone volume adds the object identifier to the lowest layer of the LSM tree of the clone volume. Therefore, in the processing process, even in the cross-node application, the device node where the source volume is located only needs to send the obtained object identifier (such as the object ID) to the device node where the clone volume is located, so that the clone volume can access the metadata contained in the split log object according to the object identifier, the content of the split log object does not need to be copied to the clone volume, the data transmission speed of the cross-node is greatly shortened, and the clone splitting efficiency is improved.
Moreover, since the multi-layer data structure of the storage node (e.g., a log-structured Merge tree such as an LSM tree) may be a hierarchically merged (i.e., merge) data management structure, the latest data is always written into the top-most layer of the data structure, and under certain conditions, the latest data may be merged from the top layer to the bottom layer. Thus, data closer to the top layer is newer, and data closer to the bottom layer is older. Thus, when accessing the LSM tree of the storage volume, the accessed data is queried from the top layer, and if the accessed data is queried at the top layer, the access is completed; if the data to be accessed is not inquired at the top layer, inquiring the next layer by layer until the data to be accessed is inquired, and finishing the data access.
Therefore, in the embodiment of the application, because the to-be-split metadata contained in the split log object is not the latest data, after the device node where the clone volume is located obtains the object identifier of the split log object, the object identifier can be written into the bottommost layer of the clone volume, so that the service cannot access the old data, the condition that whether the updated metadata exists or not is inquired is avoided, the updated metadata cannot be mutually exclusive with the service IO, the service IO is completely prevented from being blocked, and the service processing efficiency and the service processing reliability are improved.
It should be understood that, in a scenario where the source volume and the clone volume to be split are deployed in the same device node, in response to a splitting requirement of the clone volume, after the device node obtains an object identifier of the split log object according to the method described above, step S34 may write the object identifier into the bottom layer of the multi-layer data structure of the clone volume, and an implementation process is not described in detail.
Referring to fig. 4, which is a schematic flow chart of yet another optional example of the clone splitting method for a storage volume provided in the present application, this embodiment may describe an optional detailed implementation procedure of the clone splitting method for a storage volume described above, and still describe from the device node side where the source volume is located, as shown in fig. 4, the method may include:
step S41, obtaining a clone splitting request aiming at the clone volume;
s42, determining metadata to be split according to the dependency relationship between the source volume and the clone volume;
in the embodiment of the present application, in combination with the above description, each layer of metadata of a multi-layer data structure (e.g., a log tree structure such as an LSM tree) of a source volume is stored in a corresponding metadata log object of an LSS of a distributed log system, and details about a storage implementation process of the metadata are not described in detail. Therefore, the metadata to be split determined by the device node where the source volume is located is stored in the metadata log object corresponding to the depended layer of the source volume.
Illustratively, referring to the clone splitting scenario diagram shown in fig. 5, the metadata of the clone volume and the source volume are managed using an LSM tree (i.e., a log structure merge tree), and assuming that there are 4 layers of metadata of the source volume, denoted as L0, L1, L2, and L3, at this time, the metadata of the clone volume itself is only the top layer, i.e., L0, and the metadata of other layers depends on the corresponding layers of the source volume. And the data of each LSM tree is stored in a distributed log system LSS shared by the bottom layer. It should be noted that the number of LSM trees of each storage volume may be determined according to actual situations, and is not limited to the four-layer data structure shown in the drawing.
For the metadata of each storage volume, for each layer of the multi-layer data structure of the storage volume, a corresponding metadata log object may be generated in the distributed log system to store the metadata of the layer, and an object identifier of the metadata log object is written into the corresponding layer of the storage volume, so as to identify the metadata log object of the corresponding layer stored in the distributed log system according to the object identifier, and read the metadata to be read stored in the metadata log object. Similarly, for the data of each storage volume stored in the distributed log system, a corresponding object can be generated to store the data of the corresponding volume, and the implementation process is similar to that of the present application and is not described in detail.
Based on this, the device node may determine, according to the dependency relationship between the source volume and the clone volume (for example, a mapping/referencing manner is not limited by the present application to the representation manner of the dependency relationship, which may be determined as the case may be), a hierarchy in the multilayer data structure of the source volume on which the cloned volume depends, and then may determine metadata included in a metadata log object corresponding to the hierarchy as metadata to be split for the clone volume; or, when the clone volume refers to a part of metadata of the layer corresponding to the source volume, the referenced metadata included in the metadata log object corresponding to the dependent layer may be further determined according to the dependency relationship between the clone volume and the source volume, and is recorded as the metadata to be split, and the like.
Step S43, reading the determined metadata to be split;
after the above analysis, the device node where the source volume is located determines, according to but not limited to the above method, to-be-split metadata that the source volume in the distributed log system depends on by the cloned volume, the device node may send a metadata read request for the to-be-split metadata to the distributed log system, and the distributed log system responds to the metadata read request, reads the to-be-split metadata, and feeds the to-be-split metadata back to the device node.
The metadata reading request may include an object identifier of a hierarchical record on which a cloned volume of the source volume depends, and may also include a dependency relationship between the cloned volume and the source volume as needed, so that the distributed journal system analyzes the metadata reading request, may determine a metadata journal object corresponding to the included object identifier, further query, according to the dependency relationship, to-be-split metadata on which the cloned volume depends, that is, metadata shared by the source volume and the cloned volume, from each metadata included in the metadata journal object, and then feed the queried to-be-split metadata back to the device node of the source volume.
S44, determining that the metadata to be split meets the integration condition, and performing layered integration processing on the metadata to be split;
step S45, sending the integrated metadata to be split to a distributed log system, and generating a split log object containing the metadata to be split by the distributed log system;
for the multi-layer data structure of the storage volume, in combination with the above description of the data management process of the LSM tree (i.e., the log structure merge tree), the metadata to be split determined in the present application may include metadata log objects corresponding to multiple layers of the multi-layer data structure from the source volume, that is, an integration condition is satisfied, therefore, in order to enhance data read-write performance, hierarchical integration processing may be performed on the metadata to be split first, and an implementation process may be combined with a merge processing process of the LSM tree, which is not described in detail herein.
Merging the small trees in the disk into a big tree through merge processing of a plurality of metadata to be split which are depended by the cloned volume in the source volume, reordering and optimizing data read-write performance. Still by taking the clone splitting scenario shown in fig. 5 as an example, when a user performs a clone splitting operation, it is not necessary to copy the metadata of L1, L2, and L3 of the source volume to the clone volume one by one across nodes, but according to the method described above, the device node where the source volume is located merges the metadata to be split in L1, L2, and L3, and then writes the merged metadata to the new log object generated by the distributed log system LSS, that is, the newly generated split log object stores the merged metadata to be split.
It should be understood that, if the clone volume depends on a layer of metadata of the source volume, after determining the layer of depended metadata to be split, the clone volume may be directly written into a new metadata log object generated by the distributed log system to obtain a corresponding split log object, and the implementation process of this embodiment is not described in detail.
It should be noted that, in the present application, how to implement the method for generating, by using a distributed logging system, a metadata log object containing metadata according to metadata obtained by a storage volume in a device node, including but not limited to the implementation manner described in the foregoing embodiment, may be determined according to an operating principle of a distributed storage system with a log structure.
Step S46, receiving the object identification fed back after the distributed log system is generated;
and S47, sending the object identifier to the equipment node where the clone volume is located, and writing the object identifier into the bottom layer of the multilayer data structure of the clone volume by the equipment node.
As described above, the distributed log system LSS generates a new metadata log object, and usually configures one-to-one corresponding object identifiers, such as object IDs, for identifying different metadata log objects stored by the LSS, so that after the LSS generates a split log object including metadata to be split according to the above method, the LSS can configure corresponding object identifiers so that the device nodes can identify the split log object accordingly, and therefore, the distributed log system can send the object identifier of the split log object to the device node where the source volume is located, send the object identifier to the device node where the clone volume is located, and write the object identifier of the split log object into the bottom layer of its LSM tree by the clone volume.
In summary, in the clone splitting method provided in the embodiment of the present application, the device node where the source volume is located generates a new split log object for the to-be-split metadata that is relied on by the cloned volume, and stores the to-be-split metadata, and since the device node is located in the distributed log system shared by the bottom layer, the device node where the source volume is located can be used as the independent metadata owned by the cloned volume, so that the device node where the source volume is located only needs to send the object identifier of the split log object to the cloned volume, and compared with a splitting method that directly copies the to-be-split metadata and then sends the to-be-split metadata to the cloned volume, the clone splitting efficiency is greatly improved, and the consumption of data transmission on network resources is reduced.
Referring to fig. 6, a schematic flowchart of yet another alternative example of the method for clone-splitting a storage volume provided in the present application, where the method may be applied to a device node where a clone volume is located, and an implementation process of the method for clone-splitting a storage volume is described from a side of the clone volume, as shown in fig. 6, the method may include:
step S61, obtaining the object identification of the splitting log object aiming at the cloning volume;
in view of the above description of the implementation process of the clone split method for a storage volume provided by the present application from the source volume side, the split log object may be generated by the distributed log system in response to a clone split request for the clone volume by a device node where the source volume is located, and the split log object includes metadata that the source volume has and is depended on by the clone volume, and the metadata is stored in the distributed log system. Regarding the acquisition process of the metadata to be split of the clone volume depending on the source volume, the split log object for storing the metadata to be split, and the object identifier thereof, reference may be made to the description of the corresponding part of the above embodiment, which is not described in detail herein.
Step S62, the object identification is written into the bottom layer of the multi-layer data structure of the clone volume.
For the clone splitting method provided by the application, the equipment node where the source volume is located only needs to send the object identifier of the split log object to the equipment node where the clone volume is located, so that the clone splitting speed is greatly improved and network resources consumed by data transmission are saved compared with the transmission of a large amount of copied metadata to be split.
And after the device node where the clone volume is located obtains the object identifier of the split log object, the object identifier is directly written into the bottom layer of the multilayer data structure of the clone volume. For a multilayer data structure with access ordering, the storage mode of the object identifier does not affect service access to the top layer, the service is ensured not to access old data of the bottom layer, whether updated metadata exists or not is avoided, mutual exclusion with IO is completely avoided, and service IO blocking is thoroughly avoided.
For example, still with the application scenario of the source volume and the clone volume shown in fig. 5, that is, the source volume and the clone volume are deployed on different device nodes, and the clone volume refers to/depends on metadata of L1, L2, and L3 of the source volume, referring to the schematic diagram of the clone splitting process shown in fig. 7, a device node where the source volume is located merges metadata to be split that is depended by the clone volume, generates a new log object, that is, a split log object, pushes an object ID of the log object to the device node where the clone volume is located, and the clone volume writes an object ID of the log object into a lowest layer of its LSM tree, such as L3 in this embodiment, so that the clone volume has its own independent log object containing metadata to be split, and thus, after the source volume is deleted, the logable volume can still access the metadata to be split stored in the LSS corresponding to the log object according to the object ID, thereby satisfying service access requirements and improving reliability of data.
In still other embodiments provided by the present application, a user may perform a splitting operation on an equipment node where a clone volume is located, for example, after determining a source volume on which the clone volume depends and an equipment node where the source volume is located, the user may trigger the clone splitting operation, generate a clone splitting request for the clone volume, send the clone splitting request to the equipment node where the source volume is located, so that the equipment node performs a clone splitting method for a storage volume described from a source volume side, send an object identifier of an obtained split log object to the equipment node where the clone volume is located, and write the object identifier into a bottom layer of an LSM tree by the clone volume.
In the process that the device node where the source volume is located responds to the clone splitting request, the clone splitting operation does not affect service access to the clone volume, and therefore, after the device node where the clone volume is located obtains a data writing request for the clone volume, metadata of data requested to be written in can be sent to the distributed log system, the metadata is written into a metadata log object corresponding to the top layer of the multilayer data structure of the clone volume by the distributed log system, and conflict is not generated with a clone splitting process that the clone volume writes an object identifier of a split log object into the bottom layer.
In practical applications, since the clone volume is created and its metadata and data depend on the source volume, as the clone volume writes new business data, new data and metadata can be generated for the new business data in a way of ROW (a function, returning a referenced ROW number), and the old data that the clone volume is not written to still depend on the source volume. However, according to the above-described clone splitting method for a storage volume, data and metadata of each storage volume (including a clone volume and a source volume dependent thereon) are all written into a shared storage system, and reliability and availability of the data and the metadata are ensured.
Based on this, in the process that the device node where the clone volume is located responds to the data write request, new metadata and data are generated in a ROM manner for the new service data written to the top layer of the clone volume, and then, as shown in fig. 7, the metadata may be written to a metadata log object corresponding to the top layer in the distributed log system for storage.
In still other embodiments, the device node where the clone volume is located may obtain a data access request, that is, a service access request, for the clone volume, and send a data query request including an object identifier recorded in a corresponding layer to the distributed journal system according to a sequence from a top layer to a bottom layer of the multilayer data structure of the clone volume, where the distributed journal system queries whether metadata to be accessed exists in the metadata journal object having the object identifier, and if the metadata to be accessed exists, the metadata to be accessed or the corresponding data to be accessed may be fed back to the device node where the clone volume is located, that is, the device node where the clone volume is located may receive the metadata/service data to be accessed, which is fed back by the distributed journal system. The process of how to implement business read-write access to a storage volume (e.g., a source volume or a clone volume) with an LSM tree is not described in detail herein.
Referring to fig. 8, a schematic structural diagram of an alternative example of a clone splitting apparatus for a storage volume proposed in the present application, the apparatus may perform structural composition and functional description thereof from an equipment node side of an equipment where a source volume is located, and as shown in fig. 8, the apparatus may include:
a clone split request obtaining module 81 for obtaining a clone split request for a clone volume;
a to-be-split metadata determination module 82, configured to respond to the clone split request and determine to-be-split metadata; the metadata to be split refers to metadata which is depended by the clone volume and is possessed by a source volume, and the metadata is stored in a distributed log system;
a split control module 83, configured to control the distributed log system to generate a split log object for the clone volume, and obtain an object identifier of the split log object; the split log object contains the metadata to be split;
and an object identifier sending module 84, configured to send the object identifier to the device node where the clone volume is located, where the device node writes the object identifier into a bottom layer of a multi-layer data structure of the clone volume.
Optionally, the apparatus may further include:
the storage module is used for storing each layer of metadata of a multi-layer data structure of the source volume in a corresponding metadata log object of the distributed log system;
the to-be-split metadata determination module 82 may include:
the to-be-split metadata determining unit is used for determining the to-be-split metadata according to the dependency relationship between the source volume and the clone volume; and the metadata to be split is stored in a metadata log object corresponding to the depended layer of the source volume.
In some embodiments, the splitting control device 83 may include:
a to-be-split metadata reading unit, configured to read the determined to-be-split metadata;
and the to-be-split metadata sending unit is used for sending the to-be-split metadata to the distributed log system, and the distributed log system generates to-be-split log objects containing the to-be-split metadata.
Optionally, the splitting control device 83 may further include:
the hierarchical integration unit is used for determining that the metadata to be split meets the integration condition and carrying out hierarchical integration processing on the metadata to be split;
based on this, the to-be-split metadata sending unit may be specifically configured to send the to-be-split metadata after the integration processing to the distributed log system.
In still other embodiments, the splitting control module 83 may further include:
the object identification receiving unit is used for receiving the object identification fed back after the distributed log system is generated; the object identification is used to identify the split log object.
Based on the above description of the embodiments, the clone splitting request obtaining module 81 may include:
the clone splitting request receiving unit is used for receiving a clone splitting request sent by a device node where the clone volume is located; alternatively, the first and second electrodes may be,
a clone split request obtaining unit, configured to determine that a source volume is depended on by at least one clone volume, and obtain a clone split request for the selected clone volume.
Referring to fig. 9, a schematic structural diagram of an alternative example of a clone splitting apparatus for a storage volume proposed in the present application, the apparatus may perform structural composition and functional description thereof from an equipment node side of an equipment where a clone volume is located, and as shown in fig. 9, the apparatus may include:
an object identifier obtaining module 91, configured to obtain an object identifier of a split log object for a clone volume; the split log object is generated by a distributed log system controlled by a device node where a source volume is located in response to a clone split request aiming at the clone volume, and comprises metadata which is depended by the clone volume and is possessed by the source volume, and the metadata is stored in the distributed log system;
an object identifier writing module 92, configured to write the object identifier to the bottom layer of the multi-layer data structure of the clone volume.
Optionally, the apparatus may further include:
a data write request obtaining module, configured to obtain a data write request for the clone volume in a clone split request response process;
and the metadata sending module is used for sending the metadata requested to be written into the distributed log system, and the distributed log system writes the received metadata into a metadata log object corresponding to the top layer of the multilayer data structure of the clone volume.
In still other embodiments, the apparatus may further include:
a data access request obtaining module, configured to obtain a data access request for the clone volume;
a data access request sending module, configured to send a data query request including an object identifier recorded in a corresponding layer to the distributed log system according to a sequence from a top layer to a bottom layer of a multi-layer data structure of the clone volume, where the distributed log system queries whether metadata to be accessed exists in a metadata log object having the object identifier;
and the metadata to be accessed receiving module is used for receiving the metadata to be accessed fed back by the distributed log system.
It should be noted that, for various modules, units, and the like in the foregoing apparatus embodiments, all of which may be stored in a memory as program modules, and the processor executes the program modules stored in the memory to implement corresponding functions, and for functions implemented by the program modules and their combinations and achieved technical effects, reference may be made to the description of corresponding parts in the foregoing method embodiments, and this embodiment is not described again.
The present application further provides a computer-readable storage medium, on which a computer program may be stored, where the computer program may be called and loaded by a processor in an equipment node to implement the steps of the clone splitting method for a storage volume described in the foregoing embodiments, and implementation processes of the embodiments of the present application are not described in detail herein.
Referring to fig. 10, a schematic diagram of a system architecture of an alternative example of a cluster suitable for the clone splitting method for storage volumes proposed in the present application is shown, where the cluster may include a plurality of device nodes 1010 and a distributed log system 1020 connected to the plurality of device nodes 1010, and the device nodes 1010 may be deployed with at least one storage volume.
The plurality of device nodes 1010 may include servers and/or terminal devices, and the like, the node types of the device nodes 1010 are not limited in the present application, and the node types of the plurality of device nodes 1010 in the cluster may be the same or different, and the number of device nodes included in the cluster may be flexibly adjusted according to actual requirements.
In practical applications, the server may be an independent physical server, a server cluster formed by a plurality of physical servers, or a cloud server capable of implementing cloud computing; the terminal device may include, but is not limited to, a smart phone, an Augmented Reality (AR) device, a Virtual Reality (VR) device, a robot, a smart device in different fields, a desktop computer, and the like, and the terminal device shown in fig. 8 is only an example and should not bring any limitation to the functions and the scope of the embodiments of the present application.
In the embodiment of the present application, when a clone volume is created, the clone volume may be created in a device node where a source volume that depends on the clone volume is located, and the clone volume may be created in other device nodes in consideration of load and capacity balancing problems. It can be seen that a storage volume included in any one of the device nodes 1010 in the cluster may serve as a source volume of the clone volume, a storage volume in another device node may be created as a clone volume of the source volume, and each storage volume included in each device node 1010 may be managed by using a multi-layer data structure (e.g., an LSM tree structure). The working principle of the multi-layer data structure can refer to, but not be limited to, the description of the corresponding parts of the above embodiments, which are not detailed here.
It should be understood that the structure of the device node shown in fig. 10 does not constitute a limitation to the device node in the embodiment of the present application, and in practical applications, in order to implement the clone splitting method for a storage volume described in the above embodiment, the device node may further generally include a program for storing the clone splitting method for a storage volume proposed in the present application, and a processor and the like for loading and executing the program to implement the clone splitting method for a storage volume proposed in the present application. Of course, in the case that the device node is a terminal device, the device node may further include at least one input component such as a touch sensing unit that senses a touch event on the touch display panel, a keyboard, a mouse, a camera, a microphone, and the like; at least one output component such as a display, speaker, vibration mechanism, light, etc.; an antenna; a sensor module; the power module and the like can determine the hardware structure according to the type of the terminal device and the functional requirements thereof, which are not listed herein.
The distributed logging system 1020 may be used as a shared storage system at the bottom of the cluster to store data and metadata of each storage volume in each device node 1010 in the cluster, and if the distributed logging system 1020 is implemented in a shared log structured manner, the storage manner of the data and the metadata of the storage volume by the distributed logging system 1020 is not limited, which may be determined as the case may be.
In conjunction with the application scenarios of the source volume and the clone volume described above, the distributed Log system 820 may be a shared Log structured (Log structured) distributed Log storage system (such as the LSS mentioned above), and the composition structure thereof may be flexibly adjusted according to actual requirements. Optionally, according to the data management application requirement of the multilayer data structure such as the LSM tree, the distributed log system 1020 serving as the underlying storage system may generally include a memory and a Disk, such as NVMe (NVM Express, a non-volatile memory), optane (an ultra-high-speed memory compatible with NVMe storage protocol), HDD (Hard Disk Drive), and the like, and may further include a control unit EC of the corresponding memory and Disk, and the present application does not describe in detail the composition structure of the distributed log system 1020.
In practical application, each storage volume included in each device node may be allocated to the same or different types of memories and disks in the distributed log system, so that new service data obtained by the storage volume may be written into a memory corresponding to a top layer of an LSM tree of the storage volume first, and metadata generated for the service data may be written into a log object at a corresponding layer in the memory. In order to avoid the memory data loss, the data and the metadata written into the memory can be persisted to the disk, and the implementation process is not described in detail in this application.
It should be understood that the cluster structure shown in fig. 10 does not constitute a limitation to the clusters in the embodiments of the present application, and in practical applications, the clusters may include more components than those shown in fig. 10, or some components may be combined, such as monitoring devices, etc., and the present application is not specifically described herein.
Finally, it should be noted that, with respect to the above embodiments, unless the context clearly dictates otherwise, the words "a", "an" and/or "the" do not denote a singular number, but may include a plurality. In general, the terms "comprises" and "comprising" are intended to cover only the explicitly identified steps or elements as not constituting an exclusive list and that the method or apparatus may comprise further steps or elements. An element defined by the phrase "comprising a component of ' 8230 ' \8230; ' does not exclude the presence of additional identical elements in the process, method, article, or apparatus that comprises the element.
In the description of the embodiments herein, "/" means "or" unless otherwise specified, for example, a/B may mean a or B; "and/or" herein is merely an association relationship describing an associated object, and means that there may be three relationships, for example, a and/or B, and may mean: a exists alone, A and B exist simultaneously, and B exists alone. In addition, in the description of the embodiments of the present application, "a plurality" means two or more than two.
Reference herein to terms such as "first," "second," or the like, is used for descriptive purposes only and to distinguish one operation, element, or module from another operation, element, or module without necessarily requiring or implying any actual such relationship or order between such elements, operations, or modules. And are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated, whereby a feature defined as "first" or "second" may explicitly or implicitly include one or more of such features.
In addition, in the present specification, the embodiments are described in a progressive or parallel manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments may be referred to each other. For the apparatus, the cluster, and the device node disclosed in the embodiment, since they correspond to the method disclosed in the embodiment, the description is relatively simple, and reference may be made to the description of the method for relevant points.
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (10)
1. A method of clonal splitting of a storage volume, the method comprising:
obtaining a clone splitting request for a clone volume;
responding the clone splitting request, and determining metadata to be split; the metadata to be split refers to metadata which is depended by the clone volume and is possessed by a source volume, and the metadata is stored in a distributed log system;
controlling the distributed log system to generate a split log object aiming at the clone volume, and obtaining an object identifier of the split log object; the split log object contains the metadata to be split;
and sending the object identification to a device node where the clone volume is located, and writing the object identification into the bottom layer of a multi-layer data structure of the clone volume by the device node.
2. The method of claim 1, wherein each level of metadata of the multi-level data structure of the source volume is stored in a corresponding metadata log object of the distributed log system;
the responding the clone splitting request, and determining metadata to be split comprises:
determining metadata to be split according to the dependency relationship between the source volume and the clone volume; and the metadata to be split is stored in a metadata log object corresponding to the depended layer of the source volume.
3. The method of claim 1, the controlling the distributed log system to generate a split log object, comprising:
reading the determined metadata to be split;
and sending the metadata to be split to the distributed log system, and generating a log object to be split containing the metadata to be split by the distributed log system.
4. The method of claim 3, the controlling the distributed log system to generate split log objects, further comprising:
determining that the metadata to be split meets an integration condition, and performing layered integration processing on the metadata to be split;
sending the to-be-split metadata to the distributed logging system comprises:
and sending the integrated metadata to be split to the distributed log system.
5. The method of any of claims 1-4, the obtaining an object identification of the split log object, comprising:
receiving an object identifier fed back after the distributed log system is generated; the object identification is used to identify the split log object.
6. The method of any of claims 1-4, the obtaining a clone split request for a clone volume, comprising:
receiving a clone splitting request sent by a device node where a clone volume is located; alternatively, the first and second electrodes may be,
determining that a source volume is depended on by at least one clone volume, and obtaining a clone split request for the selected clone volume.
7. A method of clonal splitting of a storage volume, the method comprising:
obtaining an object identification for a split log object of a clone volume; the split log object is generated by a distributed log system controlled by a device node where a source volume is located in response to a clone split request aiming at the clone volume, and comprises metadata which is depended by the clone volume and is possessed by the source volume, and the metadata is stored in the distributed log system;
writing the object identification to a bottom layer of a multi-layered data structure of the clone volume.
8. The method of claim 7, during the clone split request response, further comprising:
obtaining a data write request for the clone volume;
sending the requested metadata to be written to the distributed logging system, and writing the received metadata to a metadata log object corresponding to a top layer of a multi-layer data structure of the clone volume by the distributed logging system.
9. The method of claim 7, further comprising:
obtaining a data access request for the clone volume;
according to the sequence of the multilayer data structure of the clone volume from the top layer to the bottom layer, sending a data query request containing an object identifier recorded by a corresponding layer to the distributed log system, and querying whether metadata to be accessed exists in a metadata log object with the object identifier by the distributed log system;
and receiving metadata to be accessed fed back by the distributed log system.
10. A cluster, the cluster comprising: a plurality of appliance nodes and a distributed log system to which the plurality of appliance nodes are connected, the appliance nodes having deployed at least one storage volume, wherein:
the appliance node is configured to implement the method for clone splitting of a storage volume according to any of claims 1 to 6 and/or the method for clone splitting of a storage volume according to any of claims 7 to 9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211173256.3A CN115686356A (en) | 2022-09-26 | 2022-09-26 | Clone splitting method and cluster of storage volume |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211173256.3A CN115686356A (en) | 2022-09-26 | 2022-09-26 | Clone splitting method and cluster of storage volume |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115686356A true CN115686356A (en) | 2023-02-03 |
Family
ID=85062582
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211173256.3A Pending CN115686356A (en) | 2022-09-26 | 2022-09-26 | Clone splitting method and cluster of storage volume |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115686356A (en) |
-
2022
- 2022-09-26 CN CN202211173256.3A patent/CN115686356A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11249956B2 (en) | Scalable distributed storage architecture | |
| US11210035B2 (en) | Creating, by host computers, respective object of virtual disk based on virtual disk blueprint | |
| US10374792B1 (en) | Layout-independent cryptographic stamp of a distributed dataset | |
| US9317519B2 (en) | Storage system for eliminating duplicated data | |
| US9558194B1 (en) | Scalable object store | |
| JP4416821B2 (en) | A distributed file system that maintains a fileset namespace accessible to clients over the network | |
| CN107480237B (en) | Data fusion method and system for heterogeneous desktop cloud platform | |
| US20010049779A1 (en) | Information processing system and storage assigning method | |
| US6968382B2 (en) | Activating a volume group without a quorum of disks in the volume group being active | |
| US20130103708A1 (en) | Apparatus and method for enabling clients to participate in data storage in distributed file system | |
| US20220075757A1 (en) | Data read method, data write method, and server | |
| US10558373B1 (en) | Scalable index store | |
| CN113391875A (en) | Container deployment method and device | |
| US5630133A (en) | Customer information control system and method with API start and cancel transaction functions in a loosely coupled parallel processing environment | |
| US20230195695A1 (en) | File Sharing Method and Terminal Device | |
| CN109716280B (en) | Flexible memory rank storage arrangement | |
| CN107329798B (en) | Data replication method and device and virtualization system | |
| CN115686356A (en) | Clone splitting method and cluster of storage volume | |
| US20230063541A1 (en) | Determining computer resource usage at multiple levels of a container orchestration system hierarchy | |
| CN111399753B (en) | Method and device for writing pictures | |
| US9336232B1 (en) | Native file access | |
| CN113051244A (en) | Data access method and device, and data acquisition method and device | |
| US11928082B1 (en) | Container image file system performance enhancement | |
| US20240031238A1 (en) | Real time qubit allocation for dynamic network topographies | |
| CN117827363A (en) | Container creation method, system and node |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |