WO2015055035A1 - Procédé et dispositif pour hacher un objet de métadonnées - Google Patents

Procédé et dispositif pour hacher un objet de métadonnées Download PDF

Info

Publication number
WO2015055035A1
WO2015055035A1 PCT/CN2014/083044 CN2014083044W WO2015055035A1 WO 2015055035 A1 WO2015055035 A1 WO 2015055035A1 CN 2014083044 W CN2014083044 W CN 2014083044W WO 2015055035 A1 WO2015055035 A1 WO 2015055035A1
Authority
WO
WIPO (PCT)
Prior art keywords
metadata object
metadata
snapshot
name
hash
Prior art date
Application number
PCT/CN2014/083044
Other languages
English (en)
Chinese (zh)
Inventor
李永健
高峰
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Publication of WO2015055035A1 publication Critical patent/WO2015055035A1/fr

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10—File systems; File servers
    • G06F16/18—File system types
    • G06F16/1873—Versioning file systems, temporal file systems, e.g. file system supporting different historic versions of files
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10—File systems; File servers
    • G06F16/11—File system administration, e.g. details of archiving or snapshots
    • G06F16/128—Details of file system snapshots on the file-level, e.g. snapshot creation, administration, deletion
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10—File systems; File servers
    • G06F16/13—File access structures, e.g. distributed indices
    • G06F16/137—Hash-based

Definitions

  • the present invention relates to the field of communications technologies, and in particular, to a method and apparatus for hashing a metadata object.
  • a snapshot is one of the important means for system disaster recovery backup.
  • the snapshot is mainly used to implement online backup and recovery of data, and can quickly restore data to a specified time point when the system fails, and can allow users to access the snapshot.
  • the snapshot version data (hereinafter referred to as snapshot data) of the metadata object generated by the protected metadata object at any point in time.
  • snapshot data The snapshot version data (hereinafter referred to as snapshot data) of the metadata object generated by the protected metadata object at any point in time.
  • snapshot data The snapshot version data (hereinafter referred to as snapshot data) of the metadata object generated by the protected metadata object at any point in time.
  • the current cluster system needs to split the directory metadata object to hash all metadata objects to different nodes. Since the snapshot is directory-based, if the directory node is split, the snapshot metadata object generated when the metadata object is protected by the snapshot is also hashed to a different metadata server node.
  • a metadata object (including a normal metadata object and a snapshot metadata object) is hashed by the name of each metadata object (including a common element) in the same directory (for example, a directory whose node is too large).
  • the name of the data object and the name of the snapshot metadata object are executed by HASH calculation, and all metadata objects are hashed to different nodes according to the result of the HASH calculation.
  • the respective metadata objects are
  • each metadata object in Figure 1 (including two general metadata objects Filel and File2 under the current metadata server node Dir and its corresponding snapshot metadata object Filel@snapl,
  • the technical problem to be solved by the embodiments of the present invention is to provide a metadata server node that hashes the metadata objects, and reduces the randomness of the metadata objects being hashed to different metadata server nodes, and the snapshot is improved. Data access efficiency, improved snapshot access performance and overall system performance.
  • a first aspect of the embodiments of the present invention provides a method for hashing a metadata object, including: determining whether a metadata object is a snapshot metadata object;
  • the metadata object is not a snapshot metadata object, calculating a hash value of the metadata object by using the name of the metadata object as an input value;
  • the metadata object is a snapshot metadata object, calculating a hash value of the metadata object by using a version name in a name of the metadata object as an input value;
  • the metadata object is hashed to a corresponding metadata server node based on a hash value of the metadata object.
  • the snapshot metadata object is a snapshot version of a metadata object generated when the metadata object is protected by a snapshot
  • the composition of the name of the snapshot metadata object includes: version name + delimiter + version number.
  • the determining whether the metadata object is a snapshot metadata object includes:
  • the calculating the metadata object by using the name of the metadata object as an input value Hash values including:
  • the name of the metadata object is used as an input value of the HASH calculation, and the metadata object is subjected to HASH calculation to determine a hash value of the metadata object.
  • the calculating by using the version name in the name of the metadata object as the input value,
  • the hash value of the metadata object including:
  • the version name in the name of the metadata object is used as an input value of the HASH calculation, and the metadata object is subjected to HASH calculation to determine a hash value of the metadata object.
  • the second aspect of the embodiment of the present invention provides an apparatus for hashing a metadata object, including: a determining module, configured to determine whether the metadata object is a snapshot metadata object;
  • a first calculating module configured to calculate a hash value of the metadata object by using a name of the metadata object as an input value when the determining result of the determining module is negative;
  • a second calculating module configured to calculate a hash value of the metadata object by using a version name in a name of the metadata object as an input value when the determining result of the determining module is YES;
  • a hashing module configured to hash the metadata object to a corresponding metadata server node according to a hash value of the metadata object.
  • the snapshot metadata object is a snapshot version of a metadata object generated when the metadata object is protected by a snapshot
  • the composition of the name of the snapshot metadata object includes: version name + delimiter + version number.
  • the determining module is specifically configured to: when determining whether the metadata object is a snapshot metadata object,
  • the first calculating module calculates the name of the metadata object as an input value
  • the hash value of a metadata object specifically: The name of the metadata object is used as an input value of the HASH calculation, and the metadata object is subjected to HASH calculation to determine a hash value of the metadata object.
  • the second calculating module takes the version name in the name of the metadata object as an input When the value is calculated as the hash value of the metadata object, it is specifically used to:
  • the version name in the name of the metadata object is used as an input value of the HASH calculation, and the metadata object is subjected to HASH calculation to determine a hash value of the metadata object.
  • the embodiment of the present invention may calculate a hash value of the metadata object according to the name of the metadata object and the version name of the snapshot metadata object, and hash the metadata object to the corresponding metadata server node according to the calculated hash value.
  • the embodiment of the present invention may hash the current metadata object and its different versions of the snapshot metadata object to the same metadata server node, and reduce the randomness of the metadata object being hashed to different metadata server nodes. Improve the access efficiency of snapshot data, reduce the information interaction between the metadata server nodes due to snapshot access, improve the access performance of the snapshot and the overall performance of the system.
  • FIG. 1 is a schematic diagram of relationship between a snapshot protected metadata object and a generated snapshot metadata object in the prior art
  • FIG. 2 is a schematic diagram of a snapshot metadata object generated by a snapshot protected metadata object in a prior art being hashed to different nodes;
  • FIG. 3 is a schematic flowchart of a method for hashing a metadata object according to an embodiment of the present invention
  • FIG. 4 is a metadata protected by a snapshot in an embodiment of a method for hashing a metadata object according to an embodiment of the present invention
  • FIG. 5 is a schematic structural diagram of an apparatus for hashing a metadata object according to an embodiment of the present invention
  • FIG. 6 is a schematic diagram of a terminal provided by an embodiment of the present invention
  • Example structure diagram Example structure diagram.
  • FIG. 3 it is a schematic flowchart of an embodiment of a method for hashing a metadata object according to an embodiment of the present invention.
  • the method for hashing a metadata object described in this embodiment includes the steps of:
  • step S101 Determine whether the metadata object is a snapshot metadata object. If the determination result is no, step S102 is performed. If the determination result is yes, step S102 is performed.
  • the snapshot metadata object described in the embodiment of the present invention is a snapshot version of the metadata object generated when the metadata object is protected by the snapshot, and the snapshot version generated by one common metadata object may include multiple.
  • the current main ways to implement snapshots include COW (Copy On Write) and ROW (Redirect On Write). If the method for implementing snapshots is COW, the data is protected by snapshots mainly referring to new data.
  • the data at the location is first read out and written to another location (the storage space reserved for the snapshot, which can be called the snapshot space, and the data stored in the space is Snapshot data), then write new data to the location;
  • the storage space reserved for the snapshot which can be called the snapshot space, and the data stored in the space is Snapshot data
  • the data is protected by snapshot mainly refers to the newly written data when the new data is first written to a storage location. Being redirected to a new storage location, the storage location of the original data becomes the snapshot space.
  • the protection of the metadata object by the snapshot is also the same as the protection of the above data by the snapshot.
  • the metadata object described in this embodiment may include: a file, a directory, and the like, as shown in FIG.
  • Data object, Filel @snapl, Filel@snap3 in Figure 1 is the snapshot metadata object generated by the metadata object Filel, File2@snapl, File2@snap3 is the metadata pair.
  • a snapshot metadata object generated by File2; Filel and File2 are two metadata objects under the metadata server node Dir, and Filel@snapl and Filel@snap3 are snapshot metadata objects of different versions of the same version generated by Filel.
  • File2@snapl and File2@snap3 are snapshot metadata objects of the same version of the same version generated by File2.
  • the namespace of the snapshot metadata object is unique to the system, and the naming rules of the names of the respective snapshot metadata objects are also uniquely determined.
  • the naming rule for the name of the snapshot metadata object can be: version name
  • delimiter + version number that is, the composition of the name of the snapshot metadata object includes: version name + delimiter + version number, where the delimiter can be "@”.
  • version name + delimiter + version number where the delimiter can be "@”.
  • filel@snapl Filel is the version name of the snapshot metadata object
  • @ is the delimiter
  • snapl is the version number.
  • the user accesses the snapshot metadata object of the file.
  • the snapshot metadata object of the file.
  • you need to find these snapshot metadata objects on the same metadata server node you do not need cross-node query.
  • hashing all snapshot metadata objects of the same file you can use the same name as the metadata object hash. Enter a value, calculate the hash value of each metadata object, and hash each metadata object based on the hash value.
  • the metadata object when it is hashed, it may first determine whether the metadata object is a snapshot metadata object, and if it is a snapshot metadata object, calculate the name by using the version name in the name of the metadata object as an input value.
  • the hash value of the metadata object if it is not the snapshot metadata object, can determine that the metadata object is an ordinary metadata object, and then the name of the metadata object can be used as an input value to calculate the dispersion of the metadata object. Column value.
  • determining whether the metadata object is a snapshot metadata object may determine, according to the name of the metadata object, whether the name of the metadata object includes a name identifier of the snapshot metadata object, where the name identifier of the snapshot metadata object Specifically, it may be a delimiter in the name of the snapshot metadata object (for example, @ ). If it is determined that the name of the metadata object includes a delimiter in the name of the snapshot metadata object, the metadata object may be determined to be known. For the snapshot metadata object, if the name of the metadata object does not include the delimiter in the name of the object of the snapshot metadata, it can be determined that the metadata object is not the snapshot metadata object, but the current metadata object. .
  • the hash value of the metadata object is calculated by using the name of the current metadata object as an input value
  • the name of the metadata object is used as an input value calculated by HASH
  • the metadata object is subjected to HASH calculation.
  • the version name in the name of the metadata object is used as an input value of the HASH calculation, and the metadata object is subjected to HASH calculation to determine the hash value of the metadata object.
  • the calculated hash values of Filel, Filel@snapl, and Filel@snap3 are the same value.
  • the metadata objects File2, File2@snapl, and File2@snap3 in Figure 1 since the hash values of File2, File2@snapl, and File2@snap3 are calculated as File2 as the input value of HASH, the calculated File2 is calculated.
  • the hash values of File2@snapl and File2@snap3 are also the same value.
  • each metadata object may be hashed to the corresponding metadata service node according to the hash value of the metadata object.
  • the hash value of the same or similar hash value may be hashed to the same metadata server node by processing the hash value of each metadata object (for example, rounding), or the hash value may be different in size.
  • Large metadata objects are hashed to different metadata server nodes, and so on. For example, when hashing each metadata object in Figure 1, since the hash values of the metadata objects Filel, Filel@snapl, and Filel@snap3 are the same value, the hash values of these metadata objects are used.
  • the embodiment of the present invention mainly calculates the hash value of the metadata object by using the name of the metadata object and the version name of the snapshot metadata object as the input value of the HASH calculation, and hashing the metadata object according to the calculated hash value to The corresponding metadata server node.
  • the hash value of the metadata object and its different versions of the snapshot metadata object is calculated in the embodiment of the present invention, the same hash value can be obtained, and the current metadata object and its different versions of the snapshot element can be obtained.
  • the data objects are hashed to the same metadata server node, which reduces the randomness of the metadata objects being hashed to different metadata server nodes, thereby improving the access efficiency of the snapshot data and reducing the snapshot access between the metadata server nodes. Increased information interaction, improved snapshot access performance and overall system performance.
  • FIG. 5 is a schematic structural diagram of an embodiment of an apparatus for hashing a metadata object according to an embodiment of the present invention.
  • the device for hashing the metadata object described in this embodiment includes: a determining module 10, configured to determine whether the metadata object is a snapshot metadata object.
  • the first calculating module 20 is configured to calculate a hash value of the metadata object by using the name of the metadata object as an input value when the determining result of the determining module is negative.
  • the second calculating module 30 is configured to calculate a hash value of the metadata object by using a version name in the name of the metadata object as an input value when the determining result of the determining module is YES.
  • the hashing module 40 is configured to hash the metadata object to a corresponding metadata server node according to a hash value of the metadata object.
  • the determining module determines whether the metadata object is a snapshot metadata object
  • the determining module is specifically configured to:
  • the first computing module calculates the hash value of the metadata object by using the name of the metadata object as an input value, specifically:
  • the name of the metadata object is used as an input value of the HASH calculation, and the metadata object is subjected to HASH calculation to determine a hash value of the metadata object.
  • the second calculating module calculates the hash value of the metadata object by using the version name in the name of the metadata object as an input value, specifically:
  • the version name in the name of the metadata object is used as an input value of the HASH calculation, and the metadata object is subjected to HASH calculation to determine a hash value of the metadata object.
  • the snapshot metadata object described in the embodiment of the present invention is a snapshot version of the metadata object generated when the metadata object is protected by the snapshot, where the snapshot version generated by one common metadata object may include multiple, Common metadata objects under the metadata server node described in the embodiments may include: files, directories, and the like.
  • Filel and File2 are ordinary metadata objects under the metadata server node Dir described in the embodiment of the present invention, and Filel @snapl and Filel@snap3 in FIG. 1 are generated by the metadata object Filel.
  • the snapshot metadata object is the snapshot metadata object generated by the metadata object File2;
  • Filel and File2 are the two metadata objects under the metadata server node Dir,
  • Filel@snapl and Filel@ Snap3 is a snapshot metadata object of the same version of the same version generated by Filel, according to the object.
  • the namespace of the snapshot metadata object is unique to the system, and the naming rules of the names of the respective snapshot metadata objects are also uniquely determined.
  • the naming rule of the name of the snapshot metadata object may be: a version name + a delimiter + a version number, that is, a composition of a name of the snapshot metadata object includes: a version name + a delimiter + a version number, where The delimiter can be "@".
  • a version name + a delimiter + a version number that is, a composition of a name of the snapshot metadata object includes: a version name + a delimiter + a version number, where The delimiter can be "@".
  • Filel@snapl Filel is the version name of the snapshot metadata object
  • @ is the delimiter
  • snapl is the version number.
  • the user accesses the snapshot metadata object of the file.
  • the snapshot metadata object of the file.
  • you need to find these snapshot metadata objects on the same metadata server node you do not need cross-node query.
  • hashing all snapshot metadata objects of the same file you can use the same name as the metadata object hash. Enter a value, calculate the hash value of each metadata object, and hash each metadata object based on the hash value.
  • the determining module 10 may first determine whether the metadata object is a snapshot metadata object, and if it is a snapshot metadata object, the second computing module 30 is in the name of the metadata object. The version name as the input value, calculate the hash value of the metadata object, if not the snapshot metadata. The object can determine that the metadata object is a normal metadata object, and the first computing module 20 can calculate the hash value of the metadata object by using the name of the metadata object as an input value.
  • the determining module 10 determines whether the metadata object is a snapshot metadata object, and determines, according to the name of the metadata object, whether the name of the metadata object includes a name identifier of the snapshot metadata object, where the snapshot metadata object is The name identifier may specifically be a delimiter in the name of the snapshot metadata object (for example, @ ). If it is determined that the name of the metadata object includes a delimiter in the name of the snapshot metadata object, it may be determined that the identifier is The metadata object is a snapshot metadata object. If the name of the metadata object does not include a delimiter in the name of the object of the snapshot metadata, it can be determined that the metadata object is not a snapshot metadata object, but the current Metadata object.
  • the name of the current metadata object is used as the input value to calculate the hash value of the metadata object, specifically, the name of the metadata object is calculated as HASH.
  • the input value, the HASH calculation is performed on the metadata object, and the hash value of the metadata object is determined;
  • the second calculation module 30 calculates the hash value of the snapshot metadata object, the version in the name of the snapshot metadata object may be The name is used as an input value of the HASH calculation, and the metadata object is subjected to HASH calculation to determine the hash value of the metadata object. For example, when hashing each metadata object in FIG.
  • the first calculation module 20 when calculating the hash value, can directly calculate Filel or File2 as the input value of HASH.
  • the input values calculate the hash values for Filel@snapl and Filel@snap3.
  • the calculated hash values of Filel, Filel@snapl, and Filel @snap3 are the same value.
  • the metadata objects File2, File2@snapl, and File2@snap3 in Figure 1 since the hash values of File2, File2@snapl, and File2@snap3 are calculated as File2 as the input value of HASH, the calculated File2 is calculated.
  • the hash values of File2@snapl and File2@snap3 are also the same value.
  • the hash module 40 may perform each element according to the hash value of the metadata object.
  • the data object is hashed to the corresponding metadata service node.
  • the hash module 40 may treat the hash values of the respective metadata objects by processing (eg, rounding) the hash values to be the same or Nearly metadata objects are hashed to the same metadata server node, or metadata objects with large differences in hash values are hashed to different metadata server nodes.
  • Hash can be hashed to the same metadata server node by Filel, Filel@snapl, and Filel@snap3; in addition, the hash values of the metadata objects File2, File2@snapl, and File2@snap3 are the same value. Hash File2, File2@snapl, and File2@snap3 to the same metadata server node, as shown in Figure 4.
  • the metadata objects of all snapshot versions of the same file in Figure 2 may be hashed to different metadata server nodes, for example, the metadata version of Filel's snapshot version Filel@snapl and Filel@snap3 may be hashed to the node Dirl and the node Dir2 respectively.
  • the user accesses the snapshot data it may need to query across the nodes to obtain the complete snapshot data.
  • all the snapshot versions of the same file are in the meta-version.
  • Data objects are hashed to the same metadata server node.
  • the metadata objects Filel@snapl and Filel@snap3 of the snapshot version of Filel are hashed to the node Dirl.
  • the device for hashing the metadata object described in the embodiment of the present invention mainly calculates the hash value of the metadata object by using the name of the metadata object and the version name of the snapshot metadata object as the input value of the HASH calculation, and according to The computed hash value hashes the metadata object to the corresponding metadata server node.
  • the device described in the embodiment of the present invention can obtain the same hash value when calculating the hash value of the metadata object and its different versions of the snapshot metadata object, and thus the current metadata object and its different versions can be obtained.
  • the snapshot metadata object is hashed to the same metadata server node, which reduces the randomness of the metadata object being hashed to different metadata server nodes, thereby improving the access efficiency of the snapshot data and reducing the snapshot between the metadata server nodes. Increased information interaction for access, improving snapshot access performance and overall system performance.
  • FIG. 6 is a schematic structural diagram of an embodiment of a terminal according to an embodiment of the present invention.
  • the terminal described in this embodiment includes:
  • the memory 100 is configured to store instructions.
  • the processor 200 is configured to read an instruction from the memory, and perform an operation according to the instruction: determining whether the metadata object is a snapshot metadata object, and if the metadata object is not a snapshot metadata object, Calculating a hash value of the metadata object as an input value, and if the metadata object is a snapshot metadata object, calculating a version name in a name of the metadata object as an input value Deriving a hash value of the metadata object, hashing the metadata object to a corresponding metadata server node according to a hash value of the metadata object.
  • the processor 200 when determining, by the processor 200, whether the metadata object is a snapshot metadata object, the processor 200 is specifically configured to:
  • the determination result is yes, it is determined that the metadata object is a snapshot metadata object, and if the determination result is no, it is determined that the metadata object is not a snapshot metadata object.
  • the processor 200 when the processor 200 calculates the hash value of the metadata object as the input value, the processor 200 is specifically configured to:
  • the name of the metadata object is used as an input value of the HASH calculation, and the metadata object is subjected to HASH calculation to determine a hash value of the metadata object.
  • the processor 200 calculates a hash value of the metadata object by using a version name in a name of the metadata object as an input value, specifically, the method is:
  • the version name in the name of the metadata object is used as an input value of the HASH calculation, and the metadata object is subjected to HASH calculation to determine a hash value of the metadata object.
  • the snapshot metadata object described in the embodiment of the present invention is a snapshot version of the metadata object generated when the metadata object is protected by the snapshot, where the snapshot version generated by one common metadata object may include multiple, Common metadata objects under the metadata server node described in the embodiments may include: files, directories, and the like.
  • Filel and File2 are ordinary metadata objects under the metadata server node Dir described in the embodiment of the present invention, and Filel @snapl and Filel@snap3 in FIG. 1 are generated by the metadata object Filel.
  • the snapshot metadata object is the snapshot metadata object generated by the metadata object File2;
  • Filel and File2 are the two metadata objects under the metadata server node Dir,
  • Filel@snapl and Filel@ Snap3 is a snapshot metadata object of the same version with different version numbers generated by Filel. According to the object.
  • the namespace of the snapshot metadata object is unique to the system, and the naming rules of the names of the respective snapshot metadata objects are also uniquely determined.
  • the naming rule of the name of the snapshot metadata object may be: a version name + a delimiter + a version number, that is, a composition of a name of the snapshot metadata object includes: a version name + a delimiter + a version number, where The delimiter can be "@".
  • a version name + a delimiter + a version number that is, a composition of a name of the snapshot metadata object includes: a version name + a delimiter + a version number, where The delimiter can be "@".
  • Filel@snapl Filel is the version name of the snapshot metadata object
  • @ is the delimiter
  • snapl is the version number.
  • the user accesses the snapshot metadata object of the file. Only need to find these snapshot metadata objects in the same metadata server node, without cross-node query, the processor 200 can hash all the snapshot metadata objects of the same file with the same name as the metadata object. The input value of the column calculation, the hash value of each metadata object is calculated, and each metadata object is hashed according to the hash value.
  • the processor 200 when the processor 200 hashes the metadata object, it may first determine whether the metadata object is a snapshot metadata object, and if it is a snapshot metadata object, use the version name in the name of the metadata object as an input value. Calculating a hash value of the metadata object. If it is not a snapshot metadata object, determining that the metadata object is an ordinary metadata object, and then calculating the metadata by using the name of the metadata object as an input value. The hash value of the object.
  • the processor 200 determines whether the metadata object is a snapshot metadata object, and determines, according to the name of the metadata object, whether the name of the metadata object includes a name identifier of the snapshot metadata object, where the snapshot metadata object is The name identifier may specifically be a delimiter in the name of the snapshot metadata object (for example, @ ). If it is determined that the name of the metadata object includes a delimiter in the name of the snapshot metadata object, it may be determined that the identifier is The metadata object is a snapshot metadata object. If the name of the metadata object does not include a delimiter in the name of the object of the snapshot metadata, it may be determined that the metadata object is not a snapshot metadata object, but the current Metadata object.
  • the processor 200 calculates the hash value of the metadata object by using the name of the current metadata object as an input value, specifically, the name of the metadata object is used as an input value calculated by HASH, and the metadata object is performed on the metadata object.
  • HASH calculation determining a hash value of the metadata object; when the processor 200 calculates a hash value of the snapshot metadata object, the version name in the name of the snapshot metadata object may be used as an input value of the HASH calculation, and the metadata is The object performs HASH calculation, determining The hash value of this metadata object. For example, when hashing each metadata object in FIG.
  • the processor 200 Since the processor 200 calculates the hash values of Filel, Filel@snapl, and Filel@snap3 with Filel as the input value of HASH, the calculated hash values of Filel, Filel@snapl, and Filel@snap3 are the same value. . Similarly, for the metadata objects File2, File2@snapl, and File2@snap3 in FIG. 1, since the processor 200 calculates the hash values of File2, File2@snapl, and File2@snap3 using File2 as the input value of HASH, the calculation is performed. The obtained hash values of File2, File2@snapl, and File2@snap3 are also the same value.
  • each of the metadata objects may be hashed according to the hash value of the metadata object to the corresponding metadata service node.
  • the processor 200 may hash the metadata objects with the same or similar hash values to the same metadata server node by processing (eg, rounding) the hash values of the respective metadata objects, or hashing Metadata objects with large differences in value are hashed to different metadata server nodes, and so on. For example, when hashing each metadata object in Figure 1, since the hash values of the metadata objects Filel, Filel@snapl, and Filel @snap3 are the same value, the hash values of these metadata objects are used.
  • the user may need to query across the nodes to obtain the complete snapshot data.
  • all the snapshot versions of the same file are in the meta-version.
  • Data objects are hashed to the same metadata server node.
  • the metadata objects Filel @snapl and Filel@snap3 of the snapshot version of Filel are hashed to the node Dirl.
  • the complete snapshot data can be obtained without cross-nodes, which reduces the information interaction between the metadata server nodes due to snapshot access, and reduces the overhead of snapshot access.
  • the terminal described in the embodiment of the present invention may calculate the hash value of the metadata object by using the name of the metadata object and the version name of the snapshot metadata object as the input value of the HASH calculation, and the metadata object according to the calculated hash value. Hash to the corresponding metadata server node.
  • the version of the snapshot metadata object is hashed to the same metadata server node, reducing the randomness of the metadata object being hashed to different metadata server nodes, thereby improving the access efficiency of the snapshot data and reducing the metadata server node Increased information interaction due to snapshot access, improving snapshot access performance and overall system performance.
  • the storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

Abstract

L'invention concerne un procédé pour hacher un objet de métadonnées, consistant à : déterminer si un objet de métadonnées est ou non un objet de métadonnées d'instantané d'écran ; si l'objet de métadonnées n'est pas un objet de métadonnées d'instantané d'écran, alors utiliser un nom de l'objet de métadonnées comme valeur d'entrée lors du calcul d'une valeur de hachage de l'objet de métadonnées ; si l'objet de métadonnées est un objet de métadonnées d'instantané d'écran, alors utiliser un nom de version dans le nom de l'objet de métadonnées comme valeur d'entrée lors du calcul d'une valeur de hachage de l'objet de métadonnées ; et hacher l'objet de métadonnées en un nœud de serveur de métadonnées correspondant sur la base de la valeur de hachage de l'objet de métadonnées. L'invention concerne également un dispositif pour hacher un objet de métadonnées. Les avantages du caractère aléatoire réduit de l'objet de métadonnées qui est haché en différents nœuds de serveur de métadonnées, l'efficacité d'accès accrue de données d'instantané d'écran, et les performances d'accès à un instantané d'écran et les performances de système globales accrues sont également décrits.
PCT/CN2014/083044 2013-10-18 2014-07-25 Procédé et dispositif pour hacher un objet de métadonnées WO2015055035A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201310493842.0A CN103559224A (zh) 2013-10-18 2013-10-18 一种对元数据对象进行散列的方法及装置
CN201310493842.0 2013-10-18

Publications (1)

Publication Number Publication Date
WO2015055035A1 true WO2015055035A1 (fr) 2015-04-23

Family

ID=50013471

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2014/083044 WO2015055035A1 (fr) 2013-10-18 2014-07-25 Procédé et dispositif pour hacher un objet de métadonnées

Country Status (2)

Country Link
CN (1) CN103559224A (fr)
WO (1) WO2015055035A1 (fr)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10037339B1 (en) 2017-12-28 2018-07-31 Dropbox, Inc. Synchronized organization directory with team member folders
CN111723050A (zh) * 2019-03-22 2020-09-29 伊姆西Ip控股有限责任公司 用于文件管理的方法、电子设备和计算机程序产品

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103559224A (zh) * 2013-10-18 2014-02-05 华为技术有限公司 一种对元数据对象进行散列的方法及装置
US10423495B1 (en) * 2014-09-08 2019-09-24 Veritas Technologies Llc Deduplication grouping
US10896165B2 (en) * 2017-05-03 2021-01-19 International Business Machines Corporation Management of snapshot in blockchain
CN110032553B (zh) * 2019-04-08 2021-04-06 中国人民大学 一种数据对象的dna生成和校验方法

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101162469A (zh) * 2007-11-09 2008-04-16 清华大学 基于快照的细粒度文件与目录版本管理方法
US20110087696A1 (en) * 2005-01-20 2011-04-14 F5 Networks, Inc. Scalable system for partitioning and accessing metadata over multiple servers
WO2012036757A1 (fr) * 2010-09-13 2012-03-22 Marvell World Trade Ltd. Mémoire cache répartie de métadonnées
CN102411637A (zh) * 2011-12-30 2012-04-11 创新科软件技术（深圳）有限公司 分布式文件系统的元数据管理方法
CN103150394A (zh) * 2013-03-25 2013-06-12 中国人民解放军国防科学技术大学 面向高性能计算的分布式文件系统元数据管理方法
CN103559224A (zh) * 2013-10-18 2014-02-05 华为技术有限公司 一种对元数据对象进行散列的方法及装置

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7228320B2 (en) * 2004-11-17 2007-06-05 Hitachi, Ltd. System and method for creating an object-level snapshot in a storage system
CN101639835A (zh) * 2008-07-30 2010-02-03 国际商业机器公司 多租户场景中应用数据库分区的方法和装置
CN102122285B (zh) * 2010-01-11 2012-10-31 卓望数码技术（深圳）有限公司 一种数据缓存系统中的数据查询系统和数据查询方法
CN101980203A (zh) * 2010-10-29 2011-02-23 中国科学院声学研究所 一种用于高清媒体的嵌入式文件系统
CN103019960B (zh) * 2012-12-03 2016-03-30 华为技术有限公司 分布式缓存方法及系统

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110087696A1 (en) * 2005-01-20 2011-04-14 F5 Networks, Inc. Scalable system for partitioning and accessing metadata over multiple servers
CN101162469A (zh) * 2007-11-09 2008-04-16 清华大学 基于快照的细粒度文件与目录版本管理方法
WO2012036757A1 (fr) * 2010-09-13 2012-03-22 Marvell World Trade Ltd. Mémoire cache répartie de métadonnées
CN102411637A (zh) * 2011-12-30 2012-04-11 创新科软件技术（深圳）有限公司 分布式文件系统的元数据管理方法
CN103150394A (zh) * 2013-03-25 2013-06-12 中国人民解放军国防科学技术大学 面向高性能计算的分布式文件系统元数据管理方法
CN103559224A (zh) * 2013-10-18 2014-02-05 华为技术有限公司 一种对元数据对象进行散列的方法及装置

Cited By (53)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11010402B2 (en) 2017-12-28 2021-05-18 Dropbox, Inc. Updating a remote tree for a client synchronization service
US11782949B2 (en) 2017-12-28 2023-10-10 Dropbox, Inc. Violation resolution in client synchronization
US10324903B1 (en) 2017-12-28 2019-06-18 Dropbox, Inc. Content management client synchronization service
US10599673B2 (en) 2017-12-28 2020-03-24 Dropbox, Inc. Content management client synchronization service
US10671638B2 (en) 2017-12-28 2020-06-02 Dropbox, Inc. Allocation and reassignment of unique identifiers for synchronization of content items
US10691720B2 (en) 2017-12-28 2020-06-23 Dropbox, Inc. Resynchronizing metadata in a content management system
US10691719B2 (en) 2017-12-28 2020-06-23 Dropbox, Inc. Cursor with last observed access state
US10691721B2 (en) 2017-12-28 2020-06-23 Dropbox, Inc. Restrictive access control list
US10726044B2 (en) 2017-12-28 2020-07-28 Dropbox, Inc. Atomic moves with lamport clocks in a content management system
US10733205B2 (en) 2017-12-28 2020-08-04 Dropbox, Inc. Violation resolution in client synchronization
US10762104B2 (en) 2017-12-28 2020-09-01 Dropbox, Inc. File journal interface for synchronizing content
US10776386B2 (en) 2017-12-28 2020-09-15 Dropbox, Inc. Content management client synchronization service
US11880384B2 (en) 2017-12-28 2024-01-23 Dropbox, Inc. Forced mount points / duplicate mounts
US10789269B2 (en) 2017-12-28 2020-09-29 Dropbox, Inc. Resynchronizing metadata in a content management system
US10789268B2 (en) 2017-12-28 2020-09-29 Dropbox, Inc. Administrator console for an organization directory
US10866963B2 (en) 2017-12-28 2020-12-15 Dropbox, Inc. File system authentication
US10866964B2 (en) 2017-12-28 2020-12-15 Dropbox, Inc. Updating a local tree for a client synchronization service
US10872098B2 (en) 2017-12-28 2020-12-22 Dropbox, Inc. Allocation and reassignment of unique identifiers for synchronization of content items
US10877993B2 (en) 2017-12-28 2020-12-29 Dropbox, Inc. Updating a local tree for a client synchronization service
US10922333B2 (en) 2017-12-28 2021-02-16 Dropbox, Inc. Efficient management of client synchronization updates
US10929427B2 (en) 2017-12-28 2021-02-23 Dropbox, Inc. Selective synchronization of content items in a content management system
US10929426B2 (en) 2017-12-28 2021-02-23 Dropbox, Inc. Traversal rights
US10936622B2 (en) 2017-12-28 2021-03-02 Dropbox, Inc. Storage interface for synchronizing content
US10949445B2 (en) 2017-12-28 2021-03-16 Dropbox, Inc. Content management client synchronization service
US10037339B1 (en) 2017-12-28 2018-07-31 Dropbox, Inc. Synchronized organization directory with team member folders
US11003685B2 (en) 2017-12-28 2021-05-11 Dropbox, Inc. Commit protocol for synchronizing content items
US10997200B2 (en) 2017-12-28 2021-05-04 Dropbox, Inc. Synchronized organization directory with team member folders
US10095879B1 (en) 2017-12-28 2018-10-09 Dropbox, Inc. Restrictive access control list
US11669544B2 (en) 2017-12-28 2023-06-06 Dropbox, Inc. Allocation and reassignment of unique identifiers for synchronization of content items
US11080297B2 (en) 2017-12-28 2021-08-03 Dropbox, Inc. Incremental client synchronization
US11120039B2 (en) 2017-12-28 2021-09-14 Dropbox, Inc. Updating a remote tree for a client synchronization service
US11176164B2 (en) 2017-12-28 2021-11-16 Dropbox, Inc. Transition to an organization directory
US11188559B2 (en) 2017-12-28 2021-11-30 Dropbox, Inc. Directory snapshots with searchable file paths
US11204938B2 (en) 2017-12-28 2021-12-21 Dropbox, Inc. Caching of file system warning queries to determine an applicable file system warning
US11308118B2 (en) 2017-12-28 2022-04-19 Dropbox, Inc. File system warnings
US11314774B2 (en) 2017-12-28 2022-04-26 Dropbox, Inc. Cursor with last observed access state
US11386116B2 (en) 2017-12-28 2022-07-12 Dropbox, Inc. Prevention of loss of unsynchronized content
US11423048B2 (en) 2017-12-28 2022-08-23 Dropbox, Inc. Content management client synchronization service
US11429634B2 (en) 2017-12-28 2022-08-30 Dropbox, Inc. Storage interface for synchronizing content
US11461365B2 (en) 2017-12-28 2022-10-04 Dropbox, Inc. Atomic moves with lamport clocks in a content management system
US11475041B2 (en) 2017-12-28 2022-10-18 Dropbox, Inc. Resynchronizing metadata in a content management system
US11500899B2 (en) 2017-12-28 2022-11-15 Dropbox, Inc. Efficient management of client synchronization updates
US11500897B2 (en) 2017-12-28 2022-11-15 Dropbox, Inc. Allocation and reassignment of unique identifiers for synchronization of content items
US11514078B2 (en) 2017-12-28 2022-11-29 Dropbox, Inc. File journal interface for synchronizing content
US11593394B2 (en) 2017-12-28 2023-02-28 Dropbox, Inc. File system warnings application programing interface (API)
US11630841B2 (en) 2017-12-28 2023-04-18 Dropbox, Inc. Traversal rights
US11657067B2 (en) 2017-12-28 2023-05-23 Dropbox Inc. Updating a remote tree for a client synchronization service
US11048720B2 (en) 2017-12-28 2021-06-29 Dropbox, Inc. Efficiently propagating diff values
US11704336B2 (en) 2017-12-28 2023-07-18 Dropbox, Inc. Efficient filename storage and retrieval
US11755616B2 (en) 2017-12-28 2023-09-12 Dropbox, Inc. Synchronized organization directory with team member folders
US11016991B2 (en) 2017-12-28 2021-05-25 Dropbox, Inc. Efficient filename storage and retrieval
US11836151B2 (en) 2017-12-28 2023-12-05 Dropbox, Inc. Synchronizing symbolic links
CN111723050A (zh) * 2019-03-22 2020-09-29 伊姆西Ip控股有限责任公司 用于文件管理的方法、电子设备和计算机程序产品

Also Published As

Publication number Publication date
CN103559224A (zh) 2014-02-05

Similar Documents

Publication Publication Date Title
US20230359644A1 (en) Cloud-based replication to cloud-external systems
US9983825B2 (en) Efficient data volume replication for block-based storage
WO2015055035A1 (fr) Procédé et dispositif pour hacher un objet de métadonnées
US9703803B2 (en) Replica identification and collision avoidance in file system replication
KR101862718B1 (ko) 블록-기반 저장을 위한 데이터 볼륨 지속성 상태 감소
US9792306B1 (en) Data transfer between dissimilar deduplication systems
US9274716B2 (en) Systems and methods for hierarchical reference counting via sibling trees
US8694469B2 (en) Cloud synthetic backups
US9684669B2 (en) Single instantiation method using file clone and file storage system utilizing the same
US9727273B1 (en) Scalable clusterwide de-duplication
WO2017201977A1 (fr) Procédé et appareil d'écriture et de lecture de données, et grappe de stockage d'objet distribuée
WO2013163832A1 (fr) Procédé et dispositif de stockage dans un nuage
WO2015039569A1 (fr) Dispositif de stockage de copies et procédé de stockage de copies
WO2014205847A1 (fr) Procédé, appareil et système de distribution de sous-tâches d'équilibre de zonage
CN114780252B (zh) 数据仓库系统的资源管理方法及装置
WO2018059239A1 (fr) Procédé et dispositif pour la mémorisation de fichier
JP2019537097A (ja) Ｉノードのアクセスパターンの追跡及びｉノードの先取り
US20190215363A1 (en) Dynamic pool-based tiering for synchronization storage
US9684668B1 (en) Systems and methods for performing lookups on distributed deduplicated data systems
WO2017145214A1 (fr) Système informatique de transfert de données d'un nœud central à un nœud périphérique
WO2019174558A1 (fr) Procédé et dispositif d'indexation de données
WO2020151337A1 (fr) Procédé et appareil de traitement de fichiers distribués, dispositif informatique et support d'informations
US10684786B2 (en) Methods for performing global deduplication on data blocks and devices thereof
US11531644B2 (en) Fractional consistent global snapshots of a distributed namespace
US20220197860A1 (en) Hybrid snapshot of a global namespace

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 14853601

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 14853601

Country of ref document: EP

Kind code of ref document: A1