CN100595761C - Metadata management method for splitting name space - Google Patents

Metadata management method for splitting name space Download PDF

Info

Publication number
CN100595761C
CN100595761C CN200710308573A CN200710308573A CN100595761C CN 100595761 C CN100595761 C CN 100595761C CN 200710308573 A CN200710308573 A CN 200710308573A CN 200710308573 A CN200710308573 A CN 200710308573A CN 100595761 C CN100595761 C CN 100595761C
Authority
CN
China
Prior art keywords
directory
meta data
data server
request
sub
Prior art date
Application number
CN200710308573A
Other languages
Chinese (zh)
Other versions
CN101196929A (en
Inventor
刑晶
唐荣锋
熊劲
马捷
Original Assignee
中国科学院计算技术研究所
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 中国科学院计算技术研究所 filed Critical 中国科学院计算技术研究所
Priority to CN200710308573A priority Critical patent/CN100595761C/en
Publication of CN101196929A publication Critical patent/CN101196929A/en
Application granted granted Critical
Publication of CN100595761C publication Critical patent/CN100595761C/en

Links

Abstract

The invention discloses a metadata managing method for splitting name space, which comprises the following steps: establishing document system; managing the established document system. Wherein, management of established document system also comprises that client establishes catalogues; client establishes documents; client executes query operation; client executes readable catalogues operation; client deletes documents; client deletes catalogues; client executes relief operation; client executes rename operation; client establishes hard links; client establishes soft links. The invention can avoid the access focus of the same catalogue while improving extensibility of metabase server.

Description

A kind of metadata management method that splits name space

Technical field

The present invention relates to the storage system architecture field, particularly relate to a kind of metadata management method that splits name space.

Background technology

Along with the informationalized continuous propelling in the world, computing machine has become indispensable instrument in the every profession and trade.The storage management software of beared information just is subject to people's attention day by day along with uprushing of quantity of information in computing machine.

The function of storage management software is that physical storage medium (as disk, dish battle array) is organized into logic groups, to realize various functions and characteristic.Operating system provides stores service by the interface that calls storage management software for the upper-layer user.

Handled data have two classes in the storage system, and a class is the data that the user uses, and another kind of is the data that storage system is used for the leading subscriber data, i.e. metadata.The characteristics of data access are that the IO amount is big, and its cpu busy percentage is low, to the network bandwidth requirement height; The characteristics of metadata access are that the IO amount is little, and its cpu busy percentage height requires high to network delay.Because the otherness of two class data, storage system is distinguished the information and the control stream of two class data, has defined the feature operation that data and metadata are handled respectively, and will be separated to individual components to the management of two class data.Wherein the parts of deal with data are called data storage server; The parts of process metadata are called meta data server.

The design of meta data server constitutes from system can be divided into two classes: a class is the cell data server, it provides the metadata service by a meta data server for total system, and the request of all clients all sends on the meta data server and is performed.The advantage of this design is that metadata request is all finished at a machine, does not need to carry out interactive operation between the machine, has the advantages that realization is simple relatively, be easy to manage.Its shortcoming is that extendability is poor, and facing fairly large intensive metadata request when distributed file system is that meta data server can become performance bottleneck, thereby reduces the handling property of total system.Another kind of is the multivariate data server, and it provides meta data server by many meta data servers jointly for whole distributed file system.In this multivariate data server system, each metadata request is divided into one of them meta data server and handles.The advantage of multivariate data server architecture is a favorable expandability, can be applicable to fairly large group of planes computing environment; Shortcoming is that the load balance between the multivariate data server is difficult to keep, and to the consistency maintenance complexity of metadata, reliability realizes complicated.

For the multivariate data server architecture, name space is the key factor that the decision metadata is handled extendability and aggregation processing performance in the tissue and the division of meta data server.Existing multivariate data server is general to adopt following four kinds of modes to divide name space on a plurality of meta data servers.

Dynamically divide name space: the core concept of this method is to regard file system as sandwich construction, whole directory tree is that unit is divided on a plurality of meta data servers with the catalogue subtree, visit capacity according to the catalogue subtree when catalogue subtree creaties directory is determined its deposit position, when visit capacity hour is placed on catalogue on the identical meta data server, when visit capacity is big, catalogue is left on the meta data server at hash value place of parent directory and directory name.This method can dynamically be adjusted the distribution of name space according to the visit load, but then can't provide effective and efficient manner to avoid for the visit focus of same catalogue.

The static division name space: this method is divided into some subregions according to directory path with the catalogue subtree, a plurality of subregions are distributed on a plurality of meta data servers, metadata operation in the subregion is all finished at meta data server at section post, does not allow to stride the metadata operation of subregion.So this method can not have extensibility preferably because the name space of multivariate data server maintenance intersects, but can not adjust the metadata distribution dynamic, can not solve the visit hot issue of same catalogue according to load variations.

Utilize hash table to divide name space: file or catalogue to be utilized value of hash function call according to his father's catalogue and name, be worth according to this file or catalogue are distributed on a plurality of meta data servers.File is carried out the visit hot issue that the hash distribution can be avoided same catalogue, but because the file in the same catalogue because hash is distributed on the different meta data servers, causes having lost locality information.Catalogue is carried out the hash distribution can keep locality information, but can not solve the visit hot issue of same catalogue equally.

Name block is divided: in this system, metadata and data all are stored on the shared data dish, metadata according to the mode of data write down with the piece be unit record to data disks, all servers can have access to metadata.A plurality of servers are realized exclusive reference to some meta data block by distributed lock mechanism.This metadata management mode is convenient to safeguard the consistance of metadata, can increase the system communication message number but use distributed lock to control visit, and is not only can be to system extension influential but also can't avoid visiting hot issue equally.

Summary of the invention

The object of the present invention is to provide a kind of metadata management method that splits name space, it can also avoid the visit hot issue of same catalogue in the extendability that improves meta data server.

A kind of metadata management method that splits name space for realizing that purpose of the present invention provides comprises the following steps:

A. create file system;

B. the file system that has created is managed.

Described steps A also further comprises:

A1. create the metadata store catalogue of meta data server, the initialization buffer structure;

A2. create the index node of root directory on the pivot data server of appointment, the call number of its index node is 1, and making its sub-directory sequence number is 0, and the file that creaties directory is 0 directory entry information to deposit the sub-directory sequence number;

A3. utilize function B to determine the sub-directory sequence number of other meta data servers, make up the sub-directory distribution array of root directory with this;

A4. send to other meta data servers according to sub-directory distribution array and create the sub-directory request, additional parent directory call number and sub-directory sequence number in the request;

A5. when other meta data servers are received the request of establishment sub-directory, create the index node of sub-directory correspondence, make its sub-directory sequence number be request indication sequence number, and the file that creaties directory is receiveed the response after finishing to the transmission of source meta data server to deposit the directory entry information of sub-directory sequence number.

Described step B also further comprises:

B1. client creaties directory;

B2. client is created file;

B3. client executing query manipulation;

B4. client executing is read directory operation;

B5. client deleted file;

B6. client deltrees;

B7. client executing releasing operation;

B8. client executing rename operation;

B9. client is created hard link;

B10. client is created soft link.

Described step B1 also further comprises:

B11. wish the path that creaties directory to resolve this path client input client, obtain parent directory index node, parent directory call number and the directory name of this path correspondence by operating system;

B12. operating system utilizes function A to obtain the meta data server identifier that creaties directory according to parent directory index node, parent directory call number and directory name, sends the request of createing directory to this meta data server then;

B13. this meta data server receives the new directory correspondence is created in the request back of createing directory on this meta data server node, and making its sub-directory sequence number is 0, and to belong to the sub-directory sequence number be 0 directory entry information to the file that creaties directory in order to deposit;

B14. utilize function B to determine the sub-directory sequence number of other meta data servers, make up the sub-directory distribution array of this catalogue with this;

B15. send to other meta data servers according to sub-directory distribution array and create the sub-directory request, additional parent directory call number and sub-directory sequence number in the request;

B16. when other meta data servers are received the request of establishment sub-directory, create corresponding index node, make its sub-directory sequence number be request indication sequence number, and the file that creaties directory is receiveed the response after finishing to the transmission of source meta data server to deposit the directory entry information of sub-directory sequence number;

B17. the source meta data server is received the directory entry information of adding this catalogue after other meta data servers are responded in the sub-directory file of its parent directory;

B18. transmit a reply message.

Among the step B1, described source meta data server is meant the meta data server that receives the request of createing directory and create new directory manipulative indexing node.

Described step B2 also further comprises:

B21. according to the parent directory index node, parent directory call number and directory name utilize function A to obtain to create the meta data server identifier of file, send to meta data server then and create file request;

B22. meta data server receives and creates the index node of creating the file correspondence after the file request on this meta data server;

B23. the directory entry information of index node is added in the sub-directory file of its parent directory;

If B24. the sub-directory call number of this parent directory index node is not equal to 0 then the attribute information of this parent directory index node is sent to the meta data server that the sub-directory call number equals 0 parent directory index node place, upgrade attribute by this parent directory index node;

B25. transmit a reply message.

Described step B3 also further comprises:

B31. according to the parent directory index node, parent directory call number and directory name utilize function A to obtain the meta data server identifier at query object place, send the inquiry file request to this meta data server then;

B32. meta data server receives after the inquiry file request query object in this meta data server;

B33. client puts 1 in the corresponding position of the client-access table of the index node that inquires;

B34. give client with the gained information-reply.

Described step B4 also further comprises:

B41. judge the meta data server at its index node place according to the parent directory call number, send to this meta data server and read catalog request;

B42. after this meta data server is received request, find the index node of parent directory call number correspondence, utilize function C to determine which meta data server the sub-directory of reading interval place that catalogue is read is distributed in by index node;

B43. send to the meta data server at the directory entry place of required reading section and read directory command;

B44. each meta data server is read the directory entry content according to the reference position off-set value and the side-play amount of the request in the read command request, sends back to the source meta data server again;

B45. the source meta data server is collected the request back and is sent a reply information to client.

Among the step B4, described source meta data server is meant and receives the meta data server of reading catalog request and reading catalog request to other meta data servers transmissions.

Described step B5 also further comprises:

B51. according to the parent directory index node, parent directory call number and directory name utilize function A to obtain the meta data server identifier at this document place, send the deleted file request to this meta data server then;

B52. after this meta data server receives the deleted file request, find deleted file, the directory entry of this document is deleted from the sub-directory file of his father's directory index node;

B53. revise the property value of the parent directory index node on the same meta data server, if the sub-directory call number of this parent directory index node is not equal to 0 then the attribute information of this parent directory index node is sent to the meta data server that the sub-directory call number equals 0 parent directory index node place, upgrade attribute by this parent directory index node.

Described step B6 also further comprises:

B61. according to the parent directory index node, parent directory call number and directory name utilize function A to obtain the meta data server identifier at this catalogue place, send the request of deltreeing to meta data server then;

B62. after this meta data server receives the request of deltreeing, find deleted catalogue, judge that whether this catalogue is empty, for sky then is changed to the deletion state with this catalogue, otherwise returns error message;

B63. the directory entry of this catalogue is deleted from the sub-directory file of his father's directory index node;

B64. revise the property value of the parent directory index node on the same meta data server, if the sub-directory call number of this parent directory index node is not equal to 0 then the attribute information of this parent directory index node is sent to the meta data server that the sub-directory call number equals 0 parent directory index node place, upgrade attribute by this parent directory index node.

Described step B7 also further comprises:

B71. get 10 of the Far Lefts of its binary representation according to call number, be converted to decimal number with these 10, this value then is the meta data server identifier at call number indication index node place, sends the request of release to this meta data server then;

B72. after this meta data server receives the request of release, find index node,, put 0 the position of requesting client correspondence in the client-access table of index node according to call number;

Whether client-access table and the link number in the attribute of B73. judging this index node are 0, as are 0, then discharge this index node;

If B731. this index node is a file, then the buffer memory with file metadata corresponding storage file and index node discharges, and transmit a request to the data storage file deletion of data storage server with the file correspondence;

If B732. this index node is a catalogue, then with catalogue on all meta data servers the sub-directory file and the buffer memory of the index node of all sub-directories discharge.

Described step B8 also further comprises:

B81. according to old parent directory index node, old parent directory call number and old directory name, utilize function A to obtain the meta data server 1 at this old parent directory place, according to new parent directory index node, new parent directory call number and new directory title utilize function A to obtain the meta data server 2 at new parent directory place.The meta-data identifier of meta data server 1 and meta data server 2 is sent the rename request together with the rename required parameter to meta data server 1;

B82. after this meta data server receives the rename request, find, the directory entry of this document is deleted from the sub-directory file of old parent directory deleted ancient deed;

B83. revise the property value of old parent directory index node, if the sub-directory call number of this old parent directory index node is not equal to 0 then attribute information that will this old parent directory index node sends to the meta data server that the sub-directory call number is not equal to 0 old parent directory index node place, upgrade attribute by this old parent directory index node;

If B84. meta data server 2 is identical with meta data server 1, then on same meta data server, continues operation, otherwise this request is forwarded on the meta data server 2;

B85. on meta data server 2, add the directory entry of the new name correspondence of Rename file to the sub-directory file of new parent directory index node, if the sub-directory call number of this new parent directory index node is not equal to 0 then attribute information that will this new parent directory index node sends to the meta data server that the sub-directory call number is not equal to 0 new parent directory index node place, upgrade attribute by this new parent directory index node;

B86. send successfully response to meta data server 1 after finishing, meta data server 1 receives that responding the back sends echo message to client.

Described step B9 also further comprises:

B91. according to the parent directory index node, parent directory call number and directory name utilize function A to obtain to create the meta data server identifier of hard link, send the hard link request of creating to this meta data server then;

B92. after this meta data server receives and creates the hard link request, the directory entry information of this hard link is added in the sub-directory file of its parent directory;

If B93. the sub-directory call number of this parent directory index node is not equal to 0 then the attribute information of this parent directory index node is sent to the meta data server that the sub-directory call number equals 0 parent directory index node place, upgrade attribute by this parent directory index node;

B94. transmit a reply message.

Described step B10 also further comprises:

B101. according to the parent directory index node, parent directory call number and directory name utilize function A to obtain to create the meta data server identifier of soft link, send to this meta data server then and create soft linking request;

B102. this meta data server is created soft link after receiving and creating soft linking request, then the directory entry information of this soft link is added in the sub-directory file of its parent directory;

If B103. the sub-directory call number of this parent directory index node is not equal to 0 then the attribute information of this parent directory index node is sent to the meta data server that the sub-directory call number equals 0 parent directory index node place, upgrade attribute by this parent directory index node;

B104. transmit a reply message.

Described function A carries out following operation:

A1. utilize hash function to obtain a cryptographic hash according to parent directory call number and directory name;

A2. obtain in the parent directory index node index value metadata corresponding identifier server in the catalogue distribution array with this cryptographic hash as index value;

A3. the meta data server identifier is returned.

Described function B carries out following operation:

B1. operating system asks the mode of mould to obtain number D by the random number that is obtained by random function to the quantity of meta data server;

B2. with second meta data server of D as sub-directory, then order is to catalogue distribution array assignment, meta data server identifier as this meta data server, after the meta data server identifier arrives maximal value, from the meta data server identifier of minimum, circulating is all meta data server assignment;

B3. Returning catalogue distribution array.

Described function C is carried out following operation:

C1. the every sub-directory side-play amount in the catalogue that the adds up distribution array, the side-play amount that is less than or equal to the reference position off-set value of request and the catalogue that adds up up to the side-play amount of the catalogue that adds up adds the reference position off-set value of sub-directory side-play amount greater than request, and writing down this sub-directory sequence number is 1;

C2. every sub-directory side-play amount in the catalogue distribution array continues to add up, add side-play amount less than the reference position off-set value of request and the side-play amount of the catalogue that adds up adds that the sub-directory side-play amount adds side-play amount more than or equal to the reference position off-set value of request up to the side-play amount of the catalogue that adds up, writing down this sub-directory sequence number is 2;

C3. a plurality of sub-directories between sequence number 1 and the sequence number 2 are noted, then a plurality of sub-directory metadata corresponding identifier servers are returned.

Described client-access table is meant to be used for writing down and by which client the index of metadata node is conducted interviews, to prevent when client conducts interviews to index node this index node release.

In the steps A 5, described source meta data server is meant the pivot data server of this appointment.

The invention has the beneficial effects as follows: the metadata management method that splits name space can be distributed to the operation to same catalogue on a plurality of MDS by catalogue being split the mode on all MDS; Rely on the method for hash to come the MDS that handles for metadata selected simultaneously, the metadata but also can accomplish that client only just can accurately locate MDS with existing information of not only can having accomplished evenly to distribute.Adopt this metadata management method when improving the MDS extendability, also to avoid the visit hot issue of same catalogue.

Description of drawings

The fractionation synoptic diagram of Fig. 1 name space.

Embodiment

In order to make purpose of the present invention, technical scheme and advantage clearer,, a kind of metadata management method that splits name space of the present invention is further elaborated below in conjunction with drawings and Examples.Should be appreciated that specific embodiment described herein only in order to explanation the present invention, and be not used in qualification the present invention.

A kind of metadata management method that splits name space of the present invention, as shown in Figure 1, be to be distributed to all meta data server (MetadataServer after adopting the mode that name space is split that catalogue is split into many parts, be called for short MDS) on, a sub-directory (sub_dir) is all arranged on each MDS.The directory entry content difference of different sub_dir storages, the skew of the directory entry of depositing increases progressively according to the sequence number of sub_dir is ascending.Each catalogue is that 0 part is that the head records with the sub_dir sequence number all, and the directory entry of catalogue is recorded place MDS the head and added.When client sends the request relevant with catalogue, select to handle the MDS of this request according to metadata request Hash (hash) value, the metadata request that so can will belong to same catalogue but belong to different sub_dir is transferred to different MDS and is handled, avoid the visit focus of same catalogue with this, simultaneously can be owing to the metadata operation of same catalogue can the concurrent aggregation processing ability of carrying out and then improve many MDS at a plurality of MDS.

Introduce a kind of metadata management method that splits name space of the present invention in detail below in conjunction with above-mentioned target, comprise the following steps:

Step S100 creates file system;

Specify certain MDS as main MDS, create root directory and root index node (root_inode) by main MDS.Concrete steps are as follows:

Step S110, the metadata store catalogue of establishment meta data server, initialization buffer structure;

Step S120 creates the index node (inode) of root directory at the main MDS of appointment, and the call number of its inode (ino) is 1, and making its sub-directory (sub_dir) sequence number is 0, and the file that creaties directory is in order to deposit the directory entry information of sub_dir0;

Step S130 utilizes function B to determine the sub_dir sequence number of other MDS, makes up the sub_dir distribution array of root directory with this;

Step S140 sends establishment sub_dir request according to sub_dir distribution array to other MDS, additional parent directory call number (dir_ino) and sub_dir sequence number in the request;

Step S150, when other MDS receive that creating sub_dir asks, create the inode of sub_dir correspondence, make its sub_dir sequence number be request indication sequence number, and the file that creaties directory is receiveed the response to source MDS transmission after finishing to deposit the directory entry information of sub_dir sequence number.

In the steps A 5, described source MDS is meant the main MDS of this appointment.

Step S200 manages the file system that has created;

Step S210, client create directory (create dir);

When need createing directory, client carries out following steps:

Step S211 wishes the path that creaties directory to resolve this path by operating system client input client, obtains parent directory index node (dir_inode), parent directory call number (dir_ino) and the directory name (name) of this path correspondence;

Step S212, operating system utilizes function A to obtain the meta data server identifier (MDSID) that creaties directory according to dir_inode, dir_ino and name, sends the request of createing directory to this MDS then;

Step S213, this MDS receive to create directory and ask the node (inode) of back in this MDS establishment new directory correspondence, and making its sub_dir sequence number is 0, and the file that creaties directory is in order to deposit the directory entry information that belongs to sub_dir0;

Step S214 utilizes function B to determine the sub_dir sequence number of other MDS, makes up the sub_dir distribution array of this catalogue with this;

Step S215 sends establishment sub_dir request according to sub_dir distribution array to other MDS, additional dir_ino and sub_dir sequence number in the request;

Step S216 when other MDS receive that creating sub_dir asks, creates corresponding inode, make its sub_dir sequence number be request indication sequence number, and the file that creaties directory is receiveed the response after finishing to source MDS transmission to deposit the directory entry information of sub_dir sequence number;

Step S217, source MDS receive that other MDS respond this catalogue is added in the back in the sub_dir of its parent directory file directory entry information;

Step S218 transmits a reply message;

Step S216 and S217In, described source MDS is meant the MDS that receives the request of createing directory and create the corresponding node of new directory (inode).

Step S220, client is created file (create file);

When need creating file, client carries out following steps:

Step S221, according to dir_inode, dir_ino and name utilize function A to obtain to create the MDSID of file, send to MDS then and create file request;

Step S222, MDS receive and create the inode that creates the file correspondence after the file request at this MDS;

Step S223 adds the directory entry information of inode in the sub_dir file of its parent directory;

If step S224 is the sub-directory call number (sub_dir_no) of this dir_inode!=0 attribute information with this dir_inode sends to the MDS at the dir_inode place of sub_dir_no=0, upgrades attribute by this dir_inode;

Step S225 transmits a reply message;

Step S230, client executing query manipulation (lookup);

When need doing query manipulation, client carries out following steps:

Step S231, according to dir_inode, dir_ino and name utilize function A to obtain the MDSID at query object place, send the inquiry file request to this MDS then;

Step S232, MDS receive after the inquiry file request query object in this MDS;

Step S233, client puts 1 in the corresponding position of the client-access table of the inode that inquires;

Step S234 gives client with the gained information-reply;

Step S240, client executing is read catalogue (READDIR) operation;

Need do when client and to carry out following steps when reading directory operation:

Step S241 judges its inode place MDS according to dir_ino, sends the readdir request to this MDS;

Step S242 after this MDS receives request, finds the inode of dir_ino correspondence, utilizes function C to determine which MDS the sub_dir at interval place that readdir reads is distributed in by inode;

Step S243 sends the readdir order to the MDS at the directory entry place of required reading section;

Step S244, each MDS reads the directory entry content according to the reference position off-set value (offset) and the side-play amount (size) of the request in the readdir request, sends back to source MDS again;

Step S245, source MDS collect the request back and send a reply information to client;

Step S250, client deleted file (remove file);

Among the step B5, described source MDS is meant and receives the meta data server of reading catalog request and reading catalog request to other meta data servers transmissions.

When need making deleted file, client carries out following steps:

Step S251, according to dir_inode, dir_ino and name utilize function A to obtain the MDSID at this document place, send the deleted file request to this MDS then;

Step S252, this MDS finds deleted file after receiving the deleted file request, and the directory entry of this document is deleted from the sub_dir file of its dir_inode;

=0 attribute information with this dir_inode sends to the MDS at the dir_inode place of sub_dir_no=0, upgrades attribute by this dir_inode.

Step S260, client deltree (remove dir);

Need do when client and to carry out following steps when deltreeing:

Step S261, according to dir_inode, dir_ino and name utilize function A to obtain the MDSID at this catalogue place, send the request of deltreeing to MDS then;

Step S262, this MDS find deleted catalogue after receiving the request of deltreeing, and judge that whether this catalogue is empty, for sky then is changed to the deletion state with this catalogue, otherwise returns error message;

Step S263 deletes the directory entry of this catalogue from the sub_dir file of its dir_inode;

=0 attribute information with this dir_inode sends to the MDS at the dir_inode place of sub_dir_no=0, upgrades attribute by this dir_inode.

Step S270, client executing releasing operation (release);

When need doing releasing operation, client carries out following steps:

Step S271 gets 10 of the Far Lefts of its binary representation according to call number (ino), is converted to decimal number with these 10, and this value then is the MDSID at ino indication index node (inode) place, sends the request of release to this MDS then;

Step S272, this MDS find inode according to ino after receiving the request of release, with the position of requesting client correspondence in the client-access table of inode, put 0;

Step S273 judges whether the client-access table of this inode and the link number (nlink) in the attribute are 0, as is 0, then discharges this inode;

Step S2731 if this inode is a file, then discharges file metadata corresponding storage file and inode buffer memory, transmit a request to the data storage file deletion of data storage server with the file correspondence;

Step S2732, if this inode is a catalogue, then with catalogue on all MDS the sub_dir file and the buffer memory of the inode of all sub-directories discharge.

Step S280, client executing rename (rename) operation;

When need doing the operation of rename name, client carries out following steps:

Step S281, according to old parent directory index node (old_dir_inode), old parent directory call number (old_dir_ino) and old directory name (oldname), utilize function A to obtain this old parent directory place MDS1, according to new parent directory index node (new_dir_inode), new parent directory call number (new_dir_ino) and new directory title (newname) utilize function A to obtain new parent directory place MDS2.The MDSID of MDS1 and MDS2 is sent the rename request together with the rename required parameter to MDS1;

Step S282 after this MDS receives the rename request, finds deleted ancient deed, and the directory entry of this document is deleted from the sub_dir file of old parent directory;

=0 attribute information with this old_dir_inode sends to the MDS at the old_dir_inode place of sub_dir_no=0, upgrades attribute by this old_dir_inode;

Step S284 if MDS2 is identical with MDS1, then continues operation, otherwise this request is forwarded on the MDS2 on same MDS;

=0 attribute information with this new_dir_inode sends to the MDS at the new_dir_inode place of sub_dir_no=0, upgrades attribute by this new_dir_inode;

Step S286 sends successfully response to MDS1 after finishing, and MDS1 receives that responding the back sends echo message to client.

Step S290, client is created hard link (link);

When need doing the operation of establishment hard link, client carries out following steps:

Step S291, according to dir_inode, dir_ino and name utilize function A to obtain to create the MDSID of hard link, send the hard link request of creating to this MDS then;

Step S292, this MDS add the directory entry information of this hard link in the sub_dir file of its parent directory after receiving and creating the hard link request;

=0 attribute information with this dir_inode sends to the MDS at the dir_inode place of sub_dir_no=0, upgrades attribute by this dir_inode;

Step S294 transmits a reply message.

Step S2100, client is created soft link (symlink);

When need doing the soft linked operation of establishment, client carries out following steps:

Step S2101, according to dir_inode, dir_ino and name utilize function A to obtain to create the MDSID of soft link, send to this MDS then and create soft linking request;

Step S2102, this MDS create soft link after receiving and creating soft linking request, then the directory entry information of this soft link are added in the sub_dir file of its parent directory;

=0 attribute information with this dir_inode sends to the MDS at the dir_inode place of sub_dir_no=0, upgrades attribute by this dir_inode;

Step S2104 transmits a reply message.

In this method, described function A carries out following operation:

(1) utilize the hash function to obtain a hash value according to dir_ino and name;

(2) obtain among the dir_inode MDSID of index value correspondence in the catalogue distribution array with this hash value as index value;

(3) MDSID is returned.

In this method, described function B carries out following operation:

(1) operating system asks the mode of mould to obtain number D by the random number that is obtained by random function to the quantity (MDS_num) of MDS;

(2) with second MDS of D as sub_dir, then order is to catalogue distribution array assignment, and as the DMSID of this MDS, after MDSID arrived maximal value, from the MDSID of minimum, circulating was all MDS assignment;

(3) Returning catalogue distribution array.

In this method, described function C is carried out following operation:

(1) add up every sub-directory side-play amount (sub_dir_size) in the catalogue distribution array, up to the reference position off-set value (offset) and the total_size+sub_dir_size>offset of side-play amount (the total_size)<=request of the catalogue that adds up, writing down this sub_dir sequence number is 1;

(2) continue to add up every sub_dir_size in the catalogue distribution array, up to total_size<offset+size and total_size+sub_dir_size>=offset+size, writing down this sub_dir sequence number is 2;

(3) a plurality of sub_dir between sequence number 1 and the sequence number 2 are noted, the MDSID with a plurality of sub_dir correspondences returns then.

In this method, described client-access table is meant to be used for writing down and by which client metadata inode is conducted interviews, to prevent when client conducts interviews to inode this inode release.

Beneficial effect of the present invention is: the metadata management method that splits name space can be distributed to the operation to same catalogue on a plurality of MDS by catalogue being split the mode on all MDS; Rely on the method for hash to come the MDS that handles for metadata selected simultaneously, the metadata but also can accomplish that client only just can accurately locate MDS with existing information of not only can having accomplished evenly to distribute.Adopt this metadata management method when improving the MDS extendability, also to avoid the visit hot issue of same catalogue.

In conjunction with the drawings to the description of the specific embodiment of the invention, others of the present invention and feature are conspicuous to those skilled in the art.

More than specific embodiments of the invention are described and illustrate it is exemplary that these embodiment should be considered to it, and be not used in and limit the invention, the present invention should make an explanation according to appended claim.

Claims (12)

1. a metadata management method that splits name space is characterized in that, comprises the following steps:
A1. create the metadata store catalogue of meta data server, the initialization buffer structure;
A2. create the index node of root directory on the pivot data server of appointment, the call number of its index node is 1, and making its sub-directory sequence number is 0, and the file that creaties directory is 0 directory entry information to deposit the sub-directory sequence number;
A3. utilize function B to determine the sub-directory sequence number of other meta data servers, make up the sub-directory distribution array of root directory with this;
A4. send to other meta data servers according to sub-directory distribution array and create the sub-directory request, additional parent directory call number and sub-directory sequence number in the request;
A5. when other meta data servers are received the request of establishment sub-directory, create the index node of sub-directory correspondence, make its sub-directory sequence number be request indication sequence number, and the file that creaties directory is receiveed the response after finishing to described pivot data server transmission to deposit the directory entry information of sub-directory sequence number;
Function B carries out following operation:
B1. operating system asks the mode of mould to obtain number D by the random number that is obtained by random function to the quantity of meta data server;
B2. with second meta data server of D as sub-directory, then order is to catalogue distribution array assignment, meta data server identifier as this meta data server, after the meta data server identifier arrives maximal value, from the meta data server identifier of minimum, circulating is all meta data server assignment;
B3. Returning catalogue distribution array;
Step B, the file system that has created is managed, and described management comprises that client creaties directory, and client is created file, the client executing query manipulation, client executing is read directory operation, the client deleted file, and client deltrees, the client executing releasing operation, client executing rename operation, client is created hard link, and client is created soft link;
Send request for the request of createing directory, establishment file request, when carrying out query manipulation request, deleted file request, the request of deltreeing, carry out the rename operation requests, create the hard link request or creating soft linking request in client, utilize function A to determine the meta data server of processes said request
Described function A carries out following operation:
A1. utilize hash function to obtain a cryptographic hash according to parent directory call number and directory name;
A2. obtain in the parent directory index node index value metadata corresponding identifier server in the catalogue distribution array with this cryptographic hash as index value;
A3. the meta data server identifier is returned;
When described client transmission request is read the directory operation request for carrying out, utilize function C to determine the meta data server of processes said request,
Described function C is carried out following operation:
C1. the every sub-directory side-play amount in the catalogue that the adds up distribution array, the side-play amount that is less than or equal to the reference position off-set value of request and the catalogue that adds up up to the side-play amount of the catalogue that adds up adds the reference position off-set value of sub-directory side-play amount greater than request, and writing down this sub-directory sequence number is 1;
C2. every sub-directory side-play amount in the catalogue distribution array continues to add up, add side-play amount less than the reference position off-set value of request and the side-play amount of the catalogue that adds up adds that the sub-directory side-play amount adds side-play amount more than or equal to the reference position off-set value of request up to the side-play amount of the catalogue that adds up, writing down this sub-directory sequence number is 2;
C3. a plurality of sub-directories between sequence number 1 and the sequence number 2 are noted, then a plurality of sub-directory metadata corresponding identifier servers are returned.
2. method according to claim 1 is characterized in that,
Client sends and asks to be the request of createing directory among the described step B,
Described step B further comprises:
B11. wish the path that creaties directory to resolve this path client input client, obtain parent directory index node, parent directory call number and the directory name of this path correspondence by operating system;
B12. operating system utilizes function A to obtain the meta data server identifier that creaties directory according to parent directory index node, parent directory call number and directory name, sends the request of createing directory to this meta data server then;
B13. this meta data server receives the new directory correspondence is created in the request back of createing directory on this meta data server node, and making its sub-directory sequence number is 0, and to belong to the sub-directory sequence number be 0 directory entry information to the file that creaties directory in order to deposit;
B14. utilize function B to determine the sub-directory sequence number of other meta data servers, make up the sub-directory distribution array of this catalogue with this;
B15. send to other meta data servers according to sub-directory distribution array and create the sub-directory request, additional parent directory call number and sub-directory sequence number in the request;
B16. when other meta data servers are received the request of establishment sub-directory, create corresponding index node, make its sub-directory sequence number be request indication sequence number, and the file that creaties directory is receiveed the response to the meta data server transmission that receives request of createing directory and establishment new directory manipulative indexing node after finishing to deposit the directory entry information of sub-directory sequence number;
B17. the meta data server that receives request of createing directory and establishment new directory manipulative indexing node is received the directory entry information of adding these catalogues after other meta data servers are responded in the sub-directory file of its parent directory;
B18. transmit a reply message.
3. method according to claim 1 is characterized in that,
Client sends and asks to be the establishment file request among the described step B,
Described step B further comprises:
B21. according to the parent directory index node, parent directory call number and directory name utilize function A to obtain to create the meta data server identifier of file, send to meta data server then and create file request;
B22. meta data server receives and creates the index node of creating the file correspondence after the file request on this meta data server;
B23. the directory entry information of index node is added in the sub-directory file of its parent directory;
If B24. the sub-directory call number of this parent directory index node is not equal to 0 then the attribute information of this parent directory index node is sent to the meta data server that the sub-directory call number equals 0 parent directory index node place, upgrade attribute by this parent directory index node;
B25. transmit a reply message.
4. method according to claim 1 is characterized in that,
Client sends and asks to be the request of execution query manipulation among the described step B,
Described step B further comprises:
B31. according to the parent directory index node, parent directory call number and directory name utilize function A to obtain the meta data server identifier at query object place, send the inquiry file request to this meta data server then;
B32. meta data server receives after the inquiry file request query object in this meta data server;
B33. client puts 1 in the corresponding position of the client-access table of the index node that inquires;
B34. give client with the gained information-reply.
5. method according to claim 1 is characterized in that,
Client sends request and reads the directory operation request for execution among the described step B,
Described step B further comprises:
B41. judge the meta data server at its index node place according to the parent directory call number, send to this meta data server and read catalog request;
B42. after this meta data server is received request, find the index node of parent directory call number correspondence, utilize function C to determine which meta data server the sub-directory of reading interval place that catalogue is read is distributed in by index node;
B43. described meta data server is read directory command to the meta data server transmission at the directory entry place of required reading section;
B44. each meta data server is read the directory entry content according to the reference position off-set value and the side-play amount of the request in the read command request, sends back to send the meta data server that directory command is read in described transmission again;
B45. sending described transmission reads to send a reply information to client after the meta data server of directory command is collected request.
6. method according to claim 1 is characterized in that,
Client sends and asks to be the deleted file request among the described step B,
Described step B further comprises:
B51. according to the parent directory index node, parent directory call number and directory name utilize function A to obtain the meta data server identifier at this document place, send the deleted file request to this meta data server then;
B52. after this meta data server receives the deleted file request, find deleted file, the directory entry of this document is deleted from the sub-directory file of his father's directory index node;
B53. revise the property value of the parent directory index node on the same meta data server, if the sub-directory call number of this parent directory index node is not equal to 0 then the attribute information of this parent directory index node is sent to the meta data server that the sub-directory call number equals 0 parent directory index node place, upgrade attribute by this parent directory index node.
7. method according to claim 1 is characterized in that,
Client sends and asks to be the request of deltreeing among the described step B,
Described step B further comprises:
B61. according to the parent directory index node, parent directory call number and directory name utilize function A to obtain the meta data server identifier at this catalogue place, send the request of deltreeing to meta data server then;
B62. after this meta data server receives the request of deltreeing, find deleted catalogue, judge that whether this catalogue is empty, for sky then is changed to the deletion state with this catalogue, otherwise returns error message;
B63. the directory entry of this catalogue is deleted from the sub-directory file of his father's directory index node;
B64. revise the property value of the parent directory index node on the same meta data server, if the sub-directory call number of this parent directory index node is not equal to 0 then the attribute information of this parent directory index node is sent to the meta data server that the sub-directory call number equals 0 parent directory index node place, upgrade attribute by this parent directory index node.
8. method according to claim 1 is characterized in that,
Client sends and asks to be the request of execution releasing operation among the described step B,
Described step B further comprises:
B71. get 10 of the Far Lefts of its binary representation according to call number, be converted to decimal number with these 10, this value then is the meta data server identifier at call number indication index node place, sends the request of release to this meta data server then;
B72. after this meta data server receives the request of release, find index node,, put 0 the position of requesting client correspondence in the client-access table of index node according to call number;
Whether client-access table and the link number in the attribute of B73. judging this index node are 0, as are 0, then discharge this index node;
If B731. this index node is a file, then the buffer memory with file metadata corresponding storage file and index node discharges, and transmit a request to the data storage file deletion of data storage server with the file correspondence;
If B732. this index node is a catalogue, then with catalogue on all meta data servers the sub-directory file and the buffer memory of the index node of all sub-directories discharge.
9. method according to claim 1 is characterized in that,
Client sends and asks to be the rename request among the described step B,
Described step B further comprises:
B81. according to old parent directory index node, old parent directory call number and old directory name, utilize function A to obtain the meta data server 1 at this old parent directory place, according to new parent directory index node, new parent directory call number and new directory title, utilize function A to obtain the meta data server 2 at new parent directory place, the meta-data identifier of meta data server 1 and meta data server 2 is sent the rename request together with the rename required parameter to meta data server 1;
B82. after this meta data server receives the rename request, find, the directory entry of this document is deleted from the sub-directory file of old parent directory deleted ancient deed;
B83. revise the property value of old parent directory index node, if the sub-directory call number of this old parent directory index node is not equal to 0 then attribute information that will this old parent directory index node sends to the meta data server that the sub-directory call number is not equal to 0 old parent directory index node place, upgrade attribute by this old parent directory index node;
If B84. meta data server 2 is identical with meta data server 1, then on same meta data server, continues operation, otherwise this request is forwarded on the meta data server 2;
B85. on meta data server 2, add the directory entry of the new name correspondence of Rename file to the sub-directory file of new parent directory index node, if the sub-directory call number of this new parent directory index node is not equal to 0 then attribute information that will this new parent directory index node sends to the meta data server that the sub-directory call number is not equal to 0 new parent directory index node place, upgrade attribute by this new parent directory index node;
B86. send successfully response to meta data server 1 after finishing, meta data server 1 receives that responding the back sends echo message to client.
10. method according to claim 1 is characterized in that,
Client sends and asks to be the request of establishment hard link among the described step B,
Described step B further comprises:
B91. according to the parent directory index node, parent directory call number and directory name utilize function A to obtain to create the meta data server identifier of hard link, send the hard link request of creating to this meta data server then;
B92. after this meta data server receives and creates the hard link request, the directory entry information of this hard link is added in the sub-directory file of its parent directory;
If B93. the sub-directory call number of this parent directory index node is not equal to 0 then the attribute information of this parent directory index node is sent to the meta data server that the sub-directory call number equals 0 parent directory index node place, upgrade attribute by this parent directory index node;
B94. transmit a reply message.
11. method according to claim 1 is characterized in that,
Client sends and asks to be the soft linking request of establishment among the described step B,
Described step B further comprises:
B101. according to the parent directory index node, parent directory call number and directory name utilize function A to obtain to create the meta data server identifier of soft link, send to this meta data server then and create soft linking request;
B102. this meta data server is created soft link after receiving and creating soft linking request, then the directory entry information of this soft link is added in the sub-directory file of its parent directory;
If B103. the sub-directory call number of this parent directory index node is not equal to 0 then the attribute information of this parent directory index node is sent to the meta data server that the sub-directory call number equals 0 parent directory index node place, upgrade attribute by this parent directory index node;
B104. transmit a reply message.
12., it is characterized in that described client-access table is meant to be used for writing down and by which client the index of metadata node is conducted interviews, to prevent when client conducts interviews to index node this index node release according to claim 4 or 8 described methods.
CN200710308573A 2007-12-29 2007-12-29 Metadata management method for splitting name space CN100595761C (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN200710308573A CN100595761C (en) 2007-12-29 2007-12-29 Metadata management method for splitting name space

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN200710308573A CN100595761C (en) 2007-12-29 2007-12-29 Metadata management method for splitting name space

Publications (2)

Publication Number Publication Date
CN101196929A CN101196929A (en) 2008-06-11
CN100595761C true CN100595761C (en) 2010-03-24

Family

ID=39547341

Family Applications (1)

Application Number Title Priority Date Filing Date
CN200710308573A CN100595761C (en) 2007-12-29 2007-12-29 Metadata management method for splitting name space

Country Status (1)

Country Link
CN (1) CN100595761C (en)

Families Citing this family (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102055795A (en) * 2010-11-04 2011-05-11 天津曙光计算机产业有限公司 Distributed file system metadata management method
CN102110146B (en) * 2011-02-16 2012-11-14 清华大学 Key-value storage-based distributed file system metadata management method
CN102438004B (en) * 2011-09-05 2017-02-08 深圳市创维软件有限公司 Method and system for acquiring metadata information of media file and multimedia player
CN103942205B (en) * 2013-01-18 2018-06-05 深圳市腾讯计算机系统有限公司 Storage, the method, apparatus and system for reading directory index
CN104090897B (en) * 2013-12-19 2016-05-04 深圳市腾讯计算机系统有限公司 Method, server and the system of accesses meta-data
CN105354304B (en) * 2015-11-06 2018-10-02 上海视云网络科技有限公司 A kind of caching LRU, method based on the link of Linux file system file
CN105550371A (en) * 2016-01-27 2016-05-04 华中科技大学 Big data environment oriented metadata organization method and system
CN106446197B (en) * 2016-09-30 2019-11-19 华为数字技术(成都)有限公司 A kind of date storage method, apparatus and system
CN106844578A (en) * 2017-01-07 2017-06-13 郑州云海信息技术有限公司 The method and device that subtree root is inquired about is loaded to meta data server

Also Published As

Publication number Publication date
CN101196929A (en) 2008-06-11

Similar Documents

Publication Publication Date Title
JP2019220195A (en) System and method for implementing a data storage service
US20160057229A1 (en) Appending to files via server-side chunking and manifest manipulation
ES2535943T3 (en) Propagation of reference count
EP2659377B1 (en) Adaptive index for data deduplication
US9052962B2 (en) Distributed storage of data in a cloud storage system
US20190370245A1 (en) System and method for conditionally updating an item with attribute granularity
US20160142512A1 (en) Indirection objects in a cloud storage system
Wang et al. Indexing multi-dimensional data in a cloud system
US10193967B2 (en) Redirecting devices requesting access to files
US8161085B2 (en) Automatic and dynamic provisioning of databases
CN103116618B (en) Based on Telefile mirror method and the system of the lasting buffer memory of client
US8645553B2 (en) Partially replicated, locally searched peer to peer file sharing system
US20150180948A1 (en) United cloud disk client, server, system and united cloud disk serving method
US9372911B2 (en) System and method for performing replica copying using a physical copy mechanism
Zeng et al. Research on cloud storage architecture and key technologies
US7487551B2 (en) Access to content addressable data over a network
US8819027B1 (en) System and method for partitioning and indexing table data using a composite primary key
Yang et al. Comparing hybrid peer-to-peer systems
US8775464B2 (en) Method and system of mapreduce implementations on indexed datasets in a distributed database environment
JP5559867B2 (en) Restore differential files and systems from peers and the cloud
CA2512312C (en) Metadata based file switch and switched file system
US7383286B2 (en) File management system with parent directory search functions
Ng et al. PeerDB: A P2P-based system for distributed data sharing
CN103150394B (en) Distributed file system metadata management method facing to high-performance calculation
US8370454B2 (en) Retrieving a replica of an electronic document in a computer network

Legal Events

Date Code Title Description
PB01 Publication
C06 Publication
SE01 Entry into force of request for substantive examination
C10 Entry into substantive examination
GR01 Patent grant
C14 Grant of patent or utility model
CB03 Change of inventor or designer information

Inventor after: Xing Jing

Inventor after: Tang Rongfeng

Inventor after: Xiong Jin

Inventor after: Ma Jie

Inventor before: Xing Jing

Inventor before: Tang Rongfeng

Inventor before: Xiong Jin

Inventor before: Ma Jie

C53 Correction of patent for invention or patent application