CN105677892B - A kind of method and device reading catalogue subitem metadata - Google Patents

A kind of method and device reading catalogue subitem metadata Download PDF

Info

Publication number
CN105677892B
CN105677892B CN201610070587.2A CN201610070587A CN105677892B CN 105677892 B CN105677892 B CN 105677892B CN 201610070587 A CN201610070587 A CN 201610070587A CN 105677892 B CN105677892 B CN 105677892B
Authority
CN
China
Prior art keywords
metadata
catalogue
subitem
metadata object
destination server
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201610070587.2A
Other languages
Chinese (zh)
Other versions
CN105677892A (en
Inventor
胡毅
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Huawei Technologies Co Ltd
Original Assignee
Huawei Technologies Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huawei Technologies Co Ltd filed Critical Huawei Technologies Co Ltd
Priority to CN201610070587.2A priority Critical patent/CN105677892B/en
Publication of CN105677892A publication Critical patent/CN105677892A/en
Application granted granted Critical
Publication of CN105677892B publication Critical patent/CN105677892B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Abstract

A kind of method and device reading catalogue subitem metadata, for solving the problems, such as that reading catalogue subitem metadata takes a long time.This method comprises: destination server receives the first request that client is sent, the first request includes the positioning identifier of k-th of the metadata object pointed in few two metadata objects;The attribute information of at least two metadata objects is obtained according to the mark of the first catalogue;Determine that this need to read catalogue subitem metadata of the kth into (k+m) a metadata object according to single delivery quantity threshold value and attribute information;According to, for determining the information of the respective storage location of metadata object, obtaining the catalogue subitem metadata that kth to (k+m) a metadata object includes from least one server parallel in attribute information;This is formed according to the catalogue subitem metadata of acquisition and delivers metadata, and is sent this to client and delivered data.

Description

A kind of method and device reading catalogue subitem metadata
Technical field
The present invention relates to field of computer technology, in particular to a kind of method and device for reading catalogue subitem metadata.
Background technique
In distributed file system, a catalogue includes heap file or subdirectory, is referred to as the subitem of catalogue, mesh The meta-data preservation of subitem is recorded in metadata object.If by the meta-data preservation of all subitems of catalogue in the same metadata It, can be because the alternative of object accesses causes the concurrency of business low in object.Therefore, usually by the catalogue subitem member of catalogue Data dispersion is stored among multiple metadata objects, several catalogue subitem metadata are preserved in each metadata object.
As shown in Figure 1, all catalogue subitem metadata of catalogue be stored according to the sequence of catalogue subitem metadata it is multiple Among metadata object, that is, 1~s of the 1st metadata object storing directory in order1A catalogue subitem metadata, the 2nd (the s of a metadata object storing directory in order1+ 1)~s2A catalogue subitem metadata, and so on.
Server in file system is receiving the catalogue subitem metadata for the first catalogue of reading that client is sent When request, the catalogue subitem metadata stored in the 1st metadata object is read in order first, judge the catalogue read Whether metadata fills up agreement buffer area, and the size of the agreement buffer area is also known as single delivery quantity threshold value, be server with The maximum for the single referable catalogue subitem metadata that client is negotiated.If catalogue subitem member in the 1st metadata object The catalogue subitem metadata in the 2nd metadata object is then read in the unfilled agreement buffer area of data in order, and so on, Until filling up agreement buffer area, the catalogue subitem metadata read is sent to customer side.
But the method for above-mentioned reading catalogue subitem metadata takes a long time in the prior art, leads to distributed field system The efficiency of system is lower.
Summary of the invention
The embodiment of the present invention provides a kind of method and device for reading catalogue subitem metadata, for solving in the prior art Read the problem of catalogue subitem metadata takes a long time.
In a first aspect, the embodiment of the present invention provides a kind of method for reading catalogue subitem metadata, the catalogue of the first catalogue Subitem metadata is located in multiple metadata objects, and multiple metadata objects are stored among multiple servers, the method packet Include: destination server receives the first request that client is sent, and first request reads first catalogue for requesting Catalogue subitem metadata, first request include the mark and positioning identifier of the first catalogue, and the positioning identifier is directed toward institute K-th of metadata object at least two metadata objects is stated, shows to read catalogue subitem since k-th of metadata object Metadata;The destination server obtains the metadata of first catalogue according to the mark of first catalogue, and described first The metadata of catalogue includes the attribute information of at least two metadata object, and the attribute information includes the first catalogue of storage Catalogue subitem metadata the catalogue subitem metadata quantity that respectively contains of metadata object, and the above-mentioned metadata pair of management The information of the metadata management node of elephant;The destination server is according in single delivery quantity threshold value and the attribute information K-th of metadata object and k-th of metadata object after the catalogue subitem that respectively contains of metadata object The quantity of metadata determines that this need to read catalogue subitem metadata of the kth into (k+m) a metadata object, wherein institute State the maximum that single delivery quantity threshold value is the catalogue subitem metadata that the destination server single can be sent to the client Amount, m is positive integer;The destination server is used to determine the kth to (k+m) a first number according in the attribute information According to the information of the respective storage location of object, the kth is obtained from least one described server parallel to (k+m) a first number The catalogue subitem metadata for including according to object;The destination server delays the catalogue subitem metadata deposit agreement of acquisition Area is rushed, this is formed and delivers metadata, and send described this to the client and deliver data, wherein this described delivery The size of metadata is not more than the single delivery quantity threshold value.
In above-mentioned implementation, destination server is when reading the catalogue subitem metadata of the first catalogue, according to each member The quantity and single delivery quantity threshold value for the catalogue subitem metadata that data object includes, determine this multiple member that need to be read Data object, then, and meanwhile it is sub from the catalogue read from corresponding metadata management node in determining multiple metadata objects Item metadata.With in the prior art using serial mode read catalogue subitem metadata by the way of compared with, can substantially reduce mesh The read access time for recording subitem metadata, improves the efficiency of file system.
With reference to first aspect, in the first possible implementation of the first aspect, the destination server is according to list After k-th of metadata object and k-th of metadata object in secondary delivery quantity threshold value and the attribute information The quantity of catalogue subitem metadata that respectively contains of metadata object, determine that this need to read kth to (k+m) a metadata Catalogue subitem metadata in object, comprising: the destination server determines: i-th of catalogue in k-th of metadata object Subitem metadata to (k+m-1) a metadata object the last one catalogue subitem metadata total amount less than the single hand over I-th of catalogue subitem metadata is paid in amount threshold value and k-th of metadata object to (k+m) a metadata object The total amount of the last one catalogue subitem metadata is not less than the single delivery quantity threshold value;The destination server determines that this is needed Read catalogue subitem metadata of the kth into (k+m) a metadata object.
With reference to first aspect or the first possible implementation of first aspect, second in first aspect are possible In implementation, the destination server sends this to the client and delivers metadata, comprising: the destination server is sentenced It is disconnected described this deliver the last one catalogue subitem metadata in metadata whether be first catalogue the last one mesh Record subitem metadata;If it is not, then the destination server sends described this to the client and delivers metadata, and indicate The client continues that described this is requested to deliver the last one catalogue subitem metadata of metadata to the destination server Catalogue subitem metadata later, guarantee agreement client can obtain all catalogue subitem metadata of the first catalogue.
Second aspect, the embodiment of the present invention provide a kind of catalogue of the first catalogue of device for reading catalogue subitem metadata Metadata is located at least two metadata objects, at least two metadata object be stored at least one server it In, described device includes: receiving module, and for receiving the first request of client transmission, first request is read for requesting The catalogue subitem metadata of first catalogue is taken, first request includes the mark and positioning identifier of the first catalogue, institute K-th of metadata object that positioning identifier is directed toward at least two metadata object is stated, k is positive integer;First obtains mould Block obtains the metadata of first catalogue, the metadata packet of first catalogue for the mark according to first catalogue Attribute information containing at least two metadata object;First determining module, for according to single delivery quantity threshold value, Yi Jisuo The metadata object after k-th of metadata object and k-th of metadata object in attribute information is stated respectively to wrap The quantity of the catalogue subitem metadata contained determines that this need to read catalogue subitem member of the kth into (k+m) a metadata object Data, wherein the single delivery quantity threshold value is the catalogue subitem metadata that described device single can be sent to the client Maximum, m is positive integer;Second obtains module, for being used to determine the kth to (k according in the attribute information + m) the respective storage location of a metadata object information, the kth is obtained from least one described server parallel to (k + m) a metadata object catalogue subitem metadata that includes;Second determining module, for the catalogue subitem member according to acquisition Data form this and deliver metadata, this described size for delivering metadata is not more than the single delivery quantity threshold value;It sends Module delivers data for sending described this to the client.
In conjunction with second aspect, in the first possible implementation of the second aspect, first determining module is specific For: determine in k-th of metadata object i-th of catalogue subitem metadata to (k+m-1) a metadata object most The total amount of the latter catalogue subitem metadata is less than i-th in the single delivery quantity threshold value and k-th of metadata object The total amount of the last one catalogue subitem metadata of a catalogue subitem metadata to (k+m) a metadata object is not less than described Single delivery quantity threshold value;Determine that this need to read catalogue subitem metadata of the kth into (k+m) a metadata object.
In conjunction with the possible implementation of the first of second aspect or second aspect, second in second aspect is possible In implementation, described device further include: judgment module, for judging that described this delivers the last one catalogue in metadata Subitem metadata whether be first catalogue the last one catalogue subitem metadata;If it is not, then passing through the transmission mould The client that block indicates continues that described this is requested to deliver the last one catalogue subitem metadata of metadata to described device Catalogue subitem metadata later.
In some possible implementations, the positioning identifier is directed toward i-th of mesh in k-th of metadata object Subitem metadata is recorded, for showing that i-th of catalogue subitem metadata is the first catalogue subitem metadata that need to be read.It is above-mentioned Possible implementation can be accurately determined this catalogue subitem metadata that need to be read, convenient for more efficient and more have a needle To property carry out the reading of catalogue subitem metadata.
In some possible implementations, i-th of catalogue subitem metadata described in k-th of metadata object is extremely J-th of catalogue subitem metadata is that described this delivers metadata in (k+m) a metadata object.
Detailed description of the invention
To describe the technical solutions in the embodiments of the present invention more clearly, make required in being described below to embodiment Attached drawing is briefly introduced, it should be apparent that, drawings in the following description are only some embodiments of the invention, for this For the those of ordinary skill in field, without any creative labor, it can also be obtained according to these attached drawings His attached drawing.
Fig. 1 is the schematic diagram of the storage mode of the catalogue subitem metadata of catalogue;
Fig. 2 is the schematic diagram of file system in the embodiment of the present invention;
Fig. 3 is the flow diagram that catalogue subitem metadata approach is read in the embodiment of the present invention;
Fig. 4 is the structural schematic block diagram that catalogue subitem metadata device is read in the embodiment of the present invention;
Fig. 5 is the structural schematic block diagram that catalogue subitem metadata equipment is read in the embodiment of the present invention.
Specific embodiment
Technical solution of the present invention is described in detail below by attached drawing and specific embodiment, it should be understood that the present invention Specific features in embodiment and embodiment are the detailed description to technical solution of the present invention, rather than to the technology of the present invention The restriction of scheme, in the absence of conflict, the technical characteristic in the embodiment of the present invention and embodiment can be combined with each other.
In file system, a catalogue includes heap file or subdirectory, is referred to as the subitem of catalogue, each catalogue Subitem is corresponding with metadata, referred to as catalogue subitem metadata.Metadata object is the object for storing metadata, catalogue Multiple catalogue subitem metadata are dispersed to be stored among multiple metadata objects.Mesh is preserved among the metadata of catalogue itself The information of the subitem of record, wherein just including the metadata object of the metadata (that is, catalogue subitem metadata) of storing directory subitem Attribute information, for example, the number of catalogue subitem metadata of each metadata object storage for another example manages the metadata pair Information of metadata management module of elephant, etc..
Fig. 2 is the schematic diagram of file system in the embodiment of the present invention, wherein each server may each comprise client generation Manage (Client Agent;Referred to as: CA) module, metadata management module and memory module, above-mentioned CA module and metadata Management module can be realized by software entity, or corresponding hardware module (such as integrated chip), above-mentioned memory module It can be all or part of hard disk of server.CA module in Servers-all is formed together CA cluster, in Servers-all Metadata management module form metadata management cluster, the hard disk in Servers-all forms storage cluster.
Wherein, the CA module in server is used to handle the request of the reading catalogue subitem metadata of client transmission, real In the situation of border, communicated by the Protocol Client end module in client with the CA module in server;Memory node is for depositing Metadata object is stored up, and storage has the catalogue subitem metadata of catalogue in metadata object;Metadata management node is for managing Metadata object, CA can read the catalogue subitem metadata in metadata object by metadata management node.
Fig. 3 is the flow diagram provided in an embodiment of the present invention for reading catalogue subitem metadata approach, and this method includes Following steps:
Step 101: destination server receives the first request that client is sent, and the first request reads the first mesh for requesting The catalogue subitem metadata of record, the first request include the mark and positioning identifier of the first catalogue, and positioning identifier is directed toward at least two K-th of metadata object in a metadata object, k are positive integer;
Step 102: destination server obtains the metadata of the first catalogue, the member of the first catalogue according to the mark of the first catalogue Data contain at least two the attribute information of metadata object;
Step 103: destination server is according to k-th of metadata object in single delivery quantity threshold value and attribute information And the quantity of catalogue subitem metadata that the metadata object after k-th of metadata object respectively contains, determine that this needs to read Take catalogue subitem metadata of the kth into (k+m) a metadata object, wherein single delivery quantity threshold value is destination server The maximum for the catalogue subitem metadata that single can be sent to client, m is positive integer;
Step 104: destination server is used to determine that kth is each to (k+m) a metadata object according in attribute information From the information of storage location, the catalogue subitem that kth includes to (k+m) a metadata object is obtained from above-mentioned storage location parallel Metadata;
Step 105: catalogue subitem metadata forms this and delivers metadata destination server based on the received, and to client End sends this and delivers data, wherein this size for delivering metadata is not more than single delivery quantity threshold value.
Specifically, the protocol client transmission of client can be received by the CA module in destination server in step 101 First request.Positioning identifier in first request reads catalogue subitem metadata for showing since which metadata object.
In step 102, after receiving the first request, destination server is first according to the first mesh for including in the first request The mark of record determines the metadata of the first catalogue.In actual conditions, the CA module of destination server may be cached with catalogue member number According to destination server can first search for the metadata of the first catalogue in the caching, can be according to the first catalogue if miss Mark location goes out the metadata of the parent directory of first catalogue, includes the metadata of the first catalogue in the metadata of parent directory Routing information, destination server can determine the metadata of the first catalogue according to the routing information.
It include the information of the subitem of the first catalogue in the metadata of first catalogue, including the son of the first catalogue of storage The attribute information of the metadata object of the metadata (that is, catalogue subitem metadata of the first catalogue) of item, attribute information includes should Which catalogue subitem metadata of the first catalogue and the metadata pipe of metadata object ownership are housed in metadata object Manage the information of module.
Then, after destination server knows k-th of metadata object and k-th of metadata according to the attribute information The metadata number that each metadata includes.
Then, in step 103, the quantity for the catalogue subitem metadata that destination server includes according to above-mentioned metadata object Information and single delivery quantity threshold value, determine which metadata object this need to read.Its specific implementation includes:
First, destination server since k-th of metadata object, carries out the cumulative of catalogue subitem metadata, until mesh The total amount of record subitem metadata reaches the single delivery quantity threshold value.
For example, a kind of feasible algorithm are as follows: calculating k-th of metadata object to (k+m-1) a metadata object Including catalogue subitem metadata total amount be not more than single delivery quantity threshold value, and calculate k-th of metadata object to (k+ M) when the catalogue subitem metadata total amount that a metadata object includes is greater than single delivery quantity threshold value, it can determine that this needs to read The catalogue subitem metadata that k-th of metadata object includes into (k+m) a metadata object.
In another example another feasible algorithm are as follows: calculating k-th of metadata object to (k+m) a metadata pair As the catalogue subitem metadata total amount that includes is no more than single delivery quantity threshold value, and k-th of metadata object is calculated to the (k+m+1) when the catalogue subitem metadata total amount that a metadata object includes is greater than single delivery quantity threshold value, this can be determined The catalogue subitem metadata that k-th of metadata object includes into (k+m) a metadata object need to be read.
Wherein, the first above-mentioned algorithm can guarantee that the catalogue subitem metadata read every time fills up protocol cache area, increase Add the number of the catalogue subitem metadata returned every time to client;And above-mentioned second algorithm can be avoided reading this can not The catalogue subitem metadata of delivery saves transfer resource.
Second, positioning identifier is directed toward i-th of catalogue subitem metadata in k-th of metadata object, for showing i-th Catalogue subitem metadata is the first catalogue subitem metadata that need to be read.
Destination server i-th of catalogue subitem metadata, carries out catalogue subitem member number since k-th of metadata object According to it is cumulative, calculate i-th of catalogue subitem metadata jth into (k+m) a metadata object in k-th of metadata object The total amount of a catalogue subitem metadata is not more than single delivery quantity threshold value, and i-th of catalogue subitem in k-th of metadata object The total amount of metadata (j+1) a catalogue subitem metadata into (k+m) a metadata object is greater than single delivery quantity threshold value, Then determine i-th of catalogue subitem metadata, j-th of catalogue into (k+m) a metadata object in k-th of metadata object Item metadata is that this need to read catalogue subitem metadata.
Above-mentioned second of implementation can be accurately determined this catalogue subitem metadata that need to be read, convenient for more efficient And more targetedly carry out the reading of catalogue subitem metadata.
In actual conditions, when the size for the catalogue subitem metadata for including to metadata object carries out above-mentioned cumulative, i.e., The information of the quantity for the catalogue subitem metadata for needing metadata object to include, it is also desirable to obtain each catalogue subitem metadata The information of size.The size information of catalogue subitem metadata may be embodied in above-mentioned attribute information in metadata object, can also To obtain the average value of catalogue subitem metadata size by destination server, carried out with the average value above-mentioned cumulative.Catalogue subitem The size average value of metadata can be default value;Alternatively, can be estimated according to the size for the catalogue subitem metadata delivered before Calculate the average value of catalogue subitem metadata size.
In addition, when it is implemented, deposit in the size of the catalogue subitem metadata in metadata object, and agreement is consigned to Catalogue subitem metadata size when client may be different, and usual the latter is less than the former, this is because would generally be using compression Algorithm will consign to agreement customer side after catalogue subitem metadata compression.Therefore, according to the size of catalogue subitem metadata When accumulated value determines the metadata object that this need to read, by above-mentioned accumulated value and this delivery quantity threshold value multiplied by deviation factor into Row compares, which can be the ratio of the data volume before and after catalogue subitem metadata compression.
Then, in step 104, destination server includes for determining kth to (k+m) a member according to above-mentioned attribute information The information of the respective storage location of data object, obtains kth to (k+m) a metadata object from least one server parallel The catalogue subitem metadata for including.
Specifically, it is described for determine kth to (k+m) a metadata object respectively storage location information be kth The information of the metadata management module respectively belonged to (k+m) a metadata object, also that is, kth is to (k+m) a metadata Object is respectively managed by the metadata management module of which server.According to the information, that is, it can determine that kth is a to (k+m) Which server is each freedom of metadata object managed, and manages the metadata management module in the server of metadata object and deposit Contain the information of the storage location of the metadata object.
In one possible implementation, destination server determines management kth to (k+m) a metadata object N number of server is requested to N number of server transmitted in parallel, wherein includes in the request sent to s-th of server (k+t) The mark of a metadata object, (k+t) a metadata object are that kth is serviced into (k+m) a metadata object by s-th The metadata object that device is managed, the integer that the value of s is 1 to N, the integer that the value of t is 0 to m.
Then, the position of (k+t) metadata object that s-th of server is stored according to the metadata management module of itself Information reads the catalogue subitem metadata in (k+t) metadata object.It then, will be in (k+t) metadata object of reading Catalogue subitem metadata return to destination server.
It, will after s-th of server receives the request of destination server transmission in alternatively possible implementation The location information of (k+t) metadata object of the metadata management module storage of itself returns to destination server, target clothes After business device receives the location information, voluntarily from the catalogue read from corresponding storage location in (k+t) metadata object Subitem metadata.
Optionally, in above-mentioned first way, destination server is in the service for being sent to k-th of metadata object of management In the request of device, beginning flag may include, which is directed toward i-th of catalogue subitem member in k-th of metadata object Data show to read from i-th of catalogue subitem metadata.The server for receiving request returns to k-th yuan to destination server I-th of catalogue subitem metadata in data object and its catalogue subitem metadata later.
Optionally, in above-mentioned first way, destination server is being sent to management (k+m) a metadata object In the request of server, end mark may include, which is directed toward j-th of mesh in (k+m) a metadata object Record subitem metadata, show in (k+m) a metadata object j-th of catalogue subitem metadata be this last one need to read Catalogue subitem metadata.Receive j-th of the server of request into destination server return (k+m) a metadata object Catalogue subitem metadata and its catalogue subitem metadata before.
It should be noted that storing the memory module of above-mentioned metadata object and managing the metadata of above-mentioned metadata object Management module can on the same server, can also be on different server.Storage metadata object memory module with Manage metadata object metadata management module not on the same server when, manage the metadata management mould of metadata object Server where block sends to the server where the memory module of storage metadata object and obtains in the metadata object The catalogue subitem metadata of acquisition is sent to mesh after obtaining catalogue subitem metadata by the request of catalogue subitem metadata Mark server.
In addition, the metadata management module of destination server may also manage kth into (k+m) a metadata object Metadata object, destination server can pass through the metadata management module of itself and obtain catalogue in corresponding metadata object Subitem metadata.For example, destination server determines (k+t) a metadata object by itself first number according to above-mentioned attribute information Managed according to management module and remaining metadata object by the server except destination server metadata management module institute Management, the then storage location of (k+t) a metadata object that destination server is stored by the metadata management module of itself Information, obtain the catalogue subitem metadata in (k+t) a metadata object from the storage location, and take from the target The catalogue subitem metadata in remaining metadata object is obtained at server except business device.
In step 105, the catalogue subitem metadata of acquisition is stored in agreement buffer area by destination server in order, until filling out Completely laundry buffer area.
In one possible implementation, the size for the catalogue subitem metadata that destination server obtains is equal or close to Size (that is, single delivery quantity threshold value) in agreement buffer area.
In alternatively possible implementation, the size for the catalogue subitem metadata that destination server obtains is more than agreement The size of buffer area, catalogue subitem metadata is stored in agreement buffer area by destination server in order, after being filled with, remaining mesh Recording subitem metadata can be directly to give up there are two types of processing mode, one, secondly to be buffered in the sheet of destination server Among ground caching, so that when client transmission is requested next time, it is stored in agreement buffer area from local cache.
Catalogue subitem member number after the catalogue subitem metadata that will be obtained is stored in agreement buffer area, in agreement buffer area Metadata is delivered according to this is formed, destination server is returned to client.
In the embodiment of the present invention, 102~step 104 of above-mentioned steps can be by the central processing unit of destination server (Central Processing Unit;It referred to as: CPU) executes, can also be executed by the CA module of destination server.
In above-mentioned technical proposal, destination server reads the request of the catalogue subitem metadata of the first catalogue receiving Afterwards, the size and single delivery quantity threshold value for the catalogue subitem metadata for including according to each metadata object, determine this Then the multiple metadata objects that need to be read obtain metadata pair from the server for storing multiple metadata object parallel The catalogue subitem metadata of elephant.With in the prior art using serial mode read catalogue subitem metadata by the way of compared with, can The read access time for substantially reducing catalogue subitem metadata, improves the efficiency of file system.
Optionally, in the embodiment of the present invention, it includes: i-th of catalogue from k-th of metadata object that this, which delivers metadata, The catalogue subitem metadata of subitem metadata j-th of catalogue subitem metadata into (k+m) a metadata object.
Optionally, in the embodiment of the present invention, step 105: destination server sends this to client and delivers metadata, packet Include following steps:
Whether the last one catalogue subitem metadata that destination server judges that this is delivered in metadata is the first catalogue The last one catalogue subitem metadata;
If it is not, then destination server sends this to client and delivers metadata, and indicate that client continues to target Server requests this to deliver the catalogue subitem metadata after the last one catalogue subitem metadata of metadata.
Specifically, destination server after having filled agreement buffer area, judges the last one mesh in agreement buffer area Record subitem metadata whether be the first directory entry the last one catalogue subitem metadata, if it is not, then showing the first catalogue Catalogue subitem metadata, which is not yet read, to be finished, and destination server will refer to when sending catalogue subitem metadata to protocol client Show the catalogue subitem member after client continues the last one the catalogue subitem metadata for requesting this to deliver to destination server Data.Client will send to destination server request again according to instruction, and the positioning identifier in request is directed toward this delivery First catalogue subitem metadata after the last one catalogue subitem metadata.
If the last one catalogue subitem metadata that aforementioned judging result shows that this is delivered in metadata is the first mesh The last one catalogue subitem metadata of record, then destination server can indicate the catalogue subitem of the first catalogue to protocol client Metadata, which has been read, to be finished.Alternatively, destination server, without above-mentioned instruction, client will not receive aforementioned instruction agreement visitor When family end continues to request the instruction information of catalogue subitem metadata to destination server, confirmation, which has been read, to be finished.
Optionally, in the embodiment of the present invention, destination server is before sending this and delivering metadata, first to catalogue subitem Metadata is ranked up.
It is explained below in method of the specific application example to above-mentioned reading catalogue subitem metadata.
The Protocol Client end module of client is in the catalogue subitem metadata for needing to read the first catalogue, to destination service The CA module of device sends the first request, the catalogue subitem of the mark comprising the first catalogue and direction the first catalogue of storage in request The positioning identifier of 1st catalogue subitem metadata of the 1st metadata object in metadata.
CA module determines the metadata of the first catalogue according to the mark of the first catalogue in the first request, the first catalogue Metadata includes following information: the number for the catalogue subitem metadata that each metadata object of the first catalogue of storage respectively contains Mesh.CA module according to above- mentioned information and single delivery quantity threshold value determine this at most can to protocol client deliver the 1st to The catalogue subitem metadata that 5th metadata object is included.
Then, the metadata management section for each metadata object of management for including in metadata of the CA module according to the first catalogue The information of point, determines that the 1st to the 2nd metadata object is managed by metadata management node A, the 3rd to the 5th metadata object It is managed respectively by metadata management node B, metadata management node C, metadata management node D.Wherein, the metadata pipe Reason node A, B, C, D are located at server A, B, C, D.
Then, CA module sends request to metadata management node A to D simultaneously, and requesting node A returns to the 1st to the 2nd member The catalogue subitem metadata that data object is included, requesting node B return to the catalogue subitem member that the 3rd metadata object is included Data, requesting node C return to the catalogue subitem metadata that the 4th metadata object is included, and requesting node D returns to the 5th member (whole or preceding j) catalogue subitem metadata that data object is included.
Then, the catalogue subitem metadata that CA receiving node A to D is respectively returned, is stored in data buffer zone, i.e., in sequence It is first stored in catalogue subitem metadata in the 1st metadata object in order, then is stored in the mesh in the 2nd metadata object in order Subitem metadata is recorded, and so on, until j-th of catalogue subitem metadata of the 5th metadata object of deposit, since data are slow It rushes area to fill up, the catalogue subitem metadata that can not be restored again into after j-th of catalogue subitem metadata of the 5th metadata object. Then the 1st catalogue subitem metadata in the 1st metadata object to the 5th metadata object j-th of catalogue subitem member number Data are delivered according to this is formed.
Then, CA judge the 5th metadata object j-th of catalogue subitem metadata whether be the first catalogue last A catalogue subitem metadata, then after determining it not and being the last one catalogue subitem metadata, CA sends this to protocol client The mark of j-th of catalogue subitem metadata of the 5th metadata object of data and the first catalogue is delivered, the mark is for referring to Show that protocol client continues remaining catalogue subitem metadata (j-th of i.e. the 5th metadata object that the first catalogue is requested to CA The mesh that the metadata object after catalogue subitem metadata and the 5th metadata object after catalogue subitem metadata includes Record subitem metadata).
Protocol Client end module receives the catalogue subitem metadata returned, and is indicated according to CA, continues to send to CA and read The request of first catalogue subitem metadata, the positioning identifier in the request are directed toward+1 catalogue of jth in the 5th metadata object Subitem metadata.
The request of CA response protocol client continues to read what this can be delivered to multiple metadata management nodal parallels Catalogue subitem metadata, until the last one catalogue subitem metadata of the first catalogue is consigned to protocol client.
Based on the same technical idea, the embodiment of the invention also provides a kind of devices for reading catalogue subitem metadata 200, it is the structural schematic block diagram of device 200 referring to fig. 4, device 200 includes:
Receiving module 201, for receiving the first request of client transmission, the first request reads the first catalogue for requesting Catalogue subitem metadata, first request include the first catalogue mark and positioning identifier, positioning identifier be directed toward at least two K-th of metadata object in metadata object, k are positive integer;
First obtains module 202, and the metadata of the first catalogue is obtained for the mark according to the first catalogue, the first catalogue Metadata contains at least two the attribute information of metadata object;
First determining module 203, for according to k-th of metadata pair in single delivery quantity threshold value and attribute information As and k-th of metadata object after the quantity of catalogue subitem metadata that respectively contains of metadata object, determine that this is needed Read catalogue subitem metadata of the kth into (k+m) a metadata object, wherein single delivery quantity threshold value is device single The maximum for the catalogue subitem metadata that can be sent to client, m is positive integer;
Second obtains module 204, for being used to determine kth to (k+m) a metadata object according in attribute information The information of respective storage location, obtains the mesh that kth includes to (k+m) a metadata object from least one server parallel Record subitem metadata;
Second determining module 205 forms this for the catalogue subitem metadata according to acquisition and delivers metadata, this friendship The size for paying metadata is not more than single delivery quantity threshold value;
Sending module 206 delivers data for sending this to client.
Optionally, in the embodiment of the present invention, positioning identifier is directed toward i-th of catalogue subitem member in k-th of metadata object Data, for showing that i-th of catalogue subitem metadata is the first catalogue subitem metadata that need to be read.
Optionally, in the embodiment of the present invention, the first determining module 203 is specifically used for:
Determine that i-th of catalogue subitem metadata in k-th of metadata object is last to (k+m-1) a metadata object The total amount of one catalogue subitem metadata is less than i-th of catalogue subitem in single delivery quantity threshold value and k-th of metadata object Metadata to (k+m) a metadata object the last one catalogue subitem metadata total amount be not less than single delivery quantity threshold Value;
Determine that this need to read catalogue subitem metadata of the kth into (k+m) a metadata object.
Optionally, in the embodiment of the present invention, i-th of catalogue subitem metadata is a to (k+m) in k-th of metadata object J-th of catalogue subitem metadata is that this delivers metadata in metadata object.
Optionally, in the embodiment of the present invention, with continued reference to Fig. 4, device further include:
Judgment module 207, for judging that this delivers whether the last one catalogue subitem metadata in metadata is the The last one catalogue subitem metadata of one catalogue;If it is not, then indicating that client continues to ask to device by sending module 206 This is asked to deliver the catalogue subitem metadata after the last one catalogue subitem metadata of metadata.
Device 200 in the present embodiment and the aforementioned method for reading catalogue subitem metadata be based on the same inventive concept under Two aspect, the implementation process of method is described in detail in front, so those skilled in the art can basis The structure and implementation process of the device 200 in the present embodiment is well understood in foregoing description, in order to illustrate the succinct of book, herein It repeats no more.
Based on the same technical idea, the embodiment of the invention also provides a kind of equipment for reading catalogue subitem metadata 300, it is the structural schematic block diagram of equipment 300 referring to Fig. 5, equipment 300 includes:
Transceiver 301, for receiving the first request of client transmission, the first request reads the first catalogue for requesting Catalogue subitem metadata, the first request include the mark and positioning identifier of the first catalogue, and positioning identifier is directed toward at least two yuan K-th of metadata object in data object, k are positive integer;
Memory 302, for storing instruction;
Processor 303 is connected with transceiver 301 and memory 302 respectively, for executing the instruction in memory, with Following steps are executed when executing instruction:
The metadata of the first catalogue is obtained according to the mark of the first catalogue, the metadata of the first catalogue contains at least two member The attribute information of data object;
According in single delivery quantity threshold value and attribute information k-th of metadata object and k-th metadata object it The quantity for the catalogue subitem metadata that metadata object afterwards respectively contains determines that this need to read kth to (k+m) a first number According to the catalogue subitem metadata in object, wherein single delivery quantity threshold value is the catalogue that server single can be sent to client The maximum of subitem metadata, m are positive integer;
According in attribute information for determining the information of kth to the respective storage location of (k+m) a metadata object, and Row obtains the catalogue subitem metadata that kth to (k+m) a metadata object includes from least one server;
This is formed according to the catalogue subitem metadata of acquisition and delivers metadata, and indicates that transceiver 301 is sent out to client This is given to deliver data, wherein this size for delivering metadata is not more than single delivery quantity threshold value.
Optionally, in the embodiment of the present invention, positioning identifier is directed toward i-th of catalogue subitem member in k-th of metadata object Data, for showing that i-th of catalogue subitem metadata is the first catalogue subitem metadata that need to be read.
Optionally, in the embodiment of the present invention, processor 303 is used for: according in single delivery quantity threshold value and attribute information K-th of metadata object and k-th of metadata object after the metadata object catalogue subitem metadata that respectively contains Quantity determines that this need to read catalogue subitem metadata of the kth into (k+m) a metadata object, comprising:
Determine that i-th of catalogue subitem metadata in k-th of metadata object is last to (k+m-1) a metadata object The total amount of one catalogue subitem metadata is less than i-th of catalogue subitem in single delivery quantity threshold value and k-th of metadata object Metadata to (k+m) a metadata object the last one catalogue subitem metadata total amount be not less than single delivery quantity threshold Value;
Determine that this need to read catalogue subitem metadata of the kth into (k+m) a metadata object.
Optionally, in the embodiment of the present invention, i-th of catalogue subitem metadata is a to (k+m) in k-th of metadata object J-th of catalogue subitem metadata is that this delivers metadata in metadata object.
Optionally, in the embodiment of the present invention, processor 303 is used for: instruction transceiver sends this to client and delivers member Data, comprising:
Judge this deliver metadata in the last one catalogue subitem metadata whether be the first catalogue the last one Catalogue subitem metadata;
If it is not, then indicating that transceiver 301 sends this to client and delivers metadata, and pass through transceiver 301 to finger Show that client continues the catalogue subitem member after the last one catalogue subitem metadata of this delivery metadata of device request Data.
It should be noted that processor 303 can be a processing element, it is also possible to the general designation of multiple processing elements. For example, processor 303 can be central processing unit, it is also possible to specific integrated circuit (Application Specific Intergrated Circuit;Referred to as: ASIC), or be arranged to implement the embodiment of the present invention one or more it is integrated Circuit, such as: one or more microprocessors (digital singnal processor;Referred to as: DSP), or, one or Multiple field programmable gate array (Field Programmable Gate Array;Referred to as: FPGA).
Memory can be a memory element, be also possible to the general designation of multiple memory elements, and executable for storing Parameter, data required for program code, resident's access network equipment or terminal operating etc..And memory may include random access memory (Random-Access Memory;It referred to as: RAM), also may include nonvolatile memory (non-volatile memory; Referred to as: NVM), such as magnetic disk storage, flash memory (Flash) etc..
Equipment 300 in the present embodiment and the aforementioned method for reading catalogue subitem metadata be based on the same inventive concept under Two aspect, the implementation process of method is described in detail in front, so those skilled in the art can basis The structure and implementation process of the equipment 300 in the present embodiment is well understood in foregoing description, in order to illustrate the succinct of book, herein It repeats no more.
The one or more technical solutions provided in the embodiment of the present invention, have at least the following technical effects or advantages:
Destination server is after the request for receiving the catalogue subitem metadata for reading the first catalogue, according to each metadata The size and single delivery quantity threshold value for the catalogue subitem metadata that object includes, determine this multiple metadata that need to be read Then object obtains the catalogue subitem metadata of metadata object parallel from the server for storing multiple metadata object. With in the prior art using serial mode read catalogue subitem metadata by the way of compared with, can substantially reduce catalogue subitem member number According to read access time, improve the efficiency of file system.
It should be understood by those skilled in the art that, the embodiment of the present invention can provide as method, system or computer program Product.Therefore, complete hardware embodiment, complete software embodiment or reality combining software and hardware aspects can be used in the present invention Apply the form of example.Moreover, it wherein includes the computer of computer usable program code that the present invention, which can be used in one or more, The computer program implemented in usable storage medium (including but not limited to magnetic disk storage, CD-ROM, optical memory etc.) produces The form of product.
The present invention be referring to according to the method for the embodiment of the present invention, the process of equipment (system) and computer program product Figure and/or block diagram describe.It should be understood that every one stream in flowchart and/or the block diagram can be realized by computer program instructions The combination of process and/or box in journey and/or box and flowchart and/or the block diagram.It can provide these computer programs Instruct the processor of general purpose computer, special purpose computer, Embedded Processor or other programmable data processing devices to produce A raw machine, so that being generated by the instruction that computer or the processor of other programmable data processing devices execute for real The device for the function of being specified in present one or more flows of the flowchart and/or one or more blocks of the block diagram.
These computer program instructions, which may also be stored in, is able to guide computer or other programmable data processing devices with spy Determine in the computer-readable memory that mode works, so that it includes referring to that instruction stored in the computer readable memory, which generates, Enable the manufacture of device, the command device realize in one box of one or more flows of the flowchart and/or block diagram or The function of being specified in multiple boxes.
Although preferred embodiments of the present invention have been described, it is created once a person skilled in the art knows basic Property concept, then additional changes and modifications may be made to these embodiments.So it includes excellent that the following claims are intended to be interpreted as It selects embodiment and falls into all change and modification of the scope of the invention.
Obviously, various changes and modifications can be made to the invention without departing from essence of the invention by those skilled in the art Mind and range.In this way, if these modifications and changes of the present invention belongs to the range of the claims in the present invention and its equivalent technologies Within, then the present invention is also intended to include these modifications and variations.

Claims (10)

1. a kind of method for reading catalogue subitem metadata, which is characterized in that the catalogue subitem metadata of the first catalogue is located at extremely In few two metadata objects, at least two metadata object is stored among at least one server, the method packet It includes:
Destination server receives the first request that client is sent, and first request reads first catalogue for requesting Catalogue subitem metadata, first request include the mark and positioning identifier of the first catalogue, and the positioning identifier is directed toward institute K-th of metadata object at least two metadata objects is stated, k is positive integer;
The destination server obtains the metadata of first catalogue, first catalogue according to the mark of first catalogue Metadata include at least two metadata object attribute information;
The destination server is according to k-th of metadata object in single delivery quantity threshold value and the attribute information And the quantity of catalogue subitem metadata that the metadata object after k-th of metadata object respectively contains, determine this Catalogue subitem metadata of the kth into (k+m) a metadata object need to be read, wherein the single delivery quantity threshold value is institute The maximum for the catalogue subitem metadata that destination server single can be sent to the client is stated, m is positive integer;
The destination server is used to determine that the kth is each to (k+m) a metadata object according in the attribute information From the information of storage location, the kth is obtained from least one described server parallel to (k+m) a metadata object packet The catalogue subitem metadata contained;
The destination server forms this according to the catalogue subitem metadata of acquisition and delivers metadata, and to the client End sends described this and delivers data, wherein this described size for delivering metadata is not more than the single delivery quantity threshold value.
2. the method as described in claim 1, which is characterized in that the positioning identifier is directed toward in k-th of metadata object I-th of catalogue subitem metadata, for showing that i-th of catalogue subitem metadata is the first catalogue subitem member that need to be read Data.
3. method according to claim 2, which is characterized in that the destination server according to single delivery quantity threshold value, and The metadata object after k-th of metadata object and k-th of metadata object in the attribute information is respectively The quantity for the catalogue subitem metadata for including determines that this need to read catalogue subitem of the kth into (k+m) a metadata object Metadata, comprising:
The destination server determines: i-th of catalogue subitem metadata is a to (k+m-1) in k-th of metadata object The total amount of the last one catalogue subitem metadata of metadata object is less than the single delivery quantity threshold value and k-th described In metadata object i-th of catalogue subitem metadata to (k+m) a metadata object the last one catalogue subitem metadata Total amount be not less than the single delivery quantity threshold value;
The destination server determines that this need to read catalogue subitem member number of the kth into (k+m) a metadata object According to.
4. method as claimed in claim 2 or claim 3, which is characterized in that i-th of catalogue described in k-th of metadata object Subitem metadata j-th of catalogue subitem metadata into (k+m) a metadata object is that described this delivers metadata.
5. the method according to claim 1, which is characterized in that the destination server is sent to the client This delivers metadata, comprising:
Described this of the destination server judgement delivers whether the last one catalogue subitem metadata in metadata is described The last one catalogue subitem metadata of first catalogue;
If it is not, then the destination server sends described this to the client and delivers metadata, and indicate the client Continue to the mesh after the last one catalogue subitem metadata of destination server request this delivery metadata at end Record subitem metadata.
6. a kind of device for reading catalogue subitem metadata, which is characterized in that the catalogue subitem metadata of the first catalogue is located at extremely In few two metadata objects, at least two metadata object is stored among at least one server, described device packet It includes:
Receiving module, for receiving the first request of client transmission, first request reads first mesh for requesting The catalogue subitem metadata of record, first request include the mark and positioning identifier of the first catalogue, and the positioning identifier refers to K-th of metadata object at least two metadata object, k are positive integer;
First obtains module, obtains the metadata of first catalogue for the mark according to first catalogue, and described first The metadata of catalogue includes the attribute information of at least two metadata object;
First determining module, for according to k-th of metadata pair in single delivery quantity threshold value and the attribute information As and k-th of metadata object after the quantity of catalogue subitem metadata that respectively contains of metadata object, determine this The secondary catalogue subitem metadata that need to read kth into (k+m) a metadata object, wherein the single delivery quantity threshold value is The maximum for the catalogue subitem metadata that described device single can be sent to the client, m is positive integer;
Second obtains module, for being used to determine the kth to (k+m) a metadata object according in the attribute information The information of respective storage location, obtains the kth to (k+m) a metadata object from least one described server parallel The catalogue subitem metadata for including;
Second determining module, for the catalogue subitem metadata according to acquisition formed this deliver metadata, it is described this The size for delivering metadata is not more than the single delivery quantity threshold value;
Sending module delivers data for sending described this to the client.
7. device as claimed in claim 6, which is characterized in that the positioning identifier is directed toward in k-th of metadata object I-th of catalogue subitem metadata, for showing that i-th of catalogue subitem metadata is the first catalogue subitem member that need to be read Data.
8. device as claimed in claim 7, which is characterized in that first determining module is specifically used for:
Determine that i-th of catalogue subitem metadata in k-th of metadata object is last to (k+m-1) a metadata object The total amount of one catalogue subitem metadata is less than in the single delivery quantity threshold value and k-th of metadata object i-th Catalogue subitem metadata to (k+m) a metadata object the last one catalogue subitem metadata total amount be not less than the list Secondary delivery quantity threshold value;
Determine that this need to read catalogue subitem metadata of the kth into (k+m) a metadata object.
9. device as claimed in claim 7 or 8, which is characterized in that i-th of catalogue described in k-th of metadata object Subitem metadata j-th of catalogue subitem metadata into (k+m) a metadata object is that described this delivers metadata.
10. such as the described in any item devices of claim 6-8, which is characterized in that described device further include:
Judgment module, for judging that described this delivers whether the last one catalogue subitem metadata in metadata is described the The last one catalogue subitem metadata of one catalogue;If it is not, then by the sending module indicate the client continue to Catalogue subitem metadata after the last one catalogue subitem metadata of described device request this delivery metadata.
CN201610070587.2A 2016-01-29 2016-01-29 A kind of method and device reading catalogue subitem metadata Active CN105677892B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201610070587.2A CN105677892B (en) 2016-01-29 2016-01-29 A kind of method and device reading catalogue subitem metadata

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201610070587.2A CN105677892B (en) 2016-01-29 2016-01-29 A kind of method and device reading catalogue subitem metadata

Publications (2)

Publication Number Publication Date
CN105677892A CN105677892A (en) 2016-06-15
CN105677892B true CN105677892B (en) 2018-12-25

Family

ID=56302882

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201610070587.2A Active CN105677892B (en) 2016-01-29 2016-01-29 A kind of method and device reading catalogue subitem metadata

Country Status (1)

Country Link
CN (1) CN105677892B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110781159B (en) * 2019-10-28 2021-02-02 柏科数据技术(深圳)股份有限公司 Ceph directory file information reading method and device, server and storage medium

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102541985A (en) * 2011-10-25 2012-07-04 曙光信息产业(北京)有限公司 Organization method of client directory cache in distributed file system
CN103916467A (en) * 2014-03-25 2014-07-09 中国科学院计算技术研究所 Load transfer method and system in metadata cluster
US8935281B1 (en) * 2005-10-31 2015-01-13 Symantec Operating Corporation Optimized content search of files
CN105094992A (en) * 2015-09-25 2015-11-25 浪潮(北京)电子信息产业有限公司 File request processing method and system

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8935281B1 (en) * 2005-10-31 2015-01-13 Symantec Operating Corporation Optimized content search of files
CN102541985A (en) * 2011-10-25 2012-07-04 曙光信息产业(北京)有限公司 Organization method of client directory cache in distributed file system
CN103916467A (en) * 2014-03-25 2014-07-09 中国科学院计算技术研究所 Load transfer method and system in metadata cluster
CN105094992A (en) * 2015-09-25 2015-11-25 浪潮(北京)电子信息产业有限公司 File request processing method and system

Also Published As

Publication number Publication date
CN105677892A (en) 2016-06-15

Similar Documents

Publication Publication Date Title
CN105324770B (en) Effectively read copy
KR102254809B1 (en) Distributed computing resources sharing system and computing apparatus thereof providing reward based on block chain
CN102523285B (en) Storage caching method of object-based distributed file system
US10901619B2 (en) Selecting pages implementing leaf nodes and internal nodes of a data set index for reuse
CN105224255B (en) A kind of storage file management method and device
CN102307206B (en) Caching system and caching method for rapidly accessing virtual machine images based on cloud storage
CN104346345B (en) The storage method and device of data
CN103607424B (en) Server connection method and server system
CN110325985A (en) The method efficiently inquired based on major key read using the atom RDMA of hash index in the memory to cache close friend
CN102480397A (en) Method and equipment for accessing internet pages
CN105683928B (en) For the method for data cache policies, server and memory devices
CN106981024A (en) A kind of trading limit calculates processing system and its processing method
CN107888666A (en) A kind of cross-region data-storage system and method for data synchronization and device
CN103491193B (en) A kind of method and apparatus of file-sharing
CN107491549A (en) A kind of data processing method and system
CN109618003A (en) A kind of servers' layout method, server and storage medium
CN106528844A (en) Data request method and apparatus, and data storage system
CN105677892B (en) A kind of method and device reading catalogue subitem metadata
CN113609167B (en) Data processing method, device and equipment based on block chain and readable storage medium
CN109062717B (en) Data caching method, data caching system, data caching disaster tolerance method, data caching disaster tolerance system and data caching system
CN104391947B (en) Magnanimity GIS data real-time processing method and system
CN110009320B (en) Resource conversion method, device, system, storage medium and computer equipment
CN105791247B (en) A kind of flow system pair service implementation method and relevant device and system
CN109241021A (en) A kind of file polling method, apparatus, equipment and computer readable storage medium
CN102833295B (en) Data manipulation method and device in distributed cache system

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant