CN105677892B - A kind of method and device reading catalogue subitem metadata - Google Patents
A kind of method and device reading catalogue subitem metadata Download PDFInfo
- Publication number
- CN105677892B CN105677892B CN201610070587.2A CN201610070587A CN105677892B CN 105677892 B CN105677892 B CN 105677892B CN 201610070587 A CN201610070587 A CN 201610070587A CN 105677892 B CN105677892 B CN 105677892B
- Authority
- CN
- China
- Prior art keywords
- metadata
- catalogue
- subitem
- metadata object
- destination server
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Abstract
A kind of method and device reading catalogue subitem metadata, for solving the problems, such as that reading catalogue subitem metadata takes a long time.This method comprises: destination server receives the first request that client is sent, the first request includes the positioning identifier of k-th of the metadata object pointed in few two metadata objects;The attribute information of at least two metadata objects is obtained according to the mark of the first catalogue;Determine that this need to read catalogue subitem metadata of the kth into (k+m) a metadata object according to single delivery quantity threshold value and attribute information;According to, for determining the information of the respective storage location of metadata object, obtaining the catalogue subitem metadata that kth to (k+m) a metadata object includes from least one server parallel in attribute information;This is formed according to the catalogue subitem metadata of acquisition and delivers metadata, and is sent this to client and delivered data.
Description
Technical field
The present invention relates to field of computer technology, in particular to a kind of method and device for reading catalogue subitem metadata.
Background technique
In distributed file system, a catalogue includes heap file or subdirectory, is referred to as the subitem of catalogue, mesh
The meta-data preservation of subitem is recorded in metadata object.If by the meta-data preservation of all subitems of catalogue in the same metadata
It, can be because the alternative of object accesses causes the concurrency of business low in object.Therefore, usually by the catalogue subitem member of catalogue
Data dispersion is stored among multiple metadata objects, several catalogue subitem metadata are preserved in each metadata object.
As shown in Figure 1, all catalogue subitem metadata of catalogue be stored according to the sequence of catalogue subitem metadata it is multiple
Among metadata object, that is, 1~s of the 1st metadata object storing directory in order1A catalogue subitem metadata, the 2nd
(the s of a metadata object storing directory in order1+ 1)~s2A catalogue subitem metadata, and so on.
Server in file system is receiving the catalogue subitem metadata for the first catalogue of reading that client is sent
When request, the catalogue subitem metadata stored in the 1st metadata object is read in order first, judge the catalogue read
Whether metadata fills up agreement buffer area, and the size of the agreement buffer area is also known as single delivery quantity threshold value, be server with
The maximum for the single referable catalogue subitem metadata that client is negotiated.If catalogue subitem member in the 1st metadata object
The catalogue subitem metadata in the 2nd metadata object is then read in the unfilled agreement buffer area of data in order, and so on,
Until filling up agreement buffer area, the catalogue subitem metadata read is sent to customer side.
But the method for above-mentioned reading catalogue subitem metadata takes a long time in the prior art, leads to distributed field system
The efficiency of system is lower.
Summary of the invention
The embodiment of the present invention provides a kind of method and device for reading catalogue subitem metadata, for solving in the prior art
Read the problem of catalogue subitem metadata takes a long time.
In a first aspect, the embodiment of the present invention provides a kind of method for reading catalogue subitem metadata, the catalogue of the first catalogue
Subitem metadata is located in multiple metadata objects, and multiple metadata objects are stored among multiple servers, the method packet
Include: destination server receives the first request that client is sent, and first request reads first catalogue for requesting
Catalogue subitem metadata, first request include the mark and positioning identifier of the first catalogue, and the positioning identifier is directed toward institute
K-th of metadata object at least two metadata objects is stated, shows to read catalogue subitem since k-th of metadata object
Metadata;The destination server obtains the metadata of first catalogue according to the mark of first catalogue, and described first
The metadata of catalogue includes the attribute information of at least two metadata object, and the attribute information includes the first catalogue of storage
Catalogue subitem metadata the catalogue subitem metadata quantity that respectively contains of metadata object, and the above-mentioned metadata pair of management
The information of the metadata management node of elephant;The destination server is according in single delivery quantity threshold value and the attribute information
K-th of metadata object and k-th of metadata object after the catalogue subitem that respectively contains of metadata object
The quantity of metadata determines that this need to read catalogue subitem metadata of the kth into (k+m) a metadata object, wherein institute
State the maximum that single delivery quantity threshold value is the catalogue subitem metadata that the destination server single can be sent to the client
Amount, m is positive integer;The destination server is used to determine the kth to (k+m) a first number according in the attribute information
According to the information of the respective storage location of object, the kth is obtained from least one described server parallel to (k+m) a first number
The catalogue subitem metadata for including according to object;The destination server delays the catalogue subitem metadata deposit agreement of acquisition
Area is rushed, this is formed and delivers metadata, and send described this to the client and deliver data, wherein this described delivery
The size of metadata is not more than the single delivery quantity threshold value.
In above-mentioned implementation, destination server is when reading the catalogue subitem metadata of the first catalogue, according to each member
The quantity and single delivery quantity threshold value for the catalogue subitem metadata that data object includes, determine this multiple member that need to be read
Data object, then, and meanwhile it is sub from the catalogue read from corresponding metadata management node in determining multiple metadata objects
Item metadata.With in the prior art using serial mode read catalogue subitem metadata by the way of compared with, can substantially reduce mesh
The read access time for recording subitem metadata, improves the efficiency of file system.
With reference to first aspect, in the first possible implementation of the first aspect, the destination server is according to list
After k-th of metadata object and k-th of metadata object in secondary delivery quantity threshold value and the attribute information
The quantity of catalogue subitem metadata that respectively contains of metadata object, determine that this need to read kth to (k+m) a metadata
Catalogue subitem metadata in object, comprising: the destination server determines: i-th of catalogue in k-th of metadata object
Subitem metadata to (k+m-1) a metadata object the last one catalogue subitem metadata total amount less than the single hand over
I-th of catalogue subitem metadata is paid in amount threshold value and k-th of metadata object to (k+m) a metadata object
The total amount of the last one catalogue subitem metadata is not less than the single delivery quantity threshold value;The destination server determines that this is needed
Read catalogue subitem metadata of the kth into (k+m) a metadata object.
With reference to first aspect or the first possible implementation of first aspect, second in first aspect are possible
In implementation, the destination server sends this to the client and delivers metadata, comprising: the destination server is sentenced
It is disconnected described this deliver the last one catalogue subitem metadata in metadata whether be first catalogue the last one mesh
Record subitem metadata;If it is not, then the destination server sends described this to the client and delivers metadata, and indicate
The client continues that described this is requested to deliver the last one catalogue subitem metadata of metadata to the destination server
Catalogue subitem metadata later, guarantee agreement client can obtain all catalogue subitem metadata of the first catalogue.
Second aspect, the embodiment of the present invention provide a kind of catalogue of the first catalogue of device for reading catalogue subitem metadata
Metadata is located at least two metadata objects, at least two metadata object be stored at least one server it
In, described device includes: receiving module, and for receiving the first request of client transmission, first request is read for requesting
The catalogue subitem metadata of first catalogue is taken, first request includes the mark and positioning identifier of the first catalogue, institute
K-th of metadata object that positioning identifier is directed toward at least two metadata object is stated, k is positive integer;First obtains mould
Block obtains the metadata of first catalogue, the metadata packet of first catalogue for the mark according to first catalogue
Attribute information containing at least two metadata object;First determining module, for according to single delivery quantity threshold value, Yi Jisuo
The metadata object after k-th of metadata object and k-th of metadata object in attribute information is stated respectively to wrap
The quantity of the catalogue subitem metadata contained determines that this need to read catalogue subitem member of the kth into (k+m) a metadata object
Data, wherein the single delivery quantity threshold value is the catalogue subitem metadata that described device single can be sent to the client
Maximum, m is positive integer;Second obtains module, for being used to determine the kth to (k according in the attribute information
+ m) the respective storage location of a metadata object information, the kth is obtained from least one described server parallel to (k
+ m) a metadata object catalogue subitem metadata that includes;Second determining module, for the catalogue subitem member according to acquisition
Data form this and deliver metadata, this described size for delivering metadata is not more than the single delivery quantity threshold value;It sends
Module delivers data for sending described this to the client.
In conjunction with second aspect, in the first possible implementation of the second aspect, first determining module is specific
For: determine in k-th of metadata object i-th of catalogue subitem metadata to (k+m-1) a metadata object most
The total amount of the latter catalogue subitem metadata is less than i-th in the single delivery quantity threshold value and k-th of metadata object
The total amount of the last one catalogue subitem metadata of a catalogue subitem metadata to (k+m) a metadata object is not less than described
Single delivery quantity threshold value;Determine that this need to read catalogue subitem metadata of the kth into (k+m) a metadata object.
In conjunction with the possible implementation of the first of second aspect or second aspect, second in second aspect is possible
In implementation, described device further include: judgment module, for judging that described this delivers the last one catalogue in metadata
Subitem metadata whether be first catalogue the last one catalogue subitem metadata;If it is not, then passing through the transmission mould
The client that block indicates continues that described this is requested to deliver the last one catalogue subitem metadata of metadata to described device
Catalogue subitem metadata later.
In some possible implementations, the positioning identifier is directed toward i-th of mesh in k-th of metadata object
Subitem metadata is recorded, for showing that i-th of catalogue subitem metadata is the first catalogue subitem metadata that need to be read.It is above-mentioned
Possible implementation can be accurately determined this catalogue subitem metadata that need to be read, convenient for more efficient and more have a needle
To property carry out the reading of catalogue subitem metadata.
In some possible implementations, i-th of catalogue subitem metadata described in k-th of metadata object is extremely
J-th of catalogue subitem metadata is that described this delivers metadata in (k+m) a metadata object.
Detailed description of the invention
To describe the technical solutions in the embodiments of the present invention more clearly, make required in being described below to embodiment
Attached drawing is briefly introduced, it should be apparent that, drawings in the following description are only some embodiments of the invention, for this
For the those of ordinary skill in field, without any creative labor, it can also be obtained according to these attached drawings
His attached drawing.
Fig. 1 is the schematic diagram of the storage mode of the catalogue subitem metadata of catalogue;
Fig. 2 is the schematic diagram of file system in the embodiment of the present invention;
Fig. 3 is the flow diagram that catalogue subitem metadata approach is read in the embodiment of the present invention;
Fig. 4 is the structural schematic block diagram that catalogue subitem metadata device is read in the embodiment of the present invention;
Fig. 5 is the structural schematic block diagram that catalogue subitem metadata equipment is read in the embodiment of the present invention.
Specific embodiment
Technical solution of the present invention is described in detail below by attached drawing and specific embodiment, it should be understood that the present invention
Specific features in embodiment and embodiment are the detailed description to technical solution of the present invention, rather than to the technology of the present invention
The restriction of scheme, in the absence of conflict, the technical characteristic in the embodiment of the present invention and embodiment can be combined with each other.
In file system, a catalogue includes heap file or subdirectory, is referred to as the subitem of catalogue, each catalogue
Subitem is corresponding with metadata, referred to as catalogue subitem metadata.Metadata object is the object for storing metadata, catalogue
Multiple catalogue subitem metadata are dispersed to be stored among multiple metadata objects.Mesh is preserved among the metadata of catalogue itself
The information of the subitem of record, wherein just including the metadata object of the metadata (that is, catalogue subitem metadata) of storing directory subitem
Attribute information, for example, the number of catalogue subitem metadata of each metadata object storage for another example manages the metadata pair
Information of metadata management module of elephant, etc..
Fig. 2 is the schematic diagram of file system in the embodiment of the present invention, wherein each server may each comprise client generation
Manage (Client Agent;Referred to as: CA) module, metadata management module and memory module, above-mentioned CA module and metadata
Management module can be realized by software entity, or corresponding hardware module (such as integrated chip), above-mentioned memory module
It can be all or part of hard disk of server.CA module in Servers-all is formed together CA cluster, in Servers-all
Metadata management module form metadata management cluster, the hard disk in Servers-all forms storage cluster.
Wherein, the CA module in server is used to handle the request of the reading catalogue subitem metadata of client transmission, real
In the situation of border, communicated by the Protocol Client end module in client with the CA module in server;Memory node is for depositing
Metadata object is stored up, and storage has the catalogue subitem metadata of catalogue in metadata object;Metadata management node is for managing
Metadata object, CA can read the catalogue subitem metadata in metadata object by metadata management node.
Fig. 3 is the flow diagram provided in an embodiment of the present invention for reading catalogue subitem metadata approach, and this method includes
Following steps:
Step 101: destination server receives the first request that client is sent, and the first request reads the first mesh for requesting
The catalogue subitem metadata of record, the first request include the mark and positioning identifier of the first catalogue, and positioning identifier is directed toward at least two
K-th of metadata object in a metadata object, k are positive integer;
Step 102: destination server obtains the metadata of the first catalogue, the member of the first catalogue according to the mark of the first catalogue
Data contain at least two the attribute information of metadata object;
Step 103: destination server is according to k-th of metadata object in single delivery quantity threshold value and attribute information
And the quantity of catalogue subitem metadata that the metadata object after k-th of metadata object respectively contains, determine that this needs to read
Take catalogue subitem metadata of the kth into (k+m) a metadata object, wherein single delivery quantity threshold value is destination server
The maximum for the catalogue subitem metadata that single can be sent to client, m is positive integer;
Step 104: destination server is used to determine that kth is each to (k+m) a metadata object according in attribute information
From the information of storage location, the catalogue subitem that kth includes to (k+m) a metadata object is obtained from above-mentioned storage location parallel
Metadata;
Step 105: catalogue subitem metadata forms this and delivers metadata destination server based on the received, and to client
End sends this and delivers data, wherein this size for delivering metadata is not more than single delivery quantity threshold value.
Specifically, the protocol client transmission of client can be received by the CA module in destination server in step 101
First request.Positioning identifier in first request reads catalogue subitem metadata for showing since which metadata object.
In step 102, after receiving the first request, destination server is first according to the first mesh for including in the first request
The mark of record determines the metadata of the first catalogue.In actual conditions, the CA module of destination server may be cached with catalogue member number
According to destination server can first search for the metadata of the first catalogue in the caching, can be according to the first catalogue if miss
Mark location goes out the metadata of the parent directory of first catalogue, includes the metadata of the first catalogue in the metadata of parent directory
Routing information, destination server can determine the metadata of the first catalogue according to the routing information.
It include the information of the subitem of the first catalogue in the metadata of first catalogue, including the son of the first catalogue of storage
The attribute information of the metadata object of the metadata (that is, catalogue subitem metadata of the first catalogue) of item, attribute information includes should
Which catalogue subitem metadata of the first catalogue and the metadata pipe of metadata object ownership are housed in metadata object
Manage the information of module.
Then, after destination server knows k-th of metadata object and k-th of metadata according to the attribute information
The metadata number that each metadata includes.
Then, in step 103, the quantity for the catalogue subitem metadata that destination server includes according to above-mentioned metadata object
Information and single delivery quantity threshold value, determine which metadata object this need to read.Its specific implementation includes:
First, destination server since k-th of metadata object, carries out the cumulative of catalogue subitem metadata, until mesh
The total amount of record subitem metadata reaches the single delivery quantity threshold value.
For example, a kind of feasible algorithm are as follows: calculating k-th of metadata object to (k+m-1) a metadata object
Including catalogue subitem metadata total amount be not more than single delivery quantity threshold value, and calculate k-th of metadata object to (k+
M) when the catalogue subitem metadata total amount that a metadata object includes is greater than single delivery quantity threshold value, it can determine that this needs to read
The catalogue subitem metadata that k-th of metadata object includes into (k+m) a metadata object.
In another example another feasible algorithm are as follows: calculating k-th of metadata object to (k+m) a metadata pair
As the catalogue subitem metadata total amount that includes is no more than single delivery quantity threshold value, and k-th of metadata object is calculated to the
(k+m+1) when the catalogue subitem metadata total amount that a metadata object includes is greater than single delivery quantity threshold value, this can be determined
The catalogue subitem metadata that k-th of metadata object includes into (k+m) a metadata object need to be read.
Wherein, the first above-mentioned algorithm can guarantee that the catalogue subitem metadata read every time fills up protocol cache area, increase
Add the number of the catalogue subitem metadata returned every time to client;And above-mentioned second algorithm can be avoided reading this can not
The catalogue subitem metadata of delivery saves transfer resource.
Second, positioning identifier is directed toward i-th of catalogue subitem metadata in k-th of metadata object, for showing i-th
Catalogue subitem metadata is the first catalogue subitem metadata that need to be read.
Destination server i-th of catalogue subitem metadata, carries out catalogue subitem member number since k-th of metadata object
According to it is cumulative, calculate i-th of catalogue subitem metadata jth into (k+m) a metadata object in k-th of metadata object
The total amount of a catalogue subitem metadata is not more than single delivery quantity threshold value, and i-th of catalogue subitem in k-th of metadata object
The total amount of metadata (j+1) a catalogue subitem metadata into (k+m) a metadata object is greater than single delivery quantity threshold value,
Then determine i-th of catalogue subitem metadata, j-th of catalogue into (k+m) a metadata object in k-th of metadata object
Item metadata is that this need to read catalogue subitem metadata.
Above-mentioned second of implementation can be accurately determined this catalogue subitem metadata that need to be read, convenient for more efficient
And more targetedly carry out the reading of catalogue subitem metadata.
In actual conditions, when the size for the catalogue subitem metadata for including to metadata object carries out above-mentioned cumulative, i.e.,
The information of the quantity for the catalogue subitem metadata for needing metadata object to include, it is also desirable to obtain each catalogue subitem metadata
The information of size.The size information of catalogue subitem metadata may be embodied in above-mentioned attribute information in metadata object, can also
To obtain the average value of catalogue subitem metadata size by destination server, carried out with the average value above-mentioned cumulative.Catalogue subitem
The size average value of metadata can be default value;Alternatively, can be estimated according to the size for the catalogue subitem metadata delivered before
Calculate the average value of catalogue subitem metadata size.
In addition, when it is implemented, deposit in the size of the catalogue subitem metadata in metadata object, and agreement is consigned to
Catalogue subitem metadata size when client may be different, and usual the latter is less than the former, this is because would generally be using compression
Algorithm will consign to agreement customer side after catalogue subitem metadata compression.Therefore, according to the size of catalogue subitem metadata
When accumulated value determines the metadata object that this need to read, by above-mentioned accumulated value and this delivery quantity threshold value multiplied by deviation factor into
Row compares, which can be the ratio of the data volume before and after catalogue subitem metadata compression.
Then, in step 104, destination server includes for determining kth to (k+m) a member according to above-mentioned attribute information
The information of the respective storage location of data object, obtains kth to (k+m) a metadata object from least one server parallel
The catalogue subitem metadata for including.
Specifically, it is described for determine kth to (k+m) a metadata object respectively storage location information be kth
The information of the metadata management module respectively belonged to (k+m) a metadata object, also that is, kth is to (k+m) a metadata
Object is respectively managed by the metadata management module of which server.According to the information, that is, it can determine that kth is a to (k+m)
Which server is each freedom of metadata object managed, and manages the metadata management module in the server of metadata object and deposit
Contain the information of the storage location of the metadata object.
In one possible implementation, destination server determines management kth to (k+m) a metadata object
N number of server is requested to N number of server transmitted in parallel, wherein includes in the request sent to s-th of server (k+t)
The mark of a metadata object, (k+t) a metadata object are that kth is serviced into (k+m) a metadata object by s-th
The metadata object that device is managed, the integer that the value of s is 1 to N, the integer that the value of t is 0 to m.
Then, the position of (k+t) metadata object that s-th of server is stored according to the metadata management module of itself
Information reads the catalogue subitem metadata in (k+t) metadata object.It then, will be in (k+t) metadata object of reading
Catalogue subitem metadata return to destination server.
It, will after s-th of server receives the request of destination server transmission in alternatively possible implementation
The location information of (k+t) metadata object of the metadata management module storage of itself returns to destination server, target clothes
After business device receives the location information, voluntarily from the catalogue read from corresponding storage location in (k+t) metadata object
Subitem metadata.
Optionally, in above-mentioned first way, destination server is in the service for being sent to k-th of metadata object of management
In the request of device, beginning flag may include, which is directed toward i-th of catalogue subitem member in k-th of metadata object
Data show to read from i-th of catalogue subitem metadata.The server for receiving request returns to k-th yuan to destination server
I-th of catalogue subitem metadata in data object and its catalogue subitem metadata later.
Optionally, in above-mentioned first way, destination server is being sent to management (k+m) a metadata object
In the request of server, end mark may include, which is directed toward j-th of mesh in (k+m) a metadata object
Record subitem metadata, show in (k+m) a metadata object j-th of catalogue subitem metadata be this last one need to read
Catalogue subitem metadata.Receive j-th of the server of request into destination server return (k+m) a metadata object
Catalogue subitem metadata and its catalogue subitem metadata before.
It should be noted that storing the memory module of above-mentioned metadata object and managing the metadata of above-mentioned metadata object
Management module can on the same server, can also be on different server.Storage metadata object memory module with
Manage metadata object metadata management module not on the same server when, manage the metadata management mould of metadata object
Server where block sends to the server where the memory module of storage metadata object and obtains in the metadata object
The catalogue subitem metadata of acquisition is sent to mesh after obtaining catalogue subitem metadata by the request of catalogue subitem metadata
Mark server.
In addition, the metadata management module of destination server may also manage kth into (k+m) a metadata object
Metadata object, destination server can pass through the metadata management module of itself and obtain catalogue in corresponding metadata object
Subitem metadata.For example, destination server determines (k+t) a metadata object by itself first number according to above-mentioned attribute information
Managed according to management module and remaining metadata object by the server except destination server metadata management module institute
Management, the then storage location of (k+t) a metadata object that destination server is stored by the metadata management module of itself
Information, obtain the catalogue subitem metadata in (k+t) a metadata object from the storage location, and take from the target
The catalogue subitem metadata in remaining metadata object is obtained at server except business device.
In step 105, the catalogue subitem metadata of acquisition is stored in agreement buffer area by destination server in order, until filling out
Completely laundry buffer area.
In one possible implementation, the size for the catalogue subitem metadata that destination server obtains is equal or close to
Size (that is, single delivery quantity threshold value) in agreement buffer area.
In alternatively possible implementation, the size for the catalogue subitem metadata that destination server obtains is more than agreement
The size of buffer area, catalogue subitem metadata is stored in agreement buffer area by destination server in order, after being filled with, remaining mesh
Recording subitem metadata can be directly to give up there are two types of processing mode, one, secondly to be buffered in the sheet of destination server
Among ground caching, so that when client transmission is requested next time, it is stored in agreement buffer area from local cache.
Catalogue subitem member number after the catalogue subitem metadata that will be obtained is stored in agreement buffer area, in agreement buffer area
Metadata is delivered according to this is formed, destination server is returned to client.
In the embodiment of the present invention, 102~step 104 of above-mentioned steps can be by the central processing unit of destination server
(Central Processing Unit;It referred to as: CPU) executes, can also be executed by the CA module of destination server.
In above-mentioned technical proposal, destination server reads the request of the catalogue subitem metadata of the first catalogue receiving
Afterwards, the size and single delivery quantity threshold value for the catalogue subitem metadata for including according to each metadata object, determine this
Then the multiple metadata objects that need to be read obtain metadata pair from the server for storing multiple metadata object parallel
The catalogue subitem metadata of elephant.With in the prior art using serial mode read catalogue subitem metadata by the way of compared with, can
The read access time for substantially reducing catalogue subitem metadata, improves the efficiency of file system.
Optionally, in the embodiment of the present invention, it includes: i-th of catalogue from k-th of metadata object that this, which delivers metadata,
The catalogue subitem metadata of subitem metadata j-th of catalogue subitem metadata into (k+m) a metadata object.
Optionally, in the embodiment of the present invention, step 105: destination server sends this to client and delivers metadata, packet
Include following steps:
Whether the last one catalogue subitem metadata that destination server judges that this is delivered in metadata is the first catalogue
The last one catalogue subitem metadata;
If it is not, then destination server sends this to client and delivers metadata, and indicate that client continues to target
Server requests this to deliver the catalogue subitem metadata after the last one catalogue subitem metadata of metadata.
Specifically, destination server after having filled agreement buffer area, judges the last one mesh in agreement buffer area
Record subitem metadata whether be the first directory entry the last one catalogue subitem metadata, if it is not, then showing the first catalogue
Catalogue subitem metadata, which is not yet read, to be finished, and destination server will refer to when sending catalogue subitem metadata to protocol client
Show the catalogue subitem member after client continues the last one the catalogue subitem metadata for requesting this to deliver to destination server
Data.Client will send to destination server request again according to instruction, and the positioning identifier in request is directed toward this delivery
First catalogue subitem metadata after the last one catalogue subitem metadata.
If the last one catalogue subitem metadata that aforementioned judging result shows that this is delivered in metadata is the first mesh
The last one catalogue subitem metadata of record, then destination server can indicate the catalogue subitem of the first catalogue to protocol client
Metadata, which has been read, to be finished.Alternatively, destination server, without above-mentioned instruction, client will not receive aforementioned instruction agreement visitor
When family end continues to request the instruction information of catalogue subitem metadata to destination server, confirmation, which has been read, to be finished.
Optionally, in the embodiment of the present invention, destination server is before sending this and delivering metadata, first to catalogue subitem
Metadata is ranked up.
It is explained below in method of the specific application example to above-mentioned reading catalogue subitem metadata.
The Protocol Client end module of client is in the catalogue subitem metadata for needing to read the first catalogue, to destination service
The CA module of device sends the first request, the catalogue subitem of the mark comprising the first catalogue and direction the first catalogue of storage in request
The positioning identifier of 1st catalogue subitem metadata of the 1st metadata object in metadata.
CA module determines the metadata of the first catalogue according to the mark of the first catalogue in the first request, the first catalogue
Metadata includes following information: the number for the catalogue subitem metadata that each metadata object of the first catalogue of storage respectively contains
Mesh.CA module according to above- mentioned information and single delivery quantity threshold value determine this at most can to protocol client deliver the 1st to
The catalogue subitem metadata that 5th metadata object is included.
Then, the metadata management section for each metadata object of management for including in metadata of the CA module according to the first catalogue
The information of point, determines that the 1st to the 2nd metadata object is managed by metadata management node A, the 3rd to the 5th metadata object
It is managed respectively by metadata management node B, metadata management node C, metadata management node D.Wherein, the metadata pipe
Reason node A, B, C, D are located at server A, B, C, D.
Then, CA module sends request to metadata management node A to D simultaneously, and requesting node A returns to the 1st to the 2nd member
The catalogue subitem metadata that data object is included, requesting node B return to the catalogue subitem member that the 3rd metadata object is included
Data, requesting node C return to the catalogue subitem metadata that the 4th metadata object is included, and requesting node D returns to the 5th member
(whole or preceding j) catalogue subitem metadata that data object is included.
Then, the catalogue subitem metadata that CA receiving node A to D is respectively returned, is stored in data buffer zone, i.e., in sequence
It is first stored in catalogue subitem metadata in the 1st metadata object in order, then is stored in the mesh in the 2nd metadata object in order
Subitem metadata is recorded, and so on, until j-th of catalogue subitem metadata of the 5th metadata object of deposit, since data are slow
It rushes area to fill up, the catalogue subitem metadata that can not be restored again into after j-th of catalogue subitem metadata of the 5th metadata object.
Then the 1st catalogue subitem metadata in the 1st metadata object to the 5th metadata object j-th of catalogue subitem member number
Data are delivered according to this is formed.
Then, CA judge the 5th metadata object j-th of catalogue subitem metadata whether be the first catalogue last
A catalogue subitem metadata, then after determining it not and being the last one catalogue subitem metadata, CA sends this to protocol client
The mark of j-th of catalogue subitem metadata of the 5th metadata object of data and the first catalogue is delivered, the mark is for referring to
Show that protocol client continues remaining catalogue subitem metadata (j-th of i.e. the 5th metadata object that the first catalogue is requested to CA
The mesh that the metadata object after catalogue subitem metadata and the 5th metadata object after catalogue subitem metadata includes
Record subitem metadata).
Protocol Client end module receives the catalogue subitem metadata returned, and is indicated according to CA, continues to send to CA and read
The request of first catalogue subitem metadata, the positioning identifier in the request are directed toward+1 catalogue of jth in the 5th metadata object
Subitem metadata.
The request of CA response protocol client continues to read what this can be delivered to multiple metadata management nodal parallels
Catalogue subitem metadata, until the last one catalogue subitem metadata of the first catalogue is consigned to protocol client.
Based on the same technical idea, the embodiment of the invention also provides a kind of devices for reading catalogue subitem metadata
200, it is the structural schematic block diagram of device 200 referring to fig. 4, device 200 includes:
Receiving module 201, for receiving the first request of client transmission, the first request reads the first catalogue for requesting
Catalogue subitem metadata, first request include the first catalogue mark and positioning identifier, positioning identifier be directed toward at least two
K-th of metadata object in metadata object, k are positive integer;
First obtains module 202, and the metadata of the first catalogue is obtained for the mark according to the first catalogue, the first catalogue
Metadata contains at least two the attribute information of metadata object;
First determining module 203, for according to k-th of metadata pair in single delivery quantity threshold value and attribute information
As and k-th of metadata object after the quantity of catalogue subitem metadata that respectively contains of metadata object, determine that this is needed
Read catalogue subitem metadata of the kth into (k+m) a metadata object, wherein single delivery quantity threshold value is device single
The maximum for the catalogue subitem metadata that can be sent to client, m is positive integer;
Second obtains module 204, for being used to determine kth to (k+m) a metadata object according in attribute information
The information of respective storage location, obtains the mesh that kth includes to (k+m) a metadata object from least one server parallel
Record subitem metadata;
Second determining module 205 forms this for the catalogue subitem metadata according to acquisition and delivers metadata, this friendship
The size for paying metadata is not more than single delivery quantity threshold value;
Sending module 206 delivers data for sending this to client.
Optionally, in the embodiment of the present invention, positioning identifier is directed toward i-th of catalogue subitem member in k-th of metadata object
Data, for showing that i-th of catalogue subitem metadata is the first catalogue subitem metadata that need to be read.
Optionally, in the embodiment of the present invention, the first determining module 203 is specifically used for:
Determine that i-th of catalogue subitem metadata in k-th of metadata object is last to (k+m-1) a metadata object
The total amount of one catalogue subitem metadata is less than i-th of catalogue subitem in single delivery quantity threshold value and k-th of metadata object
Metadata to (k+m) a metadata object the last one catalogue subitem metadata total amount be not less than single delivery quantity threshold
Value;
Determine that this need to read catalogue subitem metadata of the kth into (k+m) a metadata object.
Optionally, in the embodiment of the present invention, i-th of catalogue subitem metadata is a to (k+m) in k-th of metadata object
J-th of catalogue subitem metadata is that this delivers metadata in metadata object.
Optionally, in the embodiment of the present invention, with continued reference to Fig. 4, device further include:
Judgment module 207, for judging that this delivers whether the last one catalogue subitem metadata in metadata is the
The last one catalogue subitem metadata of one catalogue;If it is not, then indicating that client continues to ask to device by sending module 206
This is asked to deliver the catalogue subitem metadata after the last one catalogue subitem metadata of metadata.
Device 200 in the present embodiment and the aforementioned method for reading catalogue subitem metadata be based on the same inventive concept under
Two aspect, the implementation process of method is described in detail in front, so those skilled in the art can basis
The structure and implementation process of the device 200 in the present embodiment is well understood in foregoing description, in order to illustrate the succinct of book, herein
It repeats no more.
Based on the same technical idea, the embodiment of the invention also provides a kind of equipment for reading catalogue subitem metadata
300, it is the structural schematic block diagram of equipment 300 referring to Fig. 5, equipment 300 includes:
Transceiver 301, for receiving the first request of client transmission, the first request reads the first catalogue for requesting
Catalogue subitem metadata, the first request include the mark and positioning identifier of the first catalogue, and positioning identifier is directed toward at least two yuan
K-th of metadata object in data object, k are positive integer;
Memory 302, for storing instruction;
Processor 303 is connected with transceiver 301 and memory 302 respectively, for executing the instruction in memory, with
Following steps are executed when executing instruction:
The metadata of the first catalogue is obtained according to the mark of the first catalogue, the metadata of the first catalogue contains at least two member
The attribute information of data object;
According in single delivery quantity threshold value and attribute information k-th of metadata object and k-th metadata object it
The quantity for the catalogue subitem metadata that metadata object afterwards respectively contains determines that this need to read kth to (k+m) a first number
According to the catalogue subitem metadata in object, wherein single delivery quantity threshold value is the catalogue that server single can be sent to client
The maximum of subitem metadata, m are positive integer;
According in attribute information for determining the information of kth to the respective storage location of (k+m) a metadata object, and
Row obtains the catalogue subitem metadata that kth to (k+m) a metadata object includes from least one server;
This is formed according to the catalogue subitem metadata of acquisition and delivers metadata, and indicates that transceiver 301 is sent out to client
This is given to deliver data, wherein this size for delivering metadata is not more than single delivery quantity threshold value.
Optionally, in the embodiment of the present invention, positioning identifier is directed toward i-th of catalogue subitem member in k-th of metadata object
Data, for showing that i-th of catalogue subitem metadata is the first catalogue subitem metadata that need to be read.
Optionally, in the embodiment of the present invention, processor 303 is used for: according in single delivery quantity threshold value and attribute information
K-th of metadata object and k-th of metadata object after the metadata object catalogue subitem metadata that respectively contains
Quantity determines that this need to read catalogue subitem metadata of the kth into (k+m) a metadata object, comprising:
Determine that i-th of catalogue subitem metadata in k-th of metadata object is last to (k+m-1) a metadata object
The total amount of one catalogue subitem metadata is less than i-th of catalogue subitem in single delivery quantity threshold value and k-th of metadata object
Metadata to (k+m) a metadata object the last one catalogue subitem metadata total amount be not less than single delivery quantity threshold
Value;
Determine that this need to read catalogue subitem metadata of the kth into (k+m) a metadata object.
Optionally, in the embodiment of the present invention, i-th of catalogue subitem metadata is a to (k+m) in k-th of metadata object
J-th of catalogue subitem metadata is that this delivers metadata in metadata object.
Optionally, in the embodiment of the present invention, processor 303 is used for: instruction transceiver sends this to client and delivers member
Data, comprising:
Judge this deliver metadata in the last one catalogue subitem metadata whether be the first catalogue the last one
Catalogue subitem metadata;
If it is not, then indicating that transceiver 301 sends this to client and delivers metadata, and pass through transceiver 301 to finger
Show that client continues the catalogue subitem member after the last one catalogue subitem metadata of this delivery metadata of device request
Data.
It should be noted that processor 303 can be a processing element, it is also possible to the general designation of multiple processing elements.
For example, processor 303 can be central processing unit, it is also possible to specific integrated circuit (Application Specific
Intergrated Circuit;Referred to as: ASIC), or be arranged to implement the embodiment of the present invention one or more it is integrated
Circuit, such as: one or more microprocessors (digital singnal processor;Referred to as: DSP), or, one or
Multiple field programmable gate array (Field Programmable Gate Array;Referred to as: FPGA).
Memory can be a memory element, be also possible to the general designation of multiple memory elements, and executable for storing
Parameter, data required for program code, resident's access network equipment or terminal operating etc..And memory may include random access memory
(Random-Access Memory;It referred to as: RAM), also may include nonvolatile memory (non-volatile memory;
Referred to as: NVM), such as magnetic disk storage, flash memory (Flash) etc..
Equipment 300 in the present embodiment and the aforementioned method for reading catalogue subitem metadata be based on the same inventive concept under
Two aspect, the implementation process of method is described in detail in front, so those skilled in the art can basis
The structure and implementation process of the equipment 300 in the present embodiment is well understood in foregoing description, in order to illustrate the succinct of book, herein
It repeats no more.
The one or more technical solutions provided in the embodiment of the present invention, have at least the following technical effects or advantages:
Destination server is after the request for receiving the catalogue subitem metadata for reading the first catalogue, according to each metadata
The size and single delivery quantity threshold value for the catalogue subitem metadata that object includes, determine this multiple metadata that need to be read
Then object obtains the catalogue subitem metadata of metadata object parallel from the server for storing multiple metadata object.
With in the prior art using serial mode read catalogue subitem metadata by the way of compared with, can substantially reduce catalogue subitem member number
According to read access time, improve the efficiency of file system.
It should be understood by those skilled in the art that, the embodiment of the present invention can provide as method, system or computer program
Product.Therefore, complete hardware embodiment, complete software embodiment or reality combining software and hardware aspects can be used in the present invention
Apply the form of example.Moreover, it wherein includes the computer of computer usable program code that the present invention, which can be used in one or more,
The computer program implemented in usable storage medium (including but not limited to magnetic disk storage, CD-ROM, optical memory etc.) produces
The form of product.
The present invention be referring to according to the method for the embodiment of the present invention, the process of equipment (system) and computer program product
Figure and/or block diagram describe.It should be understood that every one stream in flowchart and/or the block diagram can be realized by computer program instructions
The combination of process and/or box in journey and/or box and flowchart and/or the block diagram.It can provide these computer programs
Instruct the processor of general purpose computer, special purpose computer, Embedded Processor or other programmable data processing devices to produce
A raw machine, so that being generated by the instruction that computer or the processor of other programmable data processing devices execute for real
The device for the function of being specified in present one or more flows of the flowchart and/or one or more blocks of the block diagram.
These computer program instructions, which may also be stored in, is able to guide computer or other programmable data processing devices with spy
Determine in the computer-readable memory that mode works, so that it includes referring to that instruction stored in the computer readable memory, which generates,
Enable the manufacture of device, the command device realize in one box of one or more flows of the flowchart and/or block diagram or
The function of being specified in multiple boxes.
Although preferred embodiments of the present invention have been described, it is created once a person skilled in the art knows basic
Property concept, then additional changes and modifications may be made to these embodiments.So it includes excellent that the following claims are intended to be interpreted as
It selects embodiment and falls into all change and modification of the scope of the invention.
Obviously, various changes and modifications can be made to the invention without departing from essence of the invention by those skilled in the art
Mind and range.In this way, if these modifications and changes of the present invention belongs to the range of the claims in the present invention and its equivalent technologies
Within, then the present invention is also intended to include these modifications and variations.
Claims (10)
1. a kind of method for reading catalogue subitem metadata, which is characterized in that the catalogue subitem metadata of the first catalogue is located at extremely
In few two metadata objects, at least two metadata object is stored among at least one server, the method packet
It includes:
Destination server receives the first request that client is sent, and first request reads first catalogue for requesting
Catalogue subitem metadata, first request include the mark and positioning identifier of the first catalogue, and the positioning identifier is directed toward institute
K-th of metadata object at least two metadata objects is stated, k is positive integer;
The destination server obtains the metadata of first catalogue, first catalogue according to the mark of first catalogue
Metadata include at least two metadata object attribute information;
The destination server is according to k-th of metadata object in single delivery quantity threshold value and the attribute information
And the quantity of catalogue subitem metadata that the metadata object after k-th of metadata object respectively contains, determine this
Catalogue subitem metadata of the kth into (k+m) a metadata object need to be read, wherein the single delivery quantity threshold value is institute
The maximum for the catalogue subitem metadata that destination server single can be sent to the client is stated, m is positive integer;
The destination server is used to determine that the kth is each to (k+m) a metadata object according in the attribute information
From the information of storage location, the kth is obtained from least one described server parallel to (k+m) a metadata object packet
The catalogue subitem metadata contained;
The destination server forms this according to the catalogue subitem metadata of acquisition and delivers metadata, and to the client
End sends described this and delivers data, wherein this described size for delivering metadata is not more than the single delivery quantity threshold value.
2. the method as described in claim 1, which is characterized in that the positioning identifier is directed toward in k-th of metadata object
I-th of catalogue subitem metadata, for showing that i-th of catalogue subitem metadata is the first catalogue subitem member that need to be read
Data.
3. method according to claim 2, which is characterized in that the destination server according to single delivery quantity threshold value, and
The metadata object after k-th of metadata object and k-th of metadata object in the attribute information is respectively
The quantity for the catalogue subitem metadata for including determines that this need to read catalogue subitem of the kth into (k+m) a metadata object
Metadata, comprising:
The destination server determines: i-th of catalogue subitem metadata is a to (k+m-1) in k-th of metadata object
The total amount of the last one catalogue subitem metadata of metadata object is less than the single delivery quantity threshold value and k-th described
In metadata object i-th of catalogue subitem metadata to (k+m) a metadata object the last one catalogue subitem metadata
Total amount be not less than the single delivery quantity threshold value;
The destination server determines that this need to read catalogue subitem member number of the kth into (k+m) a metadata object
According to.
4. method as claimed in claim 2 or claim 3, which is characterized in that i-th of catalogue described in k-th of metadata object
Subitem metadata j-th of catalogue subitem metadata into (k+m) a metadata object is that described this delivers metadata.
5. the method according to claim 1, which is characterized in that the destination server is sent to the client
This delivers metadata, comprising:
Described this of the destination server judgement delivers whether the last one catalogue subitem metadata in metadata is described
The last one catalogue subitem metadata of first catalogue;
If it is not, then the destination server sends described this to the client and delivers metadata, and indicate the client
Continue to the mesh after the last one catalogue subitem metadata of destination server request this delivery metadata at end
Record subitem metadata.
6. a kind of device for reading catalogue subitem metadata, which is characterized in that the catalogue subitem metadata of the first catalogue is located at extremely
In few two metadata objects, at least two metadata object is stored among at least one server, described device packet
It includes:
Receiving module, for receiving the first request of client transmission, first request reads first mesh for requesting
The catalogue subitem metadata of record, first request include the mark and positioning identifier of the first catalogue, and the positioning identifier refers to
K-th of metadata object at least two metadata object, k are positive integer;
First obtains module, obtains the metadata of first catalogue for the mark according to first catalogue, and described first
The metadata of catalogue includes the attribute information of at least two metadata object;
First determining module, for according to k-th of metadata pair in single delivery quantity threshold value and the attribute information
As and k-th of metadata object after the quantity of catalogue subitem metadata that respectively contains of metadata object, determine this
The secondary catalogue subitem metadata that need to read kth into (k+m) a metadata object, wherein the single delivery quantity threshold value is
The maximum for the catalogue subitem metadata that described device single can be sent to the client, m is positive integer;
Second obtains module, for being used to determine the kth to (k+m) a metadata object according in the attribute information
The information of respective storage location, obtains the kth to (k+m) a metadata object from least one described server parallel
The catalogue subitem metadata for including;
Second determining module, for the catalogue subitem metadata according to acquisition formed this deliver metadata, it is described this
The size for delivering metadata is not more than the single delivery quantity threshold value;
Sending module delivers data for sending described this to the client.
7. device as claimed in claim 6, which is characterized in that the positioning identifier is directed toward in k-th of metadata object
I-th of catalogue subitem metadata, for showing that i-th of catalogue subitem metadata is the first catalogue subitem member that need to be read
Data.
8. device as claimed in claim 7, which is characterized in that first determining module is specifically used for:
Determine that i-th of catalogue subitem metadata in k-th of metadata object is last to (k+m-1) a metadata object
The total amount of one catalogue subitem metadata is less than in the single delivery quantity threshold value and k-th of metadata object i-th
Catalogue subitem metadata to (k+m) a metadata object the last one catalogue subitem metadata total amount be not less than the list
Secondary delivery quantity threshold value;
Determine that this need to read catalogue subitem metadata of the kth into (k+m) a metadata object.
9. device as claimed in claim 7 or 8, which is characterized in that i-th of catalogue described in k-th of metadata object
Subitem metadata j-th of catalogue subitem metadata into (k+m) a metadata object is that described this delivers metadata.
10. such as the described in any item devices of claim 6-8, which is characterized in that described device further include:
Judgment module, for judging that described this delivers whether the last one catalogue subitem metadata in metadata is described the
The last one catalogue subitem metadata of one catalogue;If it is not, then by the sending module indicate the client continue to
Catalogue subitem metadata after the last one catalogue subitem metadata of described device request this delivery metadata.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201610070587.2A CN105677892B (en) | 2016-01-29 | 2016-01-29 | A kind of method and device reading catalogue subitem metadata |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201610070587.2A CN105677892B (en) | 2016-01-29 | 2016-01-29 | A kind of method and device reading catalogue subitem metadata |
Publications (2)
Publication Number | Publication Date |
---|---|
CN105677892A CN105677892A (en) | 2016-06-15 |
CN105677892B true CN105677892B (en) | 2018-12-25 |
Family
ID=56302882
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201610070587.2A Active CN105677892B (en) | 2016-01-29 | 2016-01-29 | A kind of method and device reading catalogue subitem metadata |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN105677892B (en) |
Families Citing this family (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN110781159B (en) * | 2019-10-28 | 2021-02-02 | 柏科数据技术(深圳)股份有限公司 | Ceph directory file information reading method and device, server and storage medium |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN102541985A (en) * | 2011-10-25 | 2012-07-04 | 曙光信息产业(北京)有限公司 | Organization method of client directory cache in distributed file system |
CN103916467A (en) * | 2014-03-25 | 2014-07-09 | 中国科学院计算技术研究所 | Load transfer method and system in metadata cluster |
US8935281B1 (en) * | 2005-10-31 | 2015-01-13 | Symantec Operating Corporation | Optimized content search of files |
CN105094992A (en) * | 2015-09-25 | 2015-11-25 | 浪潮(北京)电子信息产业有限公司 | File request processing method and system |
-
2016
- 2016-01-29 CN CN201610070587.2A patent/CN105677892B/en active Active
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US8935281B1 (en) * | 2005-10-31 | 2015-01-13 | Symantec Operating Corporation | Optimized content search of files |
CN102541985A (en) * | 2011-10-25 | 2012-07-04 | 曙光信息产业(北京)有限公司 | Organization method of client directory cache in distributed file system |
CN103916467A (en) * | 2014-03-25 | 2014-07-09 | 中国科学院计算技术研究所 | Load transfer method and system in metadata cluster |
CN105094992A (en) * | 2015-09-25 | 2015-11-25 | 浪潮(北京)电子信息产业有限公司 | File request processing method and system |
Also Published As
Publication number | Publication date |
---|---|
CN105677892A (en) | 2016-06-15 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN105324770B (en) | Effectively read copy | |
KR102254809B1 (en) | Distributed computing resources sharing system and computing apparatus thereof providing reward based on block chain | |
CN102523285B (en) | Storage caching method of object-based distributed file system | |
US10901619B2 (en) | Selecting pages implementing leaf nodes and internal nodes of a data set index for reuse | |
CN105224255B (en) | A kind of storage file management method and device | |
CN102307206B (en) | Caching system and caching method for rapidly accessing virtual machine images based on cloud storage | |
CN104346345B (en) | The storage method and device of data | |
CN103607424B (en) | Server connection method and server system | |
CN110325985A (en) | The method efficiently inquired based on major key read using the atom RDMA of hash index in the memory to cache close friend | |
CN102480397A (en) | Method and equipment for accessing internet pages | |
CN105683928B (en) | For the method for data cache policies, server and memory devices | |
CN106981024A (en) | A kind of trading limit calculates processing system and its processing method | |
CN107888666A (en) | A kind of cross-region data-storage system and method for data synchronization and device | |
CN103491193B (en) | A kind of method and apparatus of file-sharing | |
CN107491549A (en) | A kind of data processing method and system | |
CN109618003A (en) | A kind of servers' layout method, server and storage medium | |
CN106528844A (en) | Data request method and apparatus, and data storage system | |
CN105677892B (en) | A kind of method and device reading catalogue subitem metadata | |
CN113609167B (en) | Data processing method, device and equipment based on block chain and readable storage medium | |
CN109062717B (en) | Data caching method, data caching system, data caching disaster tolerance method, data caching disaster tolerance system and data caching system | |
CN104391947B (en) | Magnanimity GIS data real-time processing method and system | |
CN110009320B (en) | Resource conversion method, device, system, storage medium and computer equipment | |
CN105791247B (en) | A kind of flow system pair service implementation method and relevant device and system | |
CN109241021A (en) | A kind of file polling method, apparatus, equipment and computer readable storage medium | |
CN102833295B (en) | Data manipulation method and device in distributed cache system |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
C06 | Publication | ||
PB01 | Publication | ||
C10 | Entry into substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |