CN109684282A - A kind of method and device constructing metadata cache - Google Patents

Abstract

A kind of method and device constructing metadata cache, it include: that MDS receives metadata request, if there is no the cache node of the first catalogue and itself being ownership MDS in tree-like buffer structure, the cache node of the first catalogue can be created, the first meta data file stored in OSD is read according to the mark of the first catalogue, by the content of the first meta data file and is saved in the corresponding cache node of the first catalogue.Since the content of the first meta data file includes the metadata information of the subitem of the first catalogue, pass through above-mentioned processing, without from the meta data file for being successively read each catalogue on subtree root to the path of the first catalogue in OSD, the buffer structure for meeting metadata processing requirement can be constructed, to effectively reduce the number from OSD disk-read, shorten the processing time of metadata request, improves metadata treatment effeciency.

Description

A kind of method and device constructing metadata cache

Technical field

The present invention relates to field of computer technology, in particular to a kind of method and device for constructing metadata cache.

Background technique

In distributed file system, meta data server (metadata server, MDS) generallys use dynamic subtree point The mode in area manages metadata cache, as shown in Figure 1, the directory tree of entire user is divided into multiple dynamic subtree subregions, each MDS is responsible for managing the metadata cache of part of dynamic subtree, handles the metadata operation under dynamic subtree, and safeguard first number According to the consistency of caching and the meta data file stored in object storage device (object storage devices, OSD).

Since MDS manages metadata cache using tree-like buffer structure, when MDS is to the metadata of a certain file or catalogue When information is handled, MDS need to construct the subtree root of the dynamic subtree where this document or catalogue to this document or mesh The buffer structure of all catalogues on the path of record.

In the prior art, when MDS receives the metadata request handled the metadata of a certain file or catalogue, but There is no when the metadata cache of this document or the parent directory of catalogue in the MDS, discovery (discover) process can be triggered, first The subtree root of the dynamic subtree where this document or catalogue is inquired to current file or the path of catalogue, then by subtree root Catalogue starts successively to be searched (lookup) operation downwards.In lookup operation each time, in the metadata cache of parent directory The middle metadata information for searching subdirectory exists if the metadata information of subdirectory is not present according to the meta data file of parent directory Storage location in OSD, reads the content of the meta data file of parent directory from OSD, and the metadata for being saved in parent directory is slow In depositing.As can be seen that every building level-one metadata cache needs to carry out a lookup operation in the prior art, this makes subtree The path of root to file or catalogue to be operated is longer, and the number of lookup is more in discover process, discover The time-consuming of process is also longer.Especially in the case where MDS has just taken over the dynamic subtree originally managed by failure MDS, due to needing The buffer structure of dynamic subtree is rebuild, the discover process of former secondary metadata request triggerings after adapter tube is ninety-nine times out of a hundred Lookup operation requires to read meta data file in OSD, the time-consuming of discover process is even more significantly increased, so that first The efficiency of data processing is poor.

Summary of the invention

The application provides a kind of method and device for constructing metadata cache, to solve discover stream in the prior art Journey takes a long time, the poor technical problem of metadata treatment effeciency.

In a first aspect, the embodiment of the present application provides a kind of method for constructing metadata cache, this method comprises: metadata takes Business device receives the metadata request that client proxy device is sent, and includes first identifier and the second mark in the metadata request Know, the first identifier is the mark of the first catalogue, and the second identifier is the mark of the subitem of first catalogue；If described The corresponding cache node of first catalogue, and the metadata are not present in the tree-like buffer structure that meta data server saves Server is the ownership meta data server of the dynamic subtree where the subitem, and the meta data server is described tree-like slow It deposits and creates the corresponding cache node of first catalogue in structure；The meta data server is read according to the first identifier The first meta data file stored in storage equipment, and the content of first meta data file is saved in first catalogue Corresponding cache node, first meta data file are the meta data file of first catalogue, the first metadata text The content of part includes the metadata information of the subitem.

In the embodiment of the present application, when meta data server receives the metadata request for the subitem of the first catalogue, The cache node of the first catalogue is not present in the tree-like buffer structure of meta data server, and the meta data server is first In the case where the ownership meta data server of dynamic subtree where the subitem of catalogue, meta data server can be tied in tree-like caching The corresponding cache node of the first catalogue is created in structure, and first yuan stored in storage equipment is read according to the mark of the first catalogue The content of first meta data file is saved in the corresponding cache node of the first catalogue by data file.First meta data file Content include the first catalogue subitem metadata information, in the embodiment of the present application, by being created in tree-like buffer structure The corresponding cache node of first catalogue, the corresponding caching section of the first catalogue that the content of the first meta data file is saved in creation In point, the buffer structure for supporting that metadata operation is carried out to the subitem of the first catalogue can be constructed, without from top to bottom according to It is secondary to read the meta data file of each catalogue on subtree root to the catalogue identification path of the first catalogue simultaneously from storage equipment It saves, therefore, method provided by the embodiments of the present application can effectively reduce discover process from object storage device disk-read Number shortens the processing time of metadata request, accelerates process and returns to main business processing.

In a kind of possible design, the meta data server creates first mesh in the tree-like buffer structure Record corresponding cache node, comprising: the meta data server obtains the subtree root of the dynamic subtree where the subitem To the catalogue identification path of first catalogue；The meta data server is according to the subtree root to first catalogue Sequence, following processing successively is executed to each of catalogue identification path catalogue, until first catalogue is corresponding Cache node create complete: for any catalogue in the catalogue identification path, if the tree-like buffer structure is not present The corresponding cache node of the catalogue then creates the corresponding cache node of the catalogue in the tree-like buffer structure；

The meta data server created in the tree-like buffer structure the corresponding cache node of first catalogue it Afterwards, further includes: the meta data server creates the corresponding cache node of the subitem in the tree-like buffer structure.

Due to using tree-like buffer structure, it is necessary to which the cache node of the upper level catalogue based on creation creates next The cache node of grade catalogue.Therefore, in the embodiment of the present application, it is corresponding slow that building creates the first catalogue in tree-like buffer structure It deposits node or constructs the buffer structure of the first catalogue it is meant that successively creation from top to bottom is sub since subtree root The corresponding cache node of each catalogue in tree root catalogue to the catalogue identification path of the first catalogue.In this way, metadata can be supported Server is handled metadata information in the corresponding cache node of the first catalogue.

In a kind of possible design, the content of first meta data file is saved in institute by the meta data server After stating the corresponding cache node of the first catalogue, further includes: the meta data server is the corresponding caching of first catalogue Itself insincere label and subitem fiducial markers are arranged in node, itself fiducial markers is arranged for the corresponding cache node of the subitem It is the subtree root newly created other mesh into the catalogue identification path of first catalogue with the insincere label of subitem It records corresponding cache node and itself insincere label and the insincere label of subitem is set.

In the embodiment of the present application, meta data server is during the first catalogue of creation corresponding cache node, only meeting The basic buffer structure handled the metadata information of the subitem of the first catalogue is supported in creation, although creating subtree root The corresponding cache node of each catalogue in catalogue to the catalogue identification path of the first catalogue, but only the is obtained from storage equipment The content of one meta data file and preservation, and for the corresponding cache node of other catalogues on the catalogue identification path, then prolong Content and the preservation of meta data file are obtained afterwards.It is appreciated that caching section corresponding for catalogue any in tree-like buffer structure Point, if meta data server once received the metadata request for the subitem of the catalogue or processed the subitem of the catalogue Metadata information, then the content of the meta data file of the catalogue is preserved in the cache node, if meta data server The metadata request for the subitem of the catalogue had not been received or did not processed the metadata information of the subitem of the catalogue, that The mark of the catalogue is only preserved in the cache node, and meta data server can not be supported to first number of the subitem of the catalogue It is believed that breath is handled.Therefore, in order to which whether each cache node effectively identified in tree-like buffer structure preserves catalogue The content of meta data file, in the embodiment of the present application, corresponding state mark can be arranged for each cache node in meta data server Know, which may include " subitem fiducial markers ", " the insincere label of subitem ", " itself fiducial markers ", " itself is insincere Totally four kinds of label ".

In a kind of possible design, meta data server receive client proxy device send metadata request it Afterwards, the method also includes: if there are first catalogue is corresponding in the tree-like buffer structure that the meta data server saves Cache node, and the corresponding cache node of first catalogue is equipped with the insincere label of subitem, the Metadata Service Device reads the first meta data file stored in the storage equipment according to the first identifier, and by first metadata The content of file is saved in the corresponding cache node of first catalogue；The meta data server is corresponding by first catalogue The insincere label of subitem of cache node remove, and subitem is arranged for the corresponding cache node of first catalogue can beacon Note.

In the embodiment of the present application, if there are the corresponding cache node of the first catalogue in tree-like buffer structure, but the first catalogue Corresponding cache node is equipped with the insincere label of subitem, then it represents that is only preserved in the corresponding cache node of the first catalogue One mark, therefore, meta data server can read the first meta data file stored in storage equipment according to first identifier, and will The content of first meta data file is simultaneously stored in the corresponding cache node of the first catalogue, so as to the subsequent subitem to the first catalogue Metadata information handled.

In a kind of possible design, the meta data server reads the first metadata stored in the storage equipment Before file, further includes: corresponding slow if there is no the subitems in the tree-like buffer structure that the meta data server saves Node is deposited, the meta data server creates the corresponding cache node of the subitem in the tree-like buffer structure；

It is corresponding slow that the content of first meta data file is saved in first catalogue by the meta data server After depositing node, further includes: the meta data server be the corresponding cache node of the subitem be arranged itself fiducial markers and The insincere label of subitem.

In the embodiment of the present application, there are the corresponding cache node of the first catalogue in tree-like buffer structure, and the first catalogue In the case that corresponding cache node is equipped with the insincere label of subitem, there may be current requests in tree-like buffer structure The corresponding cache node of subitem, it is also possible to the corresponding cache node of the subitem be not present, therefore, meta data server can be tree-like In the case where the corresponding cache node of the subitem is not present in buffer structure, the corresponding cache node of creation subitem, and phase is set The status indication answered.

In a kind of possible design, the meta data server creates first mesh in the tree-like buffer structure Before recording corresponding cache node, further includes: the meta data server obtains described the saved in birth record server The birth of one catalogue records, and includes the protection level information of first catalogue in the birth record of first catalogue；

The meta data server reads the first meta data file stored in storage equipment, comprising: the metadata clothes Device be engaged according to the protection level information of the first identifier and first catalogue, determines first meta data file described The storage location in equipment is stored, reads first meta data file from the storage location.

In the embodiment of the present application, the protected level of storage location and first catalogue of first meta data file in storage equipment It is not information-related, and the protection level information of first catalogue is then located in the birth record of the first catalogue, therefore, metadata clothes Business device can first obtain the birth record of the first catalogue saved in birth record server, obtain the protection level of first catalogue Information, and then further according to the protection level information of first identifier and the first catalogue, determine that the first meta data file is set in storage Storage location in standby reads the first meta data file from the storage location.

It further include file manipulation command in the metadata request in a kind of possible design；

It is corresponding slow that the content of first meta data file is saved in first catalogue by the meta data server After depositing node, further includes: the meta data server is according to the file manipulation command, to the caching section of first catalogue The metadata information of the subitem saved in point is handled.

It further include file manipulation command in the metadata request in a kind of possible design；

After the meta data server receives the metadata request that client proxy device is sent, the method is also wrapped It includes: if there are the corresponding cache node of first catalogue in the tree-like buffer structure that the meta data server saves, and institute It states the corresponding cache node of the first catalogue and is equipped with subitem fiducial markers, the meta data server is according to the file operation Order, is directly handled the metadata information of the subitem saved in the cache node of first catalogue.

In the embodiment of the present application, if there are the corresponding cache node of the first catalogue in tree-like buffer structure, and the first catalogue Corresponding cache node is equipped with subitem fiducial markers, then it represents that preserves first yuan in the corresponding cache node of the first catalogue The content of data file, therefore, meta data server is directly to first number of the subitem in the corresponding cache node of the first catalogue It is believed that breath is handled.

In a kind of possible design, the meta data server is to the institute saved in the cache node of first catalogue State subitem metadata information handled before, further includes: if the meta data server save tree-like buffer structure in There is no the corresponding cache node of the subitem, the meta data server creates the subitem in the tree-like buffer structure Corresponding cache node, and itself fiducial markers and the insincere label of subitem are set for the corresponding cache node of the subitem.

In the embodiment of the present application, there are the corresponding cache node of the first catalogue in tree-like buffer structure, and the first catalogue In the case that corresponding cache node is equipped with subitem fiducial markers, it is possible to there is no currently ask in tree-like buffer structure The corresponding cache node of the subitem asked, therefore, meta data server can be corresponding there is no the subitem in tree-like buffer structure In the case where cache node, the corresponding cache node of creation subitem, and corresponding status indication is set.

In a kind of possible design, the meta data server is to the institute saved in the cache node of first catalogue State subitem metadata information handled after, further includes: the meta data server is by the member of treated the subitem Data information is sent to the storage equipment.

In the embodiment of the present application, the first number for the subitem that meta data server can save in the cache node to the first catalogue It is believed that the metadata information of treated the subitem is sent to object storage device, so that object is deposited after breath is handled Store up the content that equipment updates the first meta data file.

Second aspect, the embodiment of the present application provide a kind of device for constructing metadata cache, which has realization above-mentioned The function of meta data server in any possible design of first aspect or first aspect, the function can pass through hardware reality It is existing, corresponding software realization can also be executed by hardware, the hardware or software include one or more and above-mentioned function phase Corresponding module.

It include processing module and transceiver module, the processing mould in the structure of described device in a kind of possible design Block is configured as corresponding function in any design for supporting the device to execute above-mentioned first aspect or first aspect.The receipts Hair module is used to support the communication between the device and other communication equipments (such as birth record server, object storage device). Described device can also include memory module, and the memory module is coupled with processing module, preserve device for necessary journey Sequence instruction and data.As an example, processing module can be processor, and communication module can be transceiver, memory module It can be memory.

The third aspect, the embodiment of the present application provide a kind of chip, and the chip is coupled with memory, for reading and executing The software program stored in the memory, to realize the method or reality in above-mentioned first aspect in any possible design Method in existing above-mentioned second aspect in any possible design.

Fourth aspect, the embodiment of the present application provide a kind of computer readable storage medium, in the computer storage medium It is stored with computer-readable instruction, when computer is read and executes the computer-readable instruction, so that computer executes State the method in first aspect in any possible design.

5th aspect, the embodiment of the present application provides a kind of computer program product, when computer is read and executes the meter When calculation machine program product, so that computer executes the method in above-mentioned first aspect in any possible design.

Detailed description of the invention

Fig. 1 is the structural schematic diagram of the dynamic subtree subregion in the embodiment of the present application；

Fig. 2 is the system architecture diagram of metadata processing system provided by the embodiments of the present application；

Fig. 3 is the structural schematic diagram of the meta data file stored in OSD in the embodiment of the present application；

Fig. 4 is the interactive relation signal of the network element of each type in metadata processing system provided by the embodiments of the present application Figure；

Fig. 5 is a kind of flow diagram of method for constructing metadata cache provided by the embodiments of the present application；

Fig. 6 a is the structural schematic diagram of the dynamic subtree provided in the application specific embodiment；

Fig. 6 b is the structural schematic diagram of the tree-like buffer structure provided in the application specific embodiment；

Fig. 7 is a kind of structural schematic diagram of device for constructing metadata cache provided by the embodiments of the present application；

Fig. 8 is the structural schematic diagram of the device of another building metadata cache provided by the embodiments of the present application.

Specific embodiment

In order to keep the purposes, technical schemes and advantages of the application clearer, with reference to the accompanying drawings of the specification to the application Embodiment is specifically described.It should be noted that the term that presently filed embodiment part uses is only used for the application Specific embodiment explain, and be not intended to limit the application.

Fig. 2 is a kind of system architecture diagram of metadata processing system provided by the embodiments of the present application, as shown in Fig. 2, this yuan Data processing system includes by the NAS cluster of multiple client agency's (client agent, CA) module composition, by multiple MDS The MDS cluster of composition records server (birth server) by the OSD multiple data subdomains formed and birth.Separately below It describes in detail for the network element of each type in the metadata processing system.

1) CA module

CA module can receive the file access request that simultaneously customer in response end is sent, while CA module with client communication Or the upper-layer service interface of MDS, can be communicated with MDS, send metadata request to MDS.Specifically, CA module can be right The file operation word in file access request that client is sent carries out semantic parsing, converts file system for file operation word Internal file manipulation command, and then carry and be sent to MDS in metadata request.It should be noted that the embodiment of the present application In the file operation word that is related to can for newly-built (creat), read (read), write (write), in creation subdirectory (mkdir) It is any one or several, or other types of file operation word, the application are specifically not construed as limiting.It should be understood that the application CA module in embodiment can be independently deployed on a physical node, and identical physical node can also be deployed in MDS, But it handles logical AND MDS and keeps independent.

2)MDS

MDS is responsible for managing the metadata cache of one or more dynamic subtree in the directory tree of file system, externally mentions For metadata business processing, including updating the content of metadata cache, receiving the metadata request of CA module transmission, according to file Operational order executes metadata operation etc. to metadata.For example, directory tree is divided into 4 dynamic not overlapped in Fig. 1 It sets subregion (dynamic subtree can be referred to as), wherein MDS1 is responsible for managing the metadata cache of dynamic subtree 1, and MDS2 is responsible for pipe The metadata cache of dynamic subtree 2 is managed, MDS3 is responsible for managing the metadata cache of dynamic subtree 3, and MDS4 is responsible for management dynamic The metadata cache of tree 4.

In the embodiment of the present application, the father node in dynamic subtree can represent a catalogue in file system, and dynamically Leaf node in subtree can represent a catalogue or a file in file system.In order to realize the caching of metadata It manages, the tree-like buffer structure of the dynamic subtree of its management is preserved in MDS, which can regard dynamic as The caching of tree is made of gradually creating in MDS along with CA module to the gradually access of metadata, it can be by being similar to The data structure of binary tree or other types of data structure realize that the application is not especially limited this.It should be understood that working as When MDS is responsible for managing more dynamic subtrees, then the tree-like buffer structure of more dynamic subtrees is also correspondingly preserved in MDS, but The embodiment of the present application only illustrates the building member number provided by taking the tree-like buffer structure of one of dynamic subtree of MDS management as an example According to the method for caching, hereafter repeat no more.

Specifically, tree-like buffer structure includes at least one cache node, i.e. the subtree root of dynamic subtree is corresponding Cache node.Cache node in tree-like buffer structure is used to save the metadata cache of corresponding catalogue or file, and metadata is slow Deposit be catalogue or file meta data file (also referred to as object disk) caching of the content in MDS.In the embodiment of the present application, The meta data file of catalogue is persistently stored in object storage device, and metadata operation be then MDS metadata it is slow It is handled in depositing, retransmits after being disposed and stored into object storage device.When needing to carry out metadata operation, first number According to the metadata cache in server if there is no catalogue, meta data server need to read catalogue from object storage device The content of meta data file, and be saved in MDS and handled again, i.e., the metadata of catalogue is loaded into from object storage device In the caching of MDS.It should be noted that the content of metadata cache may include meta data file in the embodiment of the present application Full content can also only include the partial content of meta data file, catalogic mark.It should be noted that only including catalogue The metadata cache of mark can not support MDS to carry out metadata operation, and meta data server still need to when metadata operation Reading metadata in object storage device is gone to be saved in the caching of MDS again.

It should be noted that the structure of dynamic subtree can be adjusted according to the demand dynamic of system in the embodiment of the present application, It is not fixed and invariable.When all subitem numbers under dynamic subtree are much or amount of access overheats, dynamic subtree can also To be split into more small dynamic subtrees.The dynamic subtree of amount of access overheat can also move to relatively empty from busy MDS It is managed on not busy MDS.But a dynamic subtree can only at a time belong to a MDS, cannot belong to two simultaneously Or more MDS, otherwise will lead to metadata lower wall conflict." ownership " refers in the embodiment of the present application: if a dynamic subtree A MDS is belonged to, then all metadata processing under the dynamic subtree is all handled on ownership MDS, its in MDS cluster Its MDS cannot handle the metadata operation under the dynamic subtree.

3)MDS leader

One of MDS in MDS cluster is also used as the MDS leader in MDS cluster, which removes Have the function of outside general MDS also having the function of the management dynamic subtree such as cutting, migration, is responsible for maintaining file system The management of the global objects such as corresponding relationship and Fsblock between each dynamic subtree and corresponding ownership MDS and lower wall. In the embodiment of the present application, global object includes at least fsblock object and globalsubtree object, wherein fsblock pairs As the title of record file system and the metadata information of system root directory, in globalsubtree object record file system Each dynamic subtree root and ownership MDS corresponding relationship, the corresponding relationship pass through globalsubtree object lower wall To OSD layers.It should be noted that each global object in the embodiment of the present application has a global mark, deposited in OSD layers Contain the corresponding object disk of the two global objects.

For example, MDS leader may specify other normal MDS taking over fault MDS when a certain common MDS breaks down Dynamic subtree.It is former from read failure MDS on MDS leader after adapter tube MDS receives the adapter tube notice that MDS leader is sent Come all subtree root informations managed, the subtree root of these dynamic subtrees is created in memory according to these subtree root informations Corresponding metadata cache.With the access of business, adapter tube MDS in OSD gradually from reading and create the son under subtree root The metadata cache of item.In a kind of possible design, when a certain MDS breaks down, more dynamics of failure MDS management Subtree can also be taken over by different normal MDS respectively, in this way, the number for the dynamic subtree that adapter tube MDS need to take over can be effectively reduced Amount, the situation that the dynamic subtree for avoiding failure MDS from managing all causes the portfolio of adapter tube MDS to increase severely by a MDS adapter tube, Reduce influence when MDS breaks down to system performance.

When MDS leader breaks down, other MDS in MDS cluster, such as can be with by certain election contest rule The lesser MDS of ID is selected, selects a MDS as taking over sb.'s job MDS leader in normal MDS.It should be understood that in MDS cluster There is a managing process in each MDS, appearance can be worked as by the situation between heartbeat packet interaction MDS between each MDS When the situation of MDS leader failure, whom other normal MDS can negotiate in this way to serve as new MDS leader.The dynamic subtree of former MDS leader management will be taken over by taking over sb.'s job MDS leader not only, will also adapter tube MDS Global object's management function of leader.After taking over sb.'s job MDS leader and learning that oneself is upgraded to leader, starting leader first Adapter tube process is saved in buffer structure from reading fsblock object and globalsubtree object in OSD, leader adapter tube Success.Then, MDS leader searches dynamic of predecessor MDS leader management in the globalsubtree object of preservation Tree, and the corresponding metadata cache of subtree root of these dynamic subtrees is created in memory.With business access, gradually from The metadata cache of the subitem under subtree root is read and created in OSD.

4)birth server

Birth server is used to record the set membership information in file system between parent directory and subdirectory.Specifically , it can be used birth (birth) record to indicate the set membership between a subdirectory and its parent directory, for example, It include a subdirectory " subdir1 ", the content of the birth record of subdirectory " subdir1 " under catalogue " dir1 " are as follows: { subfid, parentfid, protectlevel, namehash }, wherein subfid refers to the mark of subdirectory " subdir1 " Know, parentfid refers to the mark of catalogue " dir1 ", and protectlevel refers to the protection level letter of subdirectory " subdir1 " Breath, namehash refer to the cryptographic Hash for 4 bytes that " subdir1 " this directory name is obtained by hash algorithm.

In the embodiment of the present application, it may include being constructed with being identified as key assignments of subdirectory in birth server, be used for The database of birth record in storage file system, the additions and deletions for providing birth record, which change, the operation such as looks into.The birth Server can be single node, be also possible to distributed, is determined, can be deployed in only by the total number that birth is recorded Vertical physical node can also be deployed in the same physical node with MDS, but handle logical AND MDS and keep independent.In this way, logical Cross to birth server repeatedly inquired it is available to birth all in any catalogue to the path of system root directory remember Record, and then obtain the mark of all higher level's catalogues of any catalogue.

It should be noted that the catalogue being related in the embodiment of the present application, file or global object mentioned hereinabove It is all made of unified naming method, system can distribute a global mark fid as text for each file, catalogue or global object Unique identification in part system.In a kind of possible design, which can be the number of 8 bytes.

5)OSD

Entire OSD layers be a persistence accumulation layer, reliable object store function is provided.It can be divided into multiple numbers According to subdomain, each data subdomain includes one or more OSD node, and each OSD node (i.e. object storage device OSD) is one Smart machine, it is responsible to manage the object being locally stored with oneself storage medium, processor, memory and network system etc..

In the embodiment of the present application, it is OSD layers internal in can be communicated by message between all OSD node.MDS is total OSD accumulation layer is enjoyed, external OSD layers provides unified object memory interface.Accordingly, it is to be understood that all MDS in MDS cluster are total Persistent storage layer is enjoyed, each MDS can see all OSD nodes, and any OSD section in accessible the OSD layers Point.

Specifically, OSD is used for the meta data file (alternatively referred to as object disk) of catalogue or file in storage file system, The meta data file of each catalogue preserves the direct subdirectory of the catalogue and/or the metadata information of subfile, and subdirectory The metadata information of subitem be then stored in the meta data file of subdirectory oneself.

In a kind of possible design, the metadata information of catalogue or file uses directory entry (dentry)+index node (inode) structure saves, and dentry is for the information such as name (or filename) that save contents, and inode is used to save contents or text The essential attribute of part, Authorization Attributes and be directed toward subitem metadata cache pointer, wherein essential attribute refers to the file of file The information such as size, access time, file type, Authorization Attributes refer to access authority management (access control list, ACL) permission, user group etc..

Fig. 3 illustrates the content of the meta data file of catalogue in OSD, as shown in figure 3, the metadata of a catalogue File includes the content of the dentry and inode of the head head and its subitem, the head head be used to store the number of subitem with And the contents such as modification time of newest subitem.In conjunction with Fig. 1 it is found that storing in the meta data file of system root directory has its subitem The content metadata of user and home, the dentry of dentry and inode and catalogue home including catalogue user and inode.Storage has the content metadata of its subitem file, test and file1, including file in the meta data file of catalogue user Dentry and inode, test dentry and inode and file1 dentry and inode.The metadata of catalogue home Storage has the content metadata of subitem lib and src, the dentry of lib and the dentry of inode, src and inode in file.

It should be noted that the meta data file of catalogue is deposited by the way of mirror image in OSD in the embodiment of the present application Storage.The quantity of mirror image is related to the protection level information of catalogue, and protection level information is then recorded in the birth record of catalogue In, protection level information is higher, and the quantity of mirror image is also more.For example, 3 Disk Localities in the corresponding OSD of protection level+2, are protected Protect 4 Disk Localities in the corresponding OSD of rank+3,5 Disk Localities in the corresponding OSD of protection level+4.MDS is by a certain mesh When the metadata lower wall of record is stored to OSD, hash algorithm selection can be passed through according to mark, the protection level information of catalogue of catalogue Several Disk Localities write-in on NLUN, each a current metadata of Disk Locality storage.

Fig. 4 illustrates CA module, MDS, birthserver, MDS in metadata processing system provided by the present application Interactive relation between leader, OSD.As shown in figure 4, CA module is used to send metadata request to MDS, and receive MDS hair The metadata request response sent.MDS is for receiving metadata request, the buffer structure that building processing metadata request needs, place Metadata request is managed, and the content of metadata cache is stored to OSD.Birthserver be used for provide birth record increasing, It the operation such as deletes, change, looking into, supporting MDS to construct buffer structure, MDS leader is for managing and recording subtree root and ownership The corresponding relationship of MDS, and store and arrive OSD.

Based on above system framework, Fig. 5 illustrates a kind of method for constructing metadata cache provided by the present application, As shown in figure 5, this method comprises the following steps:

Step S501: meta data server receives the metadata request that client proxy device is sent, the metadata request In include first identifier and second identifier, first identifier is the mark of the first catalogue, and second identifier is the subitem of first catalogue Mark.

Step S502: if there is no the corresponding caching sections of the first catalogue in the tree-like buffer structure that meta data server saves Point, and meta data server is the ownership meta data server of the dynamic subtree where the subitem, then meta data server exists The corresponding cache node of the first catalogue is created in tree-like buffer structure.

Step S503: meta data server reads the first meta data file stored in storage equipment according to first identifier, And the content of the first meta data file is saved in the corresponding cache node of the first catalogue, the first meta data file is the first catalogue Meta data file, the metadata information of subitem described in the content of the first meta data file.

In the embodiment of the present application, if the first mark comprising the first catalogue in the metadata request that client proxy device is sent Know the second identifier with the subitem of the first catalogue, then it represents that client proxy device requests meta data server under the first catalogue The metadata information of subitem with second identifier is handled.After meta data server receives metadata request, current The cache node of the first catalogue is not present in the tree-like buffer structure saved, and meta data server is to have the under the first catalogue In the case where the ownership meta data server of dynamic subtree where the subitem of two marks, meta data server can be in tree-like caching The corresponding cache node of the first catalogue is created in structure, and first stored in storage equipment is read according to the mark of the first catalogue The meta data file of catalogue, i.e. the first meta data file, and the content of first meta data file is saved in the first catalogue pair In the cache node answered.Due in the first meta data file include the first catalogue subitem metadata information, thus, metadata Server creates the corresponding cache node of the first catalogue in tree-like buffer structure, and the content of the first meta data file is saved After into the cache node of creation, meta data server just constructs the first number that can be supported to the subitem with second identifier It is believed that the buffer structure that breath is handled, without successively reading subtree from storage equipment from top to bottom according to TOC level The meta data file of each catalogue on root to the path of the first catalogue, so as to effectively reduce discover process from storage The number of equipment disk-read shortens the processing time of metadata request, accelerates process and returns to main business processing.

Specifically, in step S501, the first catalogue is a catalogue in file system, its mark is referred to as first Mark.It may include one or more subitems under first catalogue, second identifier is the mark of some subitem under the first catalogue, the son Item can be a file in file system, be also possible to a catalogue, in order to describe simplicity, herein by the subitem be known as to The subitem of operation.Meta data server can be any MDS in MDS cluster in above-mentioned metadata processing system, Client Agent Equipment can be the CA module in above-mentioned metadata processing system.

In the embodiment of the present application, client proxy device can carry first identifier and second identifier and send out to meta data server Metadata request is sent, may also include file manipulation command in the metadata request, is used to indicate meta data server according to this article Part operational order handles the metadata information of this subitem with second identifier of current request.Herein, to first number It is believed that breath carries out handling involved file manipulation command being the previously described corresponding text of any file operation word Part operational order is also possible to other file manipulation commands, is specifically not construed as limiting.

In step S502, since the metadata information of subitem to be operated is located in the first meta data file, metadata After server receives metadata request, the corresponding caching section of the first catalogue can be searched in the tree-like buffer structure currently saved Point, to determine that the metadata cache of the first catalogue whether there is.

If the corresponding cache node of the first catalogue is not present in current tree-like buffer structure, meta data server be can trigger The metadata cache of discover process building the first catalogue and subitem to be operated.Specifically, meta data server can be with First catalogue be starting, by birth record server successively obtain the first catalogue to file system system root directory path The birth record of upper all catalogues.First identifier can be sent to birth record server by meta data server, to obtain first The birth of catalogue records.Birth record server can be inquired from the birth record saved in database according to the first identifier Key assignments is that a birth of first identifier records, and the birth inquired record is sent to meta data server, birth note Record is the birth record of the first catalogue, including the guarantor of the mark of the parent directory of first identifier, the first catalogue, the first catalogue Protect the contents such as level information.

Due to the mark of the parent directory including the catalogue of being born in record of each catalogue, in this way, meta data server The mark of the parent directory of the first catalogue can be obtained after the birth record for receiving the first catalogue.Assuming that the parent directory of the first catalogue For third catalogue, third catalogue is identified as third mark.Subsequent, third can be identified and be sent to birth note by meta data server Server is recorded, the birth record of third catalogue is obtained, just available the in this way from the birth of the third catalogue inquired record The mark of the parent directory of three catalogues.The rest may be inferred, meta data server by every time from the birth of subdirectory record obtained in father The mark of catalogue retransmits to birth and records server, obtains the birth record of parent directory, obtains the mark of the parent directory of parent directory Know, until the parent directory in a certain item birth record inquired is system root directory.

By obtaining birth record all in the first catalogue to the path of the system root directory of file system, metadata clothes Business device can obtain the mark of all catalogues in first catalogue to the path of system root directory.In turn, meta data server can incite somebody to action The mark of these catalogues successively uses "/" symbol to splice according to the sequence of system root directory to the first catalogue, obtains system root directory To the catalogue identification path of the first catalogue, finally the mark splicing of subitem to be operated is identified in the catalogue with "/" symbol again Path finally, obtain system root directory to subitem to be operated catalogue identification path.

Then, the catalogue identification path of system root directory to subitem to be operated can be sent to subtree by meta data server Belong to management server (the MDS leader i.e. in MDS cluster), to inquire the son of the dynamic subtree where subitem to be operated The ownership meta data server of the mark of tree root catalogue and the dynamic subtree.Later, MDS leader can be by son to be operated The mark of the corresponding ownership meta data server of the mark and dynamic subtree of the subtree root of dynamic subtree where is sent To meta data server.

If the meta data server is not the ownership meta data server of the dynamic subtree where subitem to be operated, first number Failure response message can be sent to client proxy device according to server, to terminate current metadata treatment process.This Shen It please include the mark of the ownership meta data server of subitem to be operated in failure response message in embodiment, the mark is available In client proxy device after receiving the failure response message, metadata request is re-transmitted to subitem institute to be operated Dynamic subtree ownership meta data server, the metadata request is handled by the ownership meta data server.

If the meta data server is the ownership meta data server of the dynamic subtree where subitem to be operated, metadata Server can create the corresponding cache node of the first catalogue in current tree-like buffer structure, and corresponding in first catalogue First identifier is saved in cache node.In a kind of possible design, meta data server creates in tree-like buffer structure The corresponding cache node of one catalogue specifically includes: firstly, meta data server according to subtree belong to management server send to The mark of the subtree root of dynamic subtree where the subitem of operation determines the catalogue mark of the subtree root to the first catalogue Know path；Then, meta data server traverses on the catalogue identification path according to subtree root to the sequence of the first catalogue Each catalogue executes following processing for each catalogue on the catalogue identification path: in current tree-like buffer structure Search the currently processed corresponding cache node of this catalogue, however, it is determined that do not include that the catalogue is corresponding slow in tree-like buffer structure Node is deposited, then creates the corresponding cache node of the catalogue in tree-like buffer structure, and save the catalogue in the cache node Mark.

In this way, all catalogues on subtree root to the catalogue identification path of the first catalogue have successively been traversed, it is tree-like slow It deposits in structure just including the corresponding cache node of catalogues all on subtree root to the path of the first catalogue, that is, has constructed son The buffer structure of all catalogues in tree root catalogue to the path of first object.

Further, the subitem as the first catalogue, cache node including the first catalogue in tree-like buffer structure In the case of, it does not include the cache node of subitem to be operated in tree-like buffer structure yet.In a kind of possible design, metadata It, can also be in the slow of the first catalogue of creation after server creates the corresponding cache node of the first catalogue in tree-like buffer structure On the basis of depositing node, the cache node of subitem to be operated is created under the cache node of the first catalogue, and in the caching Second identifier is saved in node.

It should be noted that in this course, meta data server is for the corresponding cache node of a certain directory creating When, there is no the contents for the meta data file that the catalogue is read from storage equipment, and save in cache node, therefore, new to create The mark for only preserving catalogue in the cache node built for the time being, does not preserve the content of the meta data file of the catalogue, so The metadata information for the subitem that current tree-like buffer structure can't support meta data server to treat operation is handled.More It is corresponding for the directory creating for that can be for the corresponding cache node of a certain directory creating specifically, in the embodiment of the present application Dentry and inode, wherein dentry is used for the mark to save contents, metadata letter of the inode for the lower subitem that saves contents Breath.In the case where not reading meta data file from storage equipment, the part inode is empty in usual newly created cache node , meta data server cannot be supported to handle the metadata information of the subitem of catalogue.

In step S503, storage equipment can be the object storage device OSD in above-mentioned metadata processing system, metadata Server can store the first meta data file for storing in equipment according to first identifier reading object, and by the first meta data file Content be saved in the corresponding cache node of the first catalogue.In the embodiment of the present application, the content of the first meta data file includes The metadata information of each subitem under first catalogue, and the metadata information of each subitem may each comprise dentry as indicated at 3 Structure and inode structure.It is appreciated that including the metadata information of subitem to be operated, Yi Ji in the first meta data file The metadata information of other each subitems under one catalogue in addition to subitem to be operated.Meta data server is by the first metadata The content of file is saved in tree-like buffer structure after the cache node of the first catalogue, and the cache node of first catalogue is preserved The metadata information of subitem to be operated can support meta data server to treat at the metadata information of the subitem of operation Reason, so far discover process terminates.

In a kind of possible design, meta data server can be according to first identifier and from the birth of the first catalogue record The protection level information of the first obtained catalogue determines storage location of first meta data file in object storage device, And then the first meta data file is read from the storage equipment, and the content of the first meta data file is saved to the first catalogue and is corresponded to Cache node in.Specifically, meta data server can pass through Kazakhstan according to the protection level information of first identifier and the first catalogue Uncommon algorithm, is calculated storage location of first meta data file in object storage device.In practical application scene, according to The difference of the protection level information of first catalogue, the first meta data file may have multiple storage positions in object storage device It sets, and each storage location is stored with a identical first meta data file.Therefore, meta data server is true in this step Storage location of the first meta data file made in object storage device can be the first meta data file and store in object Any storage location in multiple storage locations of equipment.

After step S503, meta data server can be according to the file manipulation command in metadata request, to tree-like slow The metadata information for depositing the subitem to be operated saved in the corresponding cache node of the first catalogue in structure is handled.In one kind May also include client proxy device in possible design, in metadata request according to this document operational order is son to be operated The content metadata of item setting, for example, the content metadata that should be arranged for subitem to be operated can be as subitem to be operated The attribute informations such as file size, access time, the access authority of setting.In this way, meta data server can be operated according to this document Order and the content metadata being arranged for the subitem to be operated, modify the metadata information of subitem to be operated.

After step S503, the first mesh that meta data server can will also save in the corresponding cache node of the first catalogue The metadata information of each subitem of record is sent to object storage device, so that first metadata of the object storage device to storage File is updated.In a kind of possible design, meta data server can also only will be in the corresponding cache node of the first catalogue The metadata information of the subitem to be operated saved is sent to object storage device, to make object storage device with treated The member of original subitem that should be to be operated in meta data file of the metadata information of subitem to be operated to replace the first catalogue Data information.In addition, meta data server can also send processing success response to client proxy device, to notify client The metadata information of agent equipment request has handled completion.

Since the protection level information of each catalogue is recorded in the birth record of the catalogue oneself, meta data server root The protection level information got in mark according to catalogue and the birth record from catalogue, so that it may obtain the metadata of the catalogue Storage location of the file in object storage device, so that the content of the meta data file of catalogue is saved in tree-like buffer structure In in corresponding cache node, first number of the catalogue whether is preserved without the cache node of the parent directory by the catalogue It is believed that the limitation of breath.Even if the corresponding cache node of parent directory of the catalogue only preserves father's mesh in current tree-like buffer structure The mark of record does not save any content metadata of parent directory, will not influence first number that meta data server obtains the catalogue According to storage location of the file in object storage device, the content of the meta data file of the catalogue is read from object storage device And it is saved in corresponding cache node.

In this way, in view of tree-like buffer structure requires meta data server to construct before operation metadata information wait operate Subitem where subtree root to the path of the first catalogue on all catalogues buffer structure.In the embodiment of the present application, member There is no the first catalogue is corresponding in the tree-like buffer structure that data server can receive metadata request, but currently save In the case where cache node, only building supports meta data server to carry out the basic buffer structure of metadata operation, thus effectively The response time for shortening metadata request, improve metadata treatment effeciency.In the embodiment of the present application, metadata clothes are supported in building The basic buffer structure that business device carries out metadata operation specifically refers to, where creating subitem to be operated in tree-like buffer structure Dynamic subtree subtree root to the path of the first catalogue on the corresponding cache node of each catalogue (unless the caching of catalogue Node has existed), only for the first catalogue in the path, obtains the content of the first meta data file and be saved in the first catalogue Corresponding cache node, and for the other catalogues for being different from the first catalogue on the path, meta data server no longer removes object The content of reading meta data file and preservation in storage equipment, but the mark only to save contents in the cache node of creation, When subsequently received request operates the metadata request of the metadata information of subitem under these catalogues, then remove object storage device It is middle to read the content metadata needed.In this way, can effectively reduce discover process reads metadata from object storage device Number, shorten discover process time-consuming.

Using technical solution provided by the embodiments of the present application, in the tree-like buffer structure that is saved in meta data server, Preserve the content of the meta data file of subtree root in the corresponding cache node of subtree root, and dynamic subtree other The corresponding cache node of catalogue, then may preserve the content of the meta data file of the catalogue, it is also possible to only preserve catalogue Mark.For example, if meta data server create the cache node post-processed the catalogue subitem metadata information, then The cache node can be reserved for the content for having the meta data file of the catalogue, if meta data server does not have after creating the cache node Have the metadata information for the subitem for processing the catalogue, although meta data server process first number of the subitem of the catalogue It is believed that breath, but the content of the meta data file of the catalogue saved in the cache node because the time relatively long, temperature it is more low because Element is eliminated in the buffer, and the mark of catalogue can be only preserved in the cache node.

Therefore, for metadata text that whether effective identification metadata server read from object storage device catalogue The content of part, whether cache node preserves the content of the meta data file of catalogue in other words, in the embodiment of the present application, metadata Also status indication can be arranged for each cache node in tree-like buffer structure in server, which may include that " subitem can Beacon note ", " the insincere label of subitem ", " itself fiducial markers ", totally four kinds of " itself insincere label ".

Specifically, after meta data server can create the corresponding cache node of the first catalogue in tree-like buffer structure, " itself insincere label " and " subitem fiducial markers " is set for the corresponding cache node of the first catalogue, for subitem pair to be operated " itself fiducial markers " and " the insincere label of subitem " are set in the cache node answered, for subtree root to the road of the first catalogue The corresponding cache node setting " itself insincere label " and " the insincere label of subitem " of newly created other catalogues in diameter.

In the embodiment of the present application, a catalogue is equipped with " subitem fiducial markers " expression, and meta data server is from object The meta data file of the catalogue was read in storage equipment, in current tree-like buffer structure in the corresponding cache node of the catalogue The content of the meta data file of the catalogue is preserved, that is, preserves the metadata information of each subitem under the catalogue, it correspondingly, should The corresponding cache node of each subitem is equipped with " itself fiducial markers " under catalogue.And a catalogue is equipped with that " subitem is not Fiducial markers " indicate that meta data server does not need the meta data file that the catalogue is read from object storage device temporarily Content is currently only preserved the mark of the catalogue for the time being, is not protected in the corresponding cache node of the catalogue of tree-like buffer structure The content of the meta data file of the catalogue is deposited, correspondingly, the corresponding cache node of each subitem of the catalogue is equipped with " itself Insincere label ".

It follows that if meta data server receive client proxy device transmission metadata request when, metadata The corresponding cache node of the first catalogue is not present in the tree-like buffer structure that server currently saves, that identical element data server can The cache node of each catalogue on creating catalogue identification path of the subtree root where subitem to be operated to the first catalogue Afterwards, the content of the meta data file of the first catalogue is only obtained from object storage device and is saved in the corresponding caching of the first catalogue Node, while " subitem fiducial markers " are set for the corresponding cache node of the first catalogue, and on the catalogue identification path its It does not need the corresponding cache node setting " subitem of catalogue that the content of meta data file is obtained from object storage device temporarily Insincere label " when waiting has front desk service request request to handle the metadata information of the subitem of these catalogues, then is gone The content of corresponding meta data file and preservation are obtained in object storage device.In this way, removing the first catalogue on catalogue identification path The metadata cache content of other catalogues in addition delays preservation to cache node, can enable metadata processing system quick response Foreground request processing, the performance of adapter tube MDS declines excessive fluctuation when reducing MDS failure, while can also solve to file system The limitation of directories deep.

In a kind of possible design, meta data server can also only be the cache node setting in tree-like buffer structure " subitem fiducial markers " and " itself fiducial markers ".In this way, when the cache node of a certain catalogue is not equipped with, " subitem can beacon When note ", it can indicate that the cache node is that subitem is incredible, meta data server does not need temporarily to read the metadata of the catalogue The content of file is simultaneously saved to the cache node；It, can when " itself fiducial markers " are not set for the cache node of a catalogue Indicate that the cache node is that itself is incredible, meta data server does not need temporarily to read the metadata of the parent directory of the catalogue The content of file is simultaneously saved to the cache node of parent directory.

In consideration of it, in step S502, if meta data server receives the metadata request that client proxy device is sent Afterwards, there are the corresponding cache nodes of the first catalogue in the tree-like buffer structure that meta data server currently saves, and first mesh It records corresponding cache node and is equipped with the insincere label of subitem, that identical element data server can be also needed according to first identifier The first meta data file stored in storage equipment is read, and the content of the first meta data file is saved in the first catalogue and is corresponded to Cache node.Specifically, meta data server can determine first according to the protection level information of first identifier and the first catalogue Storage location of the meta data file in object storage device obtains the content of the first meta data file from the storage location, and It is saved in the corresponding cache node of the first catalogue；Then, meta data server can also be by the corresponding cache node of the first catalogue The insincere label of subitem is removed, and subitem fiducial markers are arranged；In turn, further according to the file manipulation command in metadata request, The metadata information of the subitem to be operated saved in the cache node of first catalogue is handled.

If meta data server is currently protected after meta data server receives the metadata request that client proxy device is sent There are the corresponding cache nodes of the first catalogue in the tree-like buffer structure deposited, and the corresponding cache node of the first catalogue is equipped with Subitem fiducial markers, that identical element data server can be according to the file manipulation commands in metadata request, directly to the first catalogue Cache node in the metadata information of subitem to be operated that saves handled.

It should be noted that after the metadata request that meta data server receives that client proxy device is sent, first number There are in the case where the corresponding cache node of the first catalogue in the tree-like buffer structure currently saved according to server, if the first catalogue Corresponding cache node is equipped with the insincere label of subitem, then there may be may not also in current tree-like buffer structure In the presence of the corresponding cache node of subitem to be operated.Therefore, meta data server can determine the first meta data file in object Before storage location in storage equipment, however, it is determined that there is no the corresponding caching sections of subitem to be operated in tree-like buffer structure Point then creates the corresponding cache node of the subitem, and itself fiducial markers and subitem is arranged for the corresponding cache node of the subitem Insincere label.

Similarly, after the metadata request that meta data server receives that client proxy device is sent, meta data server There are in the case where the corresponding cache node of the first catalogue in the tree-like buffer structure currently saved, delay if the first catalogue is corresponding It deposits node and is equipped with subitem fiducial markers, it is also possible that subitem pair to be operated is not present in current tree-like buffer structure The cache node answered.Therefore, the subitem to be operated that meta data server can save in the cache node to the first catalogue Before metadata information is handled, the corresponding cache node of the subitem is created in tree-like buffer structure, and is the subitem pair Itself fiducial markers and the insincere label of subitem are arranged in the cache node answered.

Metadata processing method provided by the embodiments of the present application is described in detail below by a specific embodiment.

Fig. 6 a illustrates the dynamic subtree in this specific embodiment, as shown in Figure 6 a, system root directory 400, The subtree root of dynamic subtree is catalogue 2100, and the lower catalogue 2100 is respectively again catalogue 2200, mesh including three subdirectories Record 2201, catalogue 2202, wherein under catalogue 2200 again including three subdirectories be respectively catalogue 2300, catalogue 2301, catalogue 2302, it is catalogue 2400 and catalogue 2401 respectively that it includes two subdirectories that catalogue 2300 is lower again.

Fig. 6 b illustrates the tree-like buffer structure corresponding with dynamic subtree shown in Fig. 6 a saved in MDS1.Assuming that This dynamic subtree in Fig. 6 a belongs to MDS2 management, but since MDS2 breaks down, MDS1 has taken over this stalk tree.? Before MDS1 receives the metadata request that client proxy device is sent, MDS1 has just taken over this dynamic subtree, saves in MDS1 It only include the corresponding cache node of subtree root 2100 in tree-like buffer structure.

Firstly, MDS1 receives the metadata request that CA module is sent, it include catalogue 2300 and catalogue in the metadata request 2400 mark.

Then, MDS1 in figure 6b shown in the corresponding cache node of catalogue 2300 is searched in tree-like buffer structure.It searches It was found that the corresponding cache node of catalogue 2300 is not present in tree-like buffer structure, then discover process is triggered.MDS1 is to birth Server sends the mark of catalogue 2300, obtains the birth record of catalogue 2300, includes mesh in the birth record of the catalogue 2300 The contents such as mark, the protection level of the mark of catalogue 2200 and catalogue 2300 of record 2300.Later, MDS1 is to birth server The mark of catalogue 2200 is sent, the birth record of catalogue 2200 is obtained, includes catalogue in the birth record of the catalogue 2200 The contents such as the protection level of 2200 mark, the mark of catalogue 2100 and catalogue 2200.The rest may be inferred, and MDS1 gets catalogue 2300 to catalogues all between system root directory 400 birth record, and according to the mark of these catalogues obtain catalogue 2300 to The catalogue identification path of system root directory 400: " 400/2100/2200/2300 ".Meta data server is by the mark of catalogue 2400 Splicing obtains the mesh of catalogue 2400 to system root directory 400 after the catalogue identification path of catalogue 2300 to system root directory Record identification path: " 400/2100/2200/2300/2400 ".

In turn, the catalogue identification path of catalogue 2400 to system root directory 400 is sent to MDS leader, obtained by MDS1 Take the subtree root and ownership MDS of the dynamic subtree where catalogue 2400.Correspondingly, MDS leader is by the mark of catalogue 2100 Know and the mark of MDS1 is back to MDS1.MDS1 determine oneself be exactly dynamic subtree where catalogue 2400 ownership MDS after, in Be, according to sequence from top to bottom, successively create directory in tree-like buffer structure 2200 cache node, catalogue 2300 it is slow Deposit the cache node of node and catalogue 2400.Here each cache node created includes dentry and inode two parts, In, dentry is used for the mark to save contents, the metadata information for each subitem that inode is used to save contents.But due to Content metadata is not read from OSD when creating these three cache nodes, so the part inode of cache node is actually It is also the metadata information for including subitem.

Further, MDS1 can be got according to the mark of catalogue 2300 and from the birth of catalogue 2300 record Protection level determines storage location of the meta data file of catalogue 2300 in OSD, and obtains catalogue from the storage location The content of 2300 meta data file, and it is saved in the cache node of catalogue 2300, include in the meta data file of catalogue 2300 The metadata information of catalogue 2400 and catalogue 2401.In this way, MDS1 can be credible for the cache node setting subitem of catalogue 2300 Label and itself insincere label are arranged the insincere label of subitem and itself fiducial markers for catalogue 2400, are catalogue 2200 Cache node is arranged that subitem is insincere and itself insincere label, and so far discover process terminates.

Finally, what MDS1 can be saved in the cache node of catalogue 2300 according to the file manipulation command in metadata request 2400 metadata information is handled, and again sends out the content metadata saved in the cache node of catalogue 2300 after being disposed It send and is stored into OSD.

In addition, requesting the metadata request handled the metadata of the subitem of catalogue 2300 when MDS1 is subsequently received When, MDS1 can be operated directly in the cache node of catalogue 2300, not need to trigger discover process again.Work as MDS1 When the metadata request that subsequently received request handles the metadata of the subitem of catalogue 2200, MDS1 can be according to catalogue The insincere label of subitem of 2200 settings, then go in OSD the content for the meta data file for reading catalogue 2200 and save to catalogue In 2200 cache node.

Since the protection level information of catalogue 2300 is present in the birth record of catalogue 2300, in creation subtree root mesh Record 2100 is to when the cache node of each catalogue, MDS1 can be directly according to the mark and mesh of catalogue 2300 on the path of catalogue 2300 The protection level information of record 2300 goes in OSD to read the content of the meta data file of catalogue 2300, rather than must first read mesh The content of the meta data file of catalogue 2300 could be read after the content of the meta data file of record 2200.In this way, processing metadata MDS1 can only read a content metadata from object storage device when request, to can effectively reduce discover stream Disk-read number in journey, improves the response speed of metadata processing, and the performance for avoiding adapter tube dynamic subtree from causing adapter tube MDS declines And since the too deep caused MDS of directories deep handles the problems such as overtime.

Based on identical inventive concept, the embodiment of the present application also provides a kind of device for constructing metadata cache, refers to Shown in Fig. 7, for a kind of structural schematic diagram for the device for constructing metadata cache provided by the embodiments of the present application.As shown in fig. 7, should Device includes: transceiver module 710 and processing module 720.

Transceiver module 710, for executing the operation for receiving the metadata request that client proxy device is sent；Processing module 720, the corresponding cache node of the first catalogue is not present in the tree-like buffer structure that the present apparatus saves for executing, and the present apparatus In the case where ownership meta data server for the dynamic subtree where the subitem of the first catalogue of current request, in tree-like caching The corresponding cache node of the first catalogue is created in structure, and the first metadata text stored in storage equipment is read according to first identifier The content of part, and the content of first meta data file is saved in the corresponding cache node of the first catalogue.It should be understood that the present invention Processing module 720 in embodiment can realize that transceiver module 710 can be by receiving by processor or processor associated circuit components It sends out device or transceiver associated circuit components is realized.

It is shown in Figure 8, for another structure of the device of the building metadata cache provided in the embodiment of the present application Schematic diagram.As shown in figure 8, the device 800 includes processor 810, memory 820 and network interface 80.Optionally, the device 800 further include input equipment 840, output equipment 850 and bus 860.Wherein, processor 80, memory 820, network interface 830 And input equipment 840, output equipment 860 are connected with each other by bus 850.Store instruction or program in memory 820, processing Device 810 is for executing the instruction or program stored in memory 820.The instruction or program stored in memory 820 is performed, The processor 810 is used to execute the operation that processing module 720 executes in above method embodiment, and network interface 830 is for executing The operation that transceiver module 710 executes in above-described embodiment.

It should be noted that device 700 or 800 provided by the embodiments of the present application can correspond to it is provided in an embodiment of the present invention Meta data server of the metadata processing method S501 into S505, and the behaviour of device 700 or the modules in device 800 Make and/or function is respectively in order to realize the corresponding process of method shown in Fig. 5, for sake of simplicity, details are not described herein.

It should be understood that the processor referred in the embodiment of the present application can be central processing unit (central Processing unit, CPU), it can also be other general processors, digital signal processor (digital signal Processor, DSP), it is specific integrated circuit (application specific integrated circuit, ASIC), existing At programmable gate array (field programmable gate array, FPGA) or other programmable logic device, discrete Door or transistor logic, discrete hardware components etc..General processor can be microprocessor or the processor can also To be any conventional processor etc..

It should also be understood that the memory referred in the embodiment of the present application can be volatile memory or non-volatile memories Device, or may include both volatile and non-volatile memories.Wherein, nonvolatile memory can be read-only memory (read-only memory, ROM), programmable read only memory (programmable ROM, PROM), erasable programmable are only Read memory (erasable PROM, EPROM), electrically erasable programmable read-only memory (electrically EPROM, ) or flash memory EEPROM.Volatile memory can be random access memory (random access memory, RAM), use Make External Cache.By exemplary but be not restricted explanation, the RAM of many forms is available, such as static random-access Memory (static RAM, SRAM), dynamic random access memory (dynamic RAM, DRAM), synchronous dynamic random-access Memory (synchronous DRAM, SDRAM), double data speed synchronous dynamic RAM (double data Rate SDRAM, DDR SDRAM), it is enhanced Synchronous Dynamic Random Access Memory (enhanced SDRAM, ESDRAM), same Step connection dynamic random access memory (synchlink DRAM, SLDRAM) and direct rambus random access memory (direct rambus RAM, DR RAM).

It should be noted that when processor is general processor, DSP, ASIC, FPGA or other programmable logic devices When part, discrete gate or transistor logic, discrete hardware components, memory (memory module) is integrated in the processor.

It should be noted that memory described herein is intended to include but is not limited to the storage of these and any other suitable type Device.

It should be understood that magnitude of the sequence numbers of the above procedures are not meant to execute suitable in the various embodiments of the application Sequence it is successive, the execution of each process sequence should be determined by its function and internal logic, the implementation without coping with the embodiment of the present invention Process constitutes any restriction.

Those of ordinary skill in the art may be aware that list described in conjunction with the examples disclosed in the embodiments of the present disclosure Member and algorithm steps can be realized with the combination of electronic hardware or computer software and electronic hardware.These functions are actually It is implemented in hardware or software, the specific application and design constraint depending on technical solution.Professional technician Each specific application can be used different methods to achieve the described function, but this realization is it is not considered that exceed Scope of the present application.

It is apparent to those skilled in the art that for convenience and simplicity of description, the system of foregoing description, The specific work process of device and unit, can refer to corresponding processes in the foregoing method embodiment, and details are not described herein.

In several embodiments provided herein, it should be understood that disclosed systems, devices and methods, it can be with It realizes by another way.For example, the apparatus embodiments described above are merely exemplary, for example, the unit It divides, only a kind of logical function partition, there may be another division manner in actual implementation, such as multiple units or components It can be combined or can be integrated into another system, or some features can be ignored or not executed.Another point, it is shown or The mutual coupling, direct-coupling or communication connection discussed can be through some interfaces, the indirect coupling of device or unit It closes or communicates to connect, can be electrical property, mechanical or other forms.

The unit as illustrated by the separation member may or may not be physically separated, aobvious as unit The component shown may or may not be physical unit, it can and it is in one place, or may be distributed over multiple In network unit.It can select some or all of unit therein according to the actual needs to realize the mesh of this embodiment scheme 's.

It, can also be in addition, each functional unit in each embodiment of the application can integrate in one processing unit It is that each unit physically exists alone, can also be integrated in one unit with two or more units.

It, can be with if the function is realized in the form of SFU software functional unit and when sold or used as an independent product It is stored in a computer readable storage medium.Based on this understanding, the technical solution of the application is substantially in other words The part of the part that contributes to existing technology or the technical solution can be embodied in the form of software products, the meter Calculation machine software product is stored in a storage medium, including some instructions are used so that a computer equipment (can be a People's computer, server or network equipment etc.) execute each embodiment the method for the application all or part of the steps. And storage medium above-mentioned includes: that USB flash disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), arbitrary access are deposited The various media that can store program code such as reservoir (Random Access Memory, RAM), magnetic or disk.

The above, the only specific embodiment of the application, but the protection scope of the application is not limited thereto, it is any Those familiar with the art within the technical scope of the present application, can easily think of the change or the replacement, and should all contain Lid is within the scope of protection of this application.Therefore, the protection scope of the application shall be subject to the protection scope of the claim.

Claims (22)

1. a kind of method for constructing metadata cache, which is characterized in that the described method includes:

Meta data server receives the metadata request that client proxy device is sent, and includes the first mark in the metadata request Know and second identifier, the first identifier are the mark of the first catalogue, the second identifier is the subitem of first catalogue Mark；

If the corresponding cache node of first catalogue is not present in the tree-like buffer structure that the meta data server saves, and The meta data server is the ownership meta data server of the dynamic subtree where the subitem, and the meta data server exists The corresponding cache node of first catalogue is created in the tree-like buffer structure；

The meta data server reads the first meta data file stored in storage equipment according to the first identifier, and will The content of first meta data file is saved in the corresponding cache node of first catalogue, and first meta data file is The content of the meta data file of first catalogue, first meta data file includes the metadata information of the subitem.

2. the method according to claim 1, wherein the meta data server is in the tree-like buffer structure Create the corresponding cache node of first catalogue, comprising:

The meta data server obtains the subtree root of the dynamic subtree where the subitem to the mesh of first catalogue Record identification path；

The meta data server successively identifies the catalogue according to the subtree root to the sequence of first catalogue Each of path catalogue executes following processing, until the corresponding cache node creation of first catalogue is completed:

For any catalogue in the catalogue identification path, if the corresponding caching of the catalogue is not present in the tree-like buffer structure Node then creates the corresponding cache node of the catalogue in the tree-like buffer structure；

After the meta data server creates the corresponding cache node of first catalogue in the tree-like buffer structure, also Include:

The meta data server creates the corresponding cache node of the subitem in the tree-like buffer structure.

3. according to the method described in claim 2, it is characterized in that, the meta data server is by first meta data file Content be saved in after the corresponding cache node of first catalogue, further includes:

The meta data server is that itself insincere label is arranged in the corresponding cache node of first catalogue and subitem is credible Itself fiducial markers and the insincere label of subitem are arranged for the corresponding cache node of the subitem in label, are the subtree root mesh It is insincere that itself is arranged in the corresponding cache node of record newly created other catalogues into the catalogue identification path of first catalogue Label and the insincere label of subitem.

4. the method according to claim 1, wherein meta data server receives what client proxy device was sent After metadata request, the method also includes:

If there are the corresponding cache node of first catalogue in the tree-like buffer structure that the meta data server saves, and institute It states the corresponding cache node of the first catalogue and is equipped with the insincere label of subitem, the meta data server is according to first mark Know, reads the first meta data file stored in the storage equipment, and the content of first meta data file is saved in The corresponding cache node of first catalogue；

The meta data server removes the insincere label of the subitem of the corresponding cache node of first catalogue, and is described Subitem fiducial markers are arranged in the corresponding cache node of first catalogue.

5. according to the method described in claim 4, it is characterized in that, being deposited in the meta data server reading storage equipment Before first meta data file of storage, further includes:

If the corresponding cache node of the subitem, the member are not present in the tree-like buffer structure that the meta data server saves Data server creates the corresponding cache node of the subitem in the tree-like buffer structure；

The content of first meta data file is saved in the corresponding caching of first catalogue and saved by the meta data server After point, further includes:

The meta data server is that itself fiducial markers and the insincere label of subitem are arranged in the corresponding cache node of the subitem.

6. the method according to any one of claims 1 to 5, which is characterized in that the meta data server is in the tree Before creating the corresponding cache node of first catalogue in shape buffer structure, further includes:

The birth that the meta data server obtains first catalogue saved in birth record server records, and described first It include the protection level information of first catalogue in the birth record of catalogue；

The meta data server reads the first meta data file stored in storage equipment, comprising:

The meta data server determines described first according to the protection level information of the first identifier and first catalogue The storage location of meta data file in the storage device reads first meta data file from the storage location.

7. according to the method described in claim 6, it is characterized in that, further including file manipulation command in the metadata request；

The content of first meta data file is saved in the corresponding caching of first catalogue and saved by the meta data server After point, further includes:

The meta data server is according to the file manipulation command, described in saving in the cache node of first catalogue The metadata information of subitem is handled.

8. the method according to claim 1, wherein further including file manipulation command in the metadata request；

After the meta data server receives the metadata request that client proxy device is sent, the method also includes:

If there are the corresponding cache node of first catalogue in the tree-like buffer structure that the meta data server saves, and institute It states the corresponding cache node of the first catalogue and is equipped with subitem fiducial markers, the meta data server is according to the file operation Order, is directly handled the metadata information of the subitem saved in the cache node of first catalogue.

9. according to the method described in claim 8, it is characterized in that, the meta data server is to the caching of first catalogue Before the metadata information of the subitem saved in node is handled, further includes:

If the corresponding cache node of the subitem, the member are not present in the tree-like buffer structure that the meta data server saves Data server creates the corresponding cache node of the subitem in the tree-like buffer structure, and corresponding slow for the subitem It deposits node and itself fiducial markers and the insincere label of subitem is set.

10. method according to claim 7 or 8, which is characterized in that the meta data server is to first catalogue After the metadata information of the subitem saved in cache node is handled, further includes:

The metadata information of treated the subitem is sent to the storage equipment by the meta data server.

11. a kind of device for constructing metadata cache, which is characterized in that described device includes:

Transceiver module includes first in the metadata request for receiving the metadata request of client proxy device transmission Mark and second identifier, the first identifier are the mark of the first catalogue, and the second identifier is the subitem of first catalogue Mark；

Processing module, for the corresponding caching section of first catalogue to be not present in the tree-like buffer structure that the present apparatus saves Point, and in the case where the ownership meta data server of dynamic subtree of the present apparatus where the subitem, in the tree-like caching The corresponding cache node of first catalogue is created in structure；

The processing module is also used to read the first meta data file stored in storage equipment according to the first identifier, and The content of first meta data file is saved in the corresponding cache node of first catalogue, first meta data file Content for the meta data file of first catalogue, first meta data file includes the metadata information of the subitem.

12. device according to claim 11, which is characterized in that the processing module is specifically used for:

The subtree root of the dynamic subtree where the subitem is obtained to the catalogue identification path of first catalogue；

According to the subtree root to the sequence of first catalogue, successively to each of catalogue identification path mesh Record executes following processing, until the corresponding cache node creation of first catalogue is completed:

For any catalogue in the catalogue identification path, if the corresponding caching of the catalogue is not present in the tree-like buffer structure Node then creates the corresponding cache node of the catalogue in the tree-like buffer structure；

After the processing module creates the corresponding cache node of first catalogue in the tree-like buffer structure, also wrap It includes:

The corresponding cache node of the subitem is created in the tree-like buffer structure.

13. device according to claim 12, which is characterized in that the processing module is by first meta data file Content is saved in after the corresponding cache node of first catalogue, further includes:

Itself insincere label and subitem fiducial markers are set for the corresponding cache node of first catalogue, are the subitem pair Itself fiducial markers and the insincere label of subitem are arranged in the cache node answered, for the subtree root to first catalogue Itself insincere label is arranged in the corresponding cache node of newly created other catalogues in catalogue identification path and subitem can not beacon Note.

14. device according to claim 11, which is characterized in that receive client proxy device hair in the transceiver module After the metadata request sent, the processing module is also used to:

There are the corresponding cache node of first catalogue in the tree-like buffer structure that the present apparatus saves, and first catalogue In the case that corresponding cache node is equipped with the insincere label of subitem, according to the first identifier, reads the storage and set First meta data file of standby middle storage, and it is corresponding that the content of first meta data file is saved in first catalogue Cache node；

The insincere label of the subitem of the corresponding cache node of first catalogue is removed, and corresponding slow for first catalogue Deposit node setting subitem fiducial markers.

15. device according to claim 14, which is characterized in that the processing module is deposited in reading the storage equipment Before first meta data file of storage, it is also used to:

In the tree-like buffer structure that the present apparatus saves in the case where cache node corresponding there is no the subitem, the processing Module creates the corresponding cache node of the subitem in the tree-like buffer structure；

The content of first meta data file is being saved in the corresponding cache node of first catalogue by the processing module Later, it is also used to:

For the corresponding cache node of the subitem, itself fiducial markers and the insincere label of subitem are set.

16. device described in any one of 1 to 15 according to claim 1, which is characterized in that the processing module is described tree-like Before creating the corresponding cache node of first catalogue in buffer structure, it is also used to:

The birth for obtaining first catalogue saved in birth record server records, in the birth record of first catalogue Protection level information including first catalogue；

The processing module reads the first meta data file stored in storage equipment, comprising:

According to the protection level information of the first identifier and first catalogue, determine first meta data file described The storage location in equipment is stored, reads first meta data file from the storage location.

17. device according to claim 16, which is characterized in that further include file operation life in the metadata request It enables；

The content of first meta data file is being saved in the corresponding cache node of first catalogue by the processing module Later, it is also used to:

Metadata information according to the file manipulation command, to the subitem saved in the cache node of first catalogue It is handled.

18. device according to claim 11, which is characterized in that further include file operation life in the metadata request It enables；

After the transceiver module receives the metadata request that client proxy device is sent, the processing module is also used to:

There are the corresponding cache node of first catalogue in the tree-like buffer structure that the present apparatus saves, and first catalogue In the case that corresponding cache node is equipped with subitem fiducial markers, according to the file manipulation command, directly to described The metadata information of the subitem saved in the cache node of one catalogue is handled.

19. device according to claim 18, which is characterized in that the processing module is in the caching to first catalogue Before the metadata information of the subitem saved in node is handled, it is also used to:

In the tree-like buffer structure that the present apparatus saves in the case where cache node corresponding there is no the subitem, in the tree Create the corresponding cache node of the subitem in shape buffer structure, and it is credible for the corresponding cache node of the subitem itself to be arranged Label and the insincere label of subitem.

20. device described in 7 or 18 according to claim 1, which is characterized in that the processing module delays first catalogue Deposit the subitem saved in node metadata information handled after, the transceiver module is also used to:

The metadata information of treated the subitem is sent to the storage equipment.

21. a kind of device for constructing metadata cache, which is characterized in that described at least one including at least one processor Reason device is coupled at least one processor:

At least one described processor, for executing the computer program stored in at least one processor or instruction, with So that described device executes the method as described in any one of claims 1 to 10.

22. a kind of computer readable storage medium, which is characterized in that be stored with computer in the computer readable storage medium Program or instruction, when computer reads and execute the computer program or instruction, so that computer executes such as claim 1 To method described in any one of 10.