CN106446197B - A kind of date storage method, apparatus and system - Google Patents
A kind of date storage method, apparatus and system Download PDFInfo
- Publication number
- CN106446197B CN106446197B CN201610866797.2A CN201610866797A CN106446197B CN 106446197 B CN106446197 B CN 106446197B CN 201610866797 A CN201610866797 A CN 201610866797A CN 106446197 B CN106446197 B CN 106446197B
- Authority
- CN
- China
- Prior art keywords
- subtree
- back end
- path
- node
- ownership
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/13—File access structures, e.g. distributed indices
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/182—Distributed file systems
Abstract
The embodiment of the invention discloses a kind of date storage methods, apparatus and system, wherein the described method includes: determining the subtree path in distributed file system；The node-home information on the subtree path is obtained, the node-home information indicates the ownership back end of link information and each subtree between the subtree on the subtree path；Store the node-home information on the subtree path.Using the embodiment of the present invention, the problem of subtree adapter tube time length when being able to solve MDS failure, and reduce the coherence request to the subtree information of MDS.
Description
Technical field
The present invention relates to field of communication technology more particularly to a kind of date storage methods, apparatus and system.
Background technique
Distributed file system (Distributed File System), which refers to, will be fixed on some file of some node
System, expands to any number of nodes/multiple file system, is connected to form cluster by numerous nodes, thus the one of composition
A Filesystem Network.Generally include following three types of node in system: protocol server (Protocol Server),
Meta data server (Meta Data Server, abbreviation MDS), data server (Data Server, abbreviation DS).The distribution
Formula file system divides there are many modes of Metadata Service, such as the mode based on dynamic subtree.Based on dynamic subtree
Under mode, each node maintenance one or more subtree is responsible for a part of Metadata Service.
Currently, needing to select a normal MDS (i.e. adapter tube person) when some MDS failure in distributed file system
The subtree for belonging to failure MDS adapter tube is come, to continue to provide Metadata Service, this process is known as subtree adapter tube or son
Tree restores.The process of subtree adapter tube is as follows: all nodes of the adapter tube person into system collect subtree routing information and attaching relation,
And duplicate removal is carried out to the subtree information of collection, the subtree information on failure MDS is regenerated, i.e., failure MDS is rebuild.It should
Under mode, since adapter tube person needs all MDS into system to send request, so that cluster is larger during adapter tube, node
When messaging amount is larger, it is be easy to cause "bottleneck", causes the adapter tube time longer.Moreover, which requires the subtree of each MDS
Information has higher consistency, and the algorithm for reconstructing for the person that is easy to cause adapter tube if the subtree information of each MDS is inconsistent is complicated.
Summary of the invention
The embodiment of the present invention provides a kind of date storage method, apparatus and system, son when being able to solve back end failure
The problem of tree adapter tube time length, and reduce the coherence request to the subtree information of back end.
In a first aspect, the embodiment of the invention provides a kind of date storage methods, comprising:
It determines the subtree path in distributed file system, obtains the node-home information on the subtree path, and store
Node-home information on the subtree path.
Wherein, which may include the root node (subtree of the subtree in distributed file system in each back end
Root node, referred to as " subtree root ") and each back end subtree root node to the path of the root node of the system on son
Tree node.The node-home information indicates the ownership of link information and each subtree between the subtree on the subtree path
Back end.So as to carry out the operation such as subtree migration or subtree adapter tube based on the node-home information of the storage, reduce
To the coherence request of subtree information in system.
In an alternate embodiment of the invention, the node-home information on the storage subtree path, can be with specifically: generates
Subtree path including the node-home information on subtree path belongs to table, and stores the subtree path ownership table.It such as can should
Subtree path belongs to table write-in disk and carries out persistent storage.
Wherein, the link information between the subtree on the subtree path includes the rope of each children tree nodes on the subtree path
Draw node number, each children tree nodes a upper children tree nodes inode number.Optionally, in subtree path ownership table also
It may include the subtree root node information etc. on the subtree path.
In an alternate embodiment of the invention, the target subtree that Radix Angelicae Sinensis belongs to the first back end needs to move to the second back end
When, the inode number of the target subtree can be obtained；The rope with the target subtree is found out from the subtree path ownership table
Draw the identical inode number of node number；By the corresponding ownership of the inode number found out in the subtree path ownership table
Back end is updated to second back end.Wherein, the inode number of the target subtree can refer specifically to target
The inode number of the subtree root node of tree.So as to the node-home information realization subtree migration based on the storage.
In an alternate embodiment of the invention, it also can receive the subtree takeover request that instruction third back end breaks down；Response
The subtree takeover request, the ownership back end that the subtree of third back end will be belonged in the subtree path ownership table update
For the 4th back end；It is returned to the 4th back end and belongs to the third number in the subtree path ownership table before updating
According to the subtree information of node, so that the 4th back end carries out subtree weight in the data buffer storage of the 4th back end
It builds.Make it possible to the node-home information based on the storage and carry out subtree adapter tube, thus when solving back end such as MDS failure
Subtree takes over time long problem.
Second aspect, the embodiment of the invention also provides a kind of data-storage systems, comprising: the first back end and in
Heart node；Wherein,
First back end, for providing data service for the subtree of first back end；
Central node, for determining the subtree path in the distributed file system where first back end；It obtains
Node-home information on the subtree path, and store the node-home information on the subtree path.
Wherein, which may include the root node of the subtree in distributed file system in each back end and each
Children tree nodes on the root node of the subtree of back end to the path of the root node of the system.The node-home information indicates
The ownership back end of the link information and each subtree between subtree on the subtree path.Optionally, the centromere
Point can be any data node such as MDS in the distributed file system, i.e., a kind of identity is superimposed on back end, according to
So have the function of providing Metadata Service；Or the node to be separately provided in the system, the embodiment of the present invention is without limitation.
In an alternate embodiment of the invention, which can also be used to generate the node-home letter including on the subtree path
The subtree path of breath belongs to table, and stores the subtree path ownership table.
Wherein, the link information between the subtree on the subtree path may include each children tree nodes on the subtree path
Inode number, each children tree nodes a upper children tree nodes inode number.Optionally, in subtree path ownership table
It may also include subtree root node information on the subtree path, children tree nodes title associated with the inode number etc..
Further, the system may also include that the second back end；Wherein,
First back end, it may also be used for send announcing removal message to central node, which refers to
The second back end for needing the inode number of the target subtree migrated and needs to move to is shown；
The central node is also used to find out the inode number with the target subtree from the subtree path ownership table
Identical inode number；More by the corresponding ownership back end of the inode number found out in the subtree path ownership table
It is newly second back end.
Wherein, which is the back end for needing to carry out subtree migration, which is to determine
The back end that needs out move to.Optionally, the second back end which moves to can be according to preset rules
It selects, such as the back end that temperature is minimum, is also possible to be randomly selected, the embodiment of the present invention is without limitation.To
It can be migrated based on the node-home information realization subtree of the storage.
Further, second back end, it may also be used for building data buffer storage, and by target described in the caching
The ownership back end of subtree is updated to second back end；
First back end is also used to returning target subtree described in the data buffer storage of first back end
Belong to back end and is updated to second back end.
That is, second back end can also construct data buffer storage, after having updated the subtree path ownership table,
The ownership back end of this in caching target subtree can be changed to oneself by the second back end, first back end
The ownership back end of the target subtree in caching is changed to second back end.In order to rapidly extracting subtree information.
In an alternate embodiment of the invention, the system can include: third back end and the 4th back end, the third data
Node be break down back end, the 4th back end be determine adapter tube the third back end subtree number
According to node；Wherein,
4th back end, for sending the son for being used to indicate the third back end and breaking down to central node
Set takeover request；
The central node will belong to third number in the subtree path ownership table for receiving the subtree takeover request
The 4th back end is updated to according to the ownership back end of the subtree of node；And before to the 4th back end returning to the update
The subtree information of the third back end is belonged in the subtree path ownership table；
4th back end is also used to the subtree information based on the third back end in the 4th back end
Data buffer storage in carry out subtree reconstruction.
Wherein, the central node, the first back end, the second back end, third back end, the 4th back end etc.
It can be between any two same back end, or be different nodes, the embodiment of the present invention is without limitation.So as to be based on
The node-home information of the storage carries out subtree migration and subtree adapter tube, subtree when thus solving back end such as MDS failure
The problem of time length is taken over, and reduces the coherence request to the subtree information in system.
The third aspect, the embodiment of the invention also provides a kind of data storage devices, comprising: path determination module, first
The data storage side of above-mentioned first aspect can be realized by above-mentioned module by obtaining module and memory module, the data storage device
The step of some or all of method.
Fourth aspect, the embodiment of the invention also provides a kind of computer storage medium, the computer storage medium is deposited
Program is contained, described program includes the steps that some or all of the date storage method of above-mentioned first aspect when executing.
5th aspect, the embodiment of the invention also provides a kind of data servers, comprising: memory and processor, it is described
Memory is connected to the processor；Wherein,
The memory is used for storage driving software；
The processor is used to read the drive software from the memory and hold under the action of the drive software
Row:
Determine the subtree path in distributed file system；
The node-home information on the subtree path is obtained, the node-home information indicates on the subtree path
Subtree between link information and each subtree ownership back end；
Store the node-home information on the subtree path.
Optionally, the processor is also used to read the drive software from the memory and in the drive software
The step of some or all of date storage method of the lower above-mentioned first aspect of execution of effect.
The implementation of the embodiments of the present invention has the following beneficial effects:
In the present invention is implemented, can be by the subtree path in detection distributed file system, obtaining includes the subtree road
The node-home information of the ownership back end of the link information and each subtree between subtree on diameter, and store the node
Attaching information makes it possible to the adapter tube stream when node-home information based on the storage carries out the migration and subtree failure between subtree
Journey to solve the problems, such as that the subtree adapter tube time is long when back end failure, and reduces the subtree information to back end
Coherence request.
Detailed description of the invention
In order to more clearly explain the embodiment of the invention or the technical proposal in the existing technology, to embodiment or will show below
There is attached drawing needed in technical description to be briefly described, it should be apparent that, the accompanying drawings in the following description is only this
Some embodiments of invention for those of ordinary skill in the art without creative efforts, can be with
It obtains other drawings based on these drawings.
Fig. 1 is a kind of architecture diagram of distributed file system provided in an embodiment of the present invention；
Fig. 2 is a kind of flow diagram of date storage method provided in an embodiment of the present invention；
Fig. 3 is seed tree grouping schematic diagram provided in an embodiment of the present invention；
Fig. 4 is a seed tree provided in an embodiment of the present invention path schematic diagram；
Fig. 5 is a kind of interaction schematic diagram of date storage method provided in an embodiment of the present invention；
Fig. 6 is seed tree migration schematic diagram provided in an embodiment of the present invention；
Fig. 7 is another seed tree path schematic diagram provided in an embodiment of the present invention；
Fig. 8 is the interaction schematic diagram of another date storage method provided in an embodiment of the present invention；
Fig. 9 is another seed tree path schematic diagram provided in an embodiment of the present invention；
Figure 10 is a kind of structural schematic diagram of data storage device provided in an embodiment of the present invention；
Figure 11 is a kind of structural schematic diagram of data-storage system provided in an embodiment of the present invention；
Figure 12 is a kind of structural schematic diagram of data server of the embodiment of the present invention.
Specific embodiment
Following will be combined with the drawings in the embodiments of the present invention, and technical solution in the embodiment of the present invention carries out clear, complete
Site preparation description, it is clear that described embodiments are only a part of the embodiments of the present invention, instead of all the embodiments.It is based on
Embodiment in the present invention, it is obtained by those of ordinary skill in the art without making creative efforts every other
Embodiment shall fall within the protection scope of the present invention.
In distributed file system, protocol server (Protocol Server), MDS, DS three types are generally included
Node, multiple nodes are connected to form cluster.Wherein, protocol server is responsible for providing a user the Network File System of standard
(Network File System, abbreviation NFS), SMB (Server Message Block) network file service function.MDS is negative
Duty provides the relevant services of metadata such as resolution path, locating file, creation file.MDS is responsible for the data of storage file.User
Program can access file system by protocol client (Protocol client).Protocol client passes through with protocol server
Front network (FE) is connected, and each node is connected by back-end network (BE) inside file system.Front network is used for customer service
It is made requests between distributed file system and data interaction, back-end network is used for each node inside distributed file system
It is made requests between equipment and data interaction.Further, the node of the three types can be logical node, and when actual deployment can
To be deployed on same physical equipment, it can also separate and dispose, the embodiment of the present invention is without limitation.
In distributed file system, in order to reach better performance, the Metadata Service of whole system can be distributed to respectively
On a MDS (or DS), each MDS (or DS) is responsible for a part of Metadata Service.Meanwhile in some or certain several MDS (or DS)
System can also continue externally to provide Metadata Service when breaking down, thus lifting system reliability.The embodiment of the present invention is with number
It is illustrated for being MDS according to node, which divides there are many modes that Metadata Service is division MDS,
For example it is based on the mode of dynamic subtree (Subtree).Specifically, system can according to the access type of business (such as getattr,
Setattr, readdir etc.), the factors such as access frequency count each file, catalogue (or catalogue fragment) and mesh in the buffer
The temperature of tree is recorded, different access types adds up different temperatures, and access frequency is higher, and temperature is higher.With the increasing of temperature
Add, so as to which whole directory tree is divided into more stalk trees, and move on different MDS, each MDS, which is responsible for providing, to be belonged to
Metadata Service in the subtree of oneself.Wherein, it only needs to cache metadata relevant to oneself, including ownership on each MDS
In the part subtree metadata (for providing Metadata Service) of oneself, the boundary point of this subtree and junior's subtree (is visited downwards
Be able to know that metadata since which place is belonged to another MDS when asking) and this subtree root to root
Between path (in node failure, can still parse traversal complete trails).
Further, with the load variation of each MDS, the variation of subtree temperature etc., subtree can be between MDS freely
Migration, and smaller subtree can be split into again, or is merged into big subtree.The division of its subtree and ownership can be with industry
The access (i.e. the load of temperature namely node) of business dynamically adjusts, rather than static.
The application is by introducing central node, to manage the subtree information in whole system concentratedly by the central node.Its
In, which (can be superimposed a kind of identity, conduct for any one MDS (or DS) node in system on MDS
MDS still has the function of providing Metadata Service), or the node to be additionally arranged, the embodiment of the present invention is without limitation.From
And solve the problems, such as that single-point size of message is greatly when subtree is taken under big cluster and the adapter tube time is long, while also simplifying adapter tube and patrolling
Volume.
The embodiment of the invention discloses a kind of date storage method, device, data server and systems, are able to solve data
The problem of subtree adapter tube time length when node failure, and reduce the coherence request to the subtree information of back end.Below
It is described in detail respectively.
Fig. 2 is referred to, Fig. 2 is a kind of flow diagram of date storage method provided in an embodiment of the present invention.Specifically,
The method of the embodiment of the present invention can be applied particularly in central node such as MDS or DS.As shown in Fig. 2, the embodiment of the present invention
The method can comprise the following steps that
101, the subtree path in distributed file system is determined.
Wherein, the subtree path of a certain subtree can refer to the root of the subtree to the path of system root directory.In the present invention
In embodiment, the subtree path of each subtree in distributed file system (hereinafter referred to as " system ") can be first determined, that is, this is
Subtree path in system may include the root of the root node of the subtree in system in each back end and the subtree of each back end
Children tree nodes on node to the path of the root node (system root directory) of system.
102, the node-home information on the subtree path is obtained, the node-home information indicates the subtree road
The ownership back end of the link information and each subtree between subtree on diameter.
Wherein, the link information between the subtree on the subtree path may include each children tree nodes inode number,
The information such as the inode number of a upper children tree nodes of each children tree nodes.The inode number is that the mark of children tree nodes is believed
Breath, each inode number uniquely determine a children tree nodes (such as catalogue, file) in the distributed file system.
103, the node-home information on the subtree path is stored.
Optionally, the node-home information on the storage subtree path, can be with specifically: generating includes the son
The subtree path for setting the node-home information on path belongs to table, and stores the subtree path ownership table.That is, the subtree road
It includes that all subtrees are to the subtree path of system root node in system that diameter, which belongs in table, and optionally, which belongs in table
It may also include subtree root node information on subtree path, title of children tree nodes associated with the inode number etc..
Fig. 3 is referred to, Fig. 3 is seed tree grouping schematic diagram provided in an embodiment of the present invention.As shown in figure 3, distributed
Include 3 back end in file system, i.e. MDS1, MDS2 and MDS3, and be divided into 4 subtrees, including subtree A, B, C and
D.Wherein, the children tree nodes in subtree A (/, var, lib, etc, usr, lib, lib.so.6) belong to MDS1, in subtree B
Children tree nodes (such as vim72, plugin) in children tree nodes (log, messages, news, news.err), subtree D belong to
Children tree nodes (share, bash, helpfiles, kill, cd, vim, site) in MDS2, subtree C belong to MDS3；The "/"
For the root node in the distributed file system, i.e. system root directory；The son on subtree path in the distributed file system
Tree node be/, var, usr, log, share, vim, vim72, it is assumed that be denoted as respectively ino1, ino2, ino3, ino4, ino5,
Ino6, ino7 (certainly, can also be identified as other forms, be only used for illustrating herein), children tree nodes on the subtree path and its
Corresponding identification information ino is associated, and (such as "/" is associated with ino1, " var " is associated with ino2, " usr " is associated with ino3, " log "
It is associated with ino4, " share " is associated with ino5, " vim " is associated with ino6, " vim72 " is associated with ino7), as shown in Figure 4.Into
One step, based on the node-home information of the children tree nodes on the subtree path, produces a subtree path and belong to table, it is as follows
Shown in table one:
Table one
| ino | Dirino | subtree flag | auth |
| ino1 | - | 1 | MDS1 |
| ino2 | ino1 | 0 | MDS1 |
| ino3 | ino1 | 0 | MDS1 |
| ino4 | ino2 | 1 | MDS2 |
| ino5 | ino3 | 1 | MDS3 |
| ino6 | ino5 | 0 | MDS3 |
| ino7 | ino6 | 1 | MDS2 |
Wherein, ino is the children tree nodes on subtree path, and dirino is a upper children tree nodes (father's section of children tree nodes
Point), subtree flag shows subtree root node (referred to as " subtree root ") information, indicates that the children tree nodes are data for " 1 "
The subtree root node of node indicates the children tree nodes not and is the subtree root node of back end for " 0 "；Auth indicates children tree nodes
Ownership back end.Subtree root information subtree flag actually can be used as a flag bit, indicate certain on path
Whether a node is subtree root (namely the changed boundary of the superior and the subordinate's node-home).Optionally, which belongs to table
In can not also store the subtree root information.At this point, the subtree root information can be also reversely pushed out from attaching information, i.e., according to ino,
Dirino and auth are determined.For example, the ownership back end of ino3 is MDS1, and the ownership back end of ino4 is MDS2,
Ownership is changed, then can determine since ino4, is a stalk tree, and ino4 is the subtree root of the subtree.Further, return
Each ino recorded in category table with/, var, usr, log, share, vim, vim72 it is associated, the ino be file or directory attribute
A part, and be in systems it is unique, be equivalent to the ID of file, catalogue etc..It as a result, cannot be with subtree in ownership table
The title of node replaces ino, because, there may be the identical file of multiple titles (or catalogue), such as crying there may be multiple in system
The catalogue of " var " leads to that the children tree nodes cannot be uniquely determined.
Optionally, which can specifically be belonged to table write-in disk and carries out persistent storage.It further, can also be
The content in the subtree path ownership table is cached in memory in the form of a tree, quickly to access the subtree path, and obtains subtree
Nodal information.
It is further alternative, it, can be based on the section of the storage after storing the node-home information on the subtree path
Point attaching information carries out subtree migration.Specifically, the target subtree that Radix Angelicae Sinensis belongs to the first back end needs to move to the second number
When according to node, central node can obtain the inode number of the target subtree；It is found out from the subtree path ownership table
Inode number identical with the inode number of the target subtree；By the rope found out in the subtree path ownership table
Draw the corresponding ownership back end of node number and is updated to second back end.Optionally, subtree migration process can be
It is triggered when node temperature is excessively high, or need to carry out to trigger when the distribution adjustment of system temperature, etc.；Needs migration
To the second back end can be what any data node in the system was selected according to preset rules, as temperature is minimum
Back end is also possible to be randomly selected, and the embodiment of the present invention is without limitation.
It is further alternative, after storing the node-home information on the subtree path, when a certain back end occurs
When failure, subtree adapter tube (subtree recovery) can be carried out based on the node-home information of the storage.Specifically, receiving instruction third
The subtree takeover request that back end breaks down；The subtree takeover request is responded, will be returned in the subtree path ownership table
The ownership back end for belonging to the subtree of the third back end is updated to the 4th back end；To the 4th back end
The subtree information for belonging to the third back end before the update in the subtree path ownership table is returned to, so that the described 4th
Back end carries out subtree reconstruction in the buffer.Wherein, which is the node to break down, the 4th data
Node is the back end for the subtree for taking over the third back end determined.Optionally, the 4th of the adapter tube
Back end can be to be selected according to preset rules, such as back end that temperature is minimum, is also possible to be randomly selected,
The embodiment of the present invention is without limitation.
In the present invention is implemented, can be by the subtree path in detection distributed file system, obtaining includes the subtree road
The node-home information of the ownership back end of the link information and each subtree between subtree on diameter, and store the node
Attaching information makes it possible to the adapter tube stream when node-home information based on the storage carries out the migration and subtree failure between subtree
Journey to solve the problems, such as that the subtree adapter tube time is long when back end such as MDS failure, and reduces the subtree information to MDS
Coherence request.
Further, Fig. 5 is referred to, Fig. 5 is a kind of interaction signal of date storage method provided in an embodiment of the present invention
Figure.The embodiment of the present invention is applied to the subtree between back end and migrates scene, that is, belongs to target of the first back end
Tree needs to move to the second back end.In embodiments of the present invention, it is assumed that need the son of MDS2 (i.e. the first back end)
Tree D (i.e. target subtree) moves to MDS1 (i.e. the second back end), as shown in Figure 6.Fig. 5 is then combined, together referring to Fig. 6, originally
Inventive embodiments the method may include following steps:
201, MDS2 sends migration request to MDS1.
Optionally, which can be when i.e. temperature is excessively high for the business access amount of MDS2, such as be more than preset heat
It is triggered when threshold value.The MDS that the needs move to can be any MDS or center in the distributed file system where MDS2
What node was determined, for example according to the temperature of each MDS, the minimum MDS1 of temperature is determined as to the MDS for needing to move to.Specifically
, the metadata information of the subtree D in MDS2 can be carried in the migration request.Wherein, which includes filename, belongs to
The information such as property, size.
202, MDS1 constructs the metadata in subtree D in the buffer.
203, MDS1 replys the response message that migration is ready to complete to MDS2.
Specifically, MDS2 is by sending migration request to MDS1, notice MDS1 starts to migrate subtree D.MDS1 can basis
Metadata information in request constructs the metadata in subtree D in the buffer, and can reply a migration to MDS2 and be ready to complete
Message.
204, MDS2 sends announcing removal message to central node.
In addition, can also send an announcing removal when determining that the subtree D in MDS2 needs to migrate to central node and disappear
Breath, to notify the ownership of central node subtree D to become MDS1.
205, central node updates subtree path ownership table.
Specifically, central node after receiving the announcing removal message, can modify subtree path ownership table, it will be sub
The ownership back end of tree D is updated to MDS1 by MDS2, obtains new subtree path ownership table, and the new subtree path is returned
Category table is persisted to disk, namely is saved in disk.Wherein, which can be as shown in following table two:
Table two
| ino | dirino | subtree flag | auth |
| ino1 | - | 1 | MDS1 |
| ino2 | ino1 | 0 | MDS1 |
| ino3 | ino1 | 0 | MDS1 |
| ino4 | ino2 | 1 | MDS2 |
| ino5 | ino3 | 1 | MDS3 |
| ino6 | ino5 | 0 | MDS3 |
| ino7 | ino6 | 1 | MDS1 |
Further, it can also update the subtree path structure in caching, i.e., it is new to cache this in the form of a tree in memory
Subtree path belongs to table, as shown in fig. 7, being updated subtree path structure schematic diagram.
206, central node returns to the response message being updated successfully to MDS2.
207, MDS2 notifies MDS1 to migrate successfully.
Specifically, can return to one after central node updates subtree path ownership telogenesis function to MDS2 and be updated successfully
Response message, to inform the MDS2 ownership table to change.After MDS2 receives the response message of central node, it can be sent out to MDS1
Message is sent, to notify subtree to migrate successfully.
208, MDS1 changes the attaching information of subtree D in caching.
209, MDS1 is to the successful response message of MDS2 remigration.
210, MDS2 changes the attaching information of subtree D in caching.
Specifically, the subtree attaching information in caching can be changed to after MDS1 receives the successful notification message of migration
Oneself, and a response message can be returned to MDS2.After MDS2 receives the response message, the subtree in caching can be belonged to
Information is changed to MDS1.
Further, Fig. 8 is referred to, Fig. 8 is that the interaction of another date storage method provided in an embodiment of the present invention is shown
It is intended to.The embodiment of the present invention can be applied to subtree adapter tube scene when back end failure, i.e., when event occurs in third back end
Barrier, needs the subtree by the third back end to move to the 4th back end.In embodiments of the present invention, it is assumed that detect
MDS2 (i.e. third back end) failure, needs to move to the subtree of MDS2 MDS1 (i.e. the 4th back end).Such as Fig. 8 institute
Show, the embodiment of the present invention the method may include following steps:
301, when MDS2 failure, MDS1 sends takeover request to central node.
Specifically, when MDS2 failure, it may be determined that take over the MDS of the subtree of the MDS2 out.Optionally, the MDS2 failure feelings
Condition can be any MDS in the distributed file system where MDS2 in addition to MDS2 or central node detects.This is needed
The MDS to be taken over can be any MDS in the distributed file system where MDS2 or central node determines, such as root
According to the temperature of each MDS, the minimum MDS1 of temperature is determined as to the MDS of adapter tube.If the MDS of adapter tube is MDS1, MDS1 can be in
Heart node sends takeover request (if it is determined that the back end of adapter tube is center node, then can not send the takeover request), this connects
The identification information of MDS2 can be carried in pipe request, to inform that the central node MDS2 breaks down, MDS1 needs adapter tube MDS2's
Subtree.
302, central node updates subtree path ownership table.
303, central node returns to the subtree information of MDS2 to MDS1.
Specifically, central node after receiving the takeover request, can search subtree path ownership table, will belong to
The attaching information of all subtrees of MDS2 is changed to MDS1, obtains new subtree path ownership table, and the new subtree path is returned
Category table is persisted to disk, namely is saved in disk.Wherein, which can be as shown in following table three:
Table three
| ino | dirino | subtree flag | auth |
| ino1 | - | 1 | MDS1 |
| ino2 | ino1 | 0 | MDS1 |
| ino3 | ino1 | 0 | MDS1 |
| ino4 | ino2 | 1 | MDS1 |
| ino5 | ino3 | 1 | MDS3 |
| ino6 | ino5 | 0 | MDS3 |
| ino7 | ino6 | 1 | MDS1 |
Further, it can also update the subtree path structure in caching, cache the new son in the form of a tree in memory
It sets path and belongs to table, as shown in figure 9, being updated subtree path structure schematic diagram.
Further, central node can will also belong to all subtrees of MDS2 after updating subtree path ownership table
Information returns to MDS1.
304, MDS1 rebuilds the subtree information of MDS2 in the buffer.
Specifically, the subtree for originally belonging to MDS2 can cached after MDS1 receives the subtree information of MDS2
In reconstruct come.What it is due to subtree path ownership table record is the subtree information (only having recorded ino) simplified, and MDS4 is rebuild
Metadata cache out is also incomplete, referred to as empty type.The metadata of this type can be triggered by business and be read
Disk, so that gradually filling is complete.As a result, when some MDS node failure, adapter tube person directly can obtain failure from central node
The subtree information of MDS is requested without sending to all MDS, and this reduces when message transmission quantity and MDS failure in system
Subtree takes over the time, and reduces the coherence request to the subtree information of MDS.
It is further alternative, if the MDS of failure is center node, it may further determine that out new central node, such as determine
New central node be MDS1.Then MDS1 can read subtree path ownership table (because the subtree path belongs to from low layer storage
Table is stored in disk), and the subtree information of whole system can be reconstructed in the buffer.Optionally, the event of the central node
Barrier is it may is that any MDS is detected in system.The MDS of needs adapter tube is also possible to any MDS in system and determines
Out, it does not repeat herein.Further, it needs to take over the presence or absence of subtree on the central node of MDS1 also detectable failure, if
In the presence of can then be searched from the ownership table of subtree path and belong to the subtree of central node, and be belonged to and be changed to MDS1, obtained new
Subtree path belong to table, and the persistence subtree path ownership table of the new subtree path ownership table to disk, before replacing.
In embodiments of the present invention, attaching relation and the subtree road of subtree and MDS can be managed concentratedly by introducing central point
Diameter information, and being persisted in disk, so that when back end (including MDS or central node) failure in system, Neng Goutong
Reading persistence information is crossed to rebuild subtree path ownership table, without to all MDS send request obtained subtree information into
Row subtree is rebuild, to reduce the message bursts amount on hosting node, subtree takes over the time when reducing MDS failure, so that
The adapter tube time is not influenced by cluster scale, and switching time generally can be controlled in 5s.In addition, leading to when carrying out subtree adapter tube
Cross the subtree path ownership table being subject on central node reduces simultaneously so that reducing the complexity for rebuilding subtree information
To the coherence request of the subtree information on each MDS.
0, Figure 10 is a kind of structural schematic diagram of data storage device provided in an embodiment of the present invention referring to Figure 1.Specifically
, as shown in Figure 10, the data storage device of the embodiment of the present invention may include that path determination module 11, first obtains module
12 and memory module 13.Wherein,
The path determination module 11, for determining the subtree path in distributed file system.
Wherein, which may include the root node of the subtree in distributed file system in each back end and each
Children tree nodes on the root node of the subtree of back end to the path of the root node of the system.
Described first obtains module 12, for obtaining the node-home information on the subtree path.
Wherein, which indicates the link information between the subtree on the subtree path and each height
The ownership back end of tree.
The memory module 13, for storing the node-home information on the subtree path.
Optionally, in embodiments of the present invention, the memory module 13 can be specifically used for:
It generates the subtree path including the node-home information on the subtree path and belongs to table, and store the subtree road
Diameter belongs to table.
Wherein, the link information between the subtree on the subtree path includes the rope of each children tree nodes on the subtree path
Draw node number, each children tree nodes a upper children tree nodes inode number.Optionally, in subtree path ownership table also
It may include the subtree root node information etc. on the subtree path.
Further alternative, described device may also include that
Second obtains module, and the target subtree for belonging to the first back end for Radix Angelicae Sinensis needs to move to the second back end
When, obtain the inode number of the target subtree；
Searching module, for finding out the inode number phase with the target subtree from the subtree path ownership table
Same inode number；
First update module, for by the corresponding ownership of the inode number found out in the subtree path ownership table
Back end is updated to second back end.
Wherein, which is the back end for needing to carry out subtree migration, which is to determine
The back end that needs out move to.Optionally, the second back end which moves to can be according to preset rules
It selects, such as the back end that temperature is minimum, is also possible to be randomly selected, the embodiment of the present invention is without limitation.To
It can be migrated based on the node-home information realization subtree of the storage.
Further alternative, described device may also include that
Receiving module, the subtree takeover request to break down for receiving instruction third back end；
Second update module will belong to institute in the subtree path ownership table for responding the subtree takeover request
The ownership back end for stating the subtree of third back end is updated to the 4th back end；
Sending module, for returning to the subtree information for belonging to the third back end to the 4th back end,
So that the 4th back end carries out subtree reconstruction in the data buffer storage of the 4th back end.
Wherein, the central node, the first back end, the second back end, third back end, the 4th back end etc.
It can be between any two same back end, or be different nodes, the embodiment of the present invention is without limitation.So as to be based on
The node-home information of the storage carries out subtree migration and subtree adapter tube.
In the present invention is implemented, can be by the subtree path in detection distributed file system, obtaining includes the subtree road
The node-home information of the ownership back end of the link information and each subtree between subtree on diameter, and store the node
Attaching information makes it possible to the adapter tube stream when node-home information based on the storage carries out the migration and subtree failure between subtree
Journey to solve the problems, such as that the subtree adapter tube time is long when back end failure, and reduces the subtree information to back end
Coherence request.
1, Figure 11 is a kind of structural schematic diagram of data-storage system provided in an embodiment of the present invention referring to Figure 1.Specifically
, as shown in figure 11, the data-storage system of the embodiment of the present invention may include the first back end 2 and central node 1；
Wherein,
First back end 2, for providing data service for the subtree of first back end 1；
The central node 1, the subtree road in distributed file system for determining 2 place of the first back end
Diameter；The node-home information on the subtree path is obtained, and stores the node-home information on the subtree path.
Wherein, which indicates the link information and each subtree between the subtree on the subtree path
Ownership back end.
Optionally, in embodiments of the present invention,
The central node 1 is also used to generate the subtree path including the node-home information on the subtree path and returns
Belong to table, and stores the subtree path ownership table；
Wherein, the link information between the subtree on the subtree path includes each children tree nodes on the subtree path
Inode number, each children tree nodes a upper children tree nodes inode number.Optionally, in subtree path ownership table
It may also include the subtree root node information on the subtree path.
It is further alternative, the system also includes: the second back end 3；Wherein,
First back end 2, for sending announcing removal message to the central node 1, the announcing removal disappears
Breath indicates the second back end 3 for needing the inode number of the target subtree migrated and needs to move to；
The central node 1, for finding out the index section with the target subtree from the subtree path ownership table
The identical inode number of point number；By the corresponding ownership data of the inode number found out in the subtree path ownership table
Node updates are second back end 3.
First back end 2 is used to provide data service, second number for the subtree of first back end
It is used to provide data service for the subtree of second back end according to node 3.
It is further alternative, in embodiments of the present invention,
Second back end 3, is also used to construct data buffer storage, and by the ownership of target subtree described in the caching
Back end is updated to second back end 3；
First back end 2 is also used to target subtree described in the data buffer storage of first back end 2
Ownership back end is updated to second back end 3.
That is, second back end 3 can also construct data buffer storage, have updated the subtree path ownership table it
Afterwards, the ownership back end of the target subtree of this in caching can be changed to oneself by the second back end 3, first back end
The ownership back end of this in caching target subtree can be changed to second back end 3 by 2.In order to rapidly extracting subtree
Information.
It is further alternative, the system also includes: third back end 4 and the 4th back end 5, the third data
Node 4 is the back end to break down；Wherein,
4th back end 5 goes out for being used to indicate the third back end 4 to the central node 1 transmission
The subtree takeover request of existing failure；
The central node 1 will belong to institute in the subtree path ownership table for receiving the subtree takeover request
The ownership back end for stating the subtree of third back end 4 is updated to the 4th back end 5, and to the 4th back end 5
Return to the subtree information that the third back end 4 is belonged in the update foregoing description subtree path ownership table；
4th back end 5 is also used to the subtree information based on the third back end 4 in the 4th data
Subtree reconstruction is carried out in the data buffer storage of node.
In embodiments of the present invention, it can be in communication with each other between each back end in the distributed file system.In this
Heart node, the first back end, the second back end, third back end, the 4th back end etc. can be same between any two
One back end, or be different nodes, the embodiment of the present invention is without limitation.Specifically, the center in the embodiment of the present invention
It is corresponding real that node, the first back end, the second back end, third back end, the 4th back end can refer to above-mentioned Fig. 1-9
The associated description in example is applied, details are not described herein again.
2, Figure 12 is a kind of structural schematic diagram of data server provided in an embodiment of the present invention referring to Figure 1.Specifically,
As shown in figure 12, the data server of the embodiment of the present invention includes: communication interface 300, memory 200 and processor 100,
The processor 100 is connect with the communication interface 300 and the memory 200 respectively.The memory 200 can be high speed
RAM memory is also possible to non-labile memory (non-volatile memory), for example, at least a disk storage
Device.Data connection can be carried out by bus between the communication interface 300, memory 200 and processor 100, it can also be with
Data connection by other means.It is illustrated in the present embodiment with bus connection.Specifically, described in the embodiment of the present invention
Data server can be corresponding with central node of the above-mentioned Fig. 1 into Figure 11 corresponding embodiment, specifically can be and distribution
Back end such as MDS or DS in file system, specifically please refer to associated description of the Fig. 1 into Figure 11 corresponding embodiment.Wherein,
The memory 200 is used for storage driving software；
The processor 100 reads the drive software from the memory and holds under the action of the drive software
Row:
Determine the subtree path in distributed file system；
The node-home information on the subtree path is obtained, the node-home information indicates on the subtree path
Subtree between link information and each subtree ownership back end；
Store the node-home information on the subtree path.
Optionally, the processor 100 is executed under the action of the drive software on the storage subtree path
Node-home information, specifically execute following steps:
It generates the subtree path including the node-home information on the subtree path and belongs to table, and store the subtree road
Diameter belongs to table；
Wherein, the link information between the subtree on the subtree path includes each children tree nodes on the subtree path
Inode number, each children tree nodes a upper children tree nodes inode number.Optionally, in subtree path ownership table
It may also include the subtree root node information etc. on the subtree path.
Optionally, ownership table in subtree path can also be stored with the subtree root node information on the subtree path.
Optionally, the processor 100 is also used to execute following steps under the action of the drive software:
When the target subtree that Radix Angelicae Sinensis belongs to the first back end needs to move to the second back end, target is obtained
The inode number of tree；
Index node identical with the inode number of the target subtree is found out from the subtree path ownership table
Number；
The corresponding ownership back end of the inode number found out in the subtree path ownership table is updated to institute
State the second back end.
Optionally, the processor 100 is also used to execute following steps under the action of the drive software:
The subtree takeover request that instruction third back end breaks down is received by the communication interface 300；
The subtree takeover request is responded, the son of the third back end will be belonged in the subtree path ownership table
The ownership back end of tree is updated to the 4th back end；
The update foregoing description subtree path is returned to the 4th back end by the communication interface 300 to belong to
The subtree information of the third back end is belonged in table, so that the 4th back end is in the 4th back end
Subtree reconstruction is carried out in data buffer storage.
In the present invention is implemented, can be by the subtree path in detection distributed file system, obtaining includes the subtree road
The node-home information of the ownership back end of the link information and each subtree between subtree on diameter, and store the node
Attaching information makes it possible to the adapter tube stream when node-home information based on the storage carries out the migration and subtree failure between subtree
Journey to solve the problems, such as that the subtree adapter tube time is long when back end such as MDS failure, and reduces the subtree information to MDS
Coherence request.
In several embodiments provided by the present invention, it should be understood that disclosed device and method can pass through it
Its mode is realized.For example, the apparatus embodiments described above are merely exemplary, for example, the division of the module, only
Only a kind of logical function partition, there may be another division manner in actual implementation, such as multiple module or components can be tied
Another system is closed or is desirably integrated into, or some features can be ignored or not executed.Another point, it is shown or discussed
Mutual coupling, direct-coupling or communication connection can be through some interfaces, the INDIRECT COUPLING or logical of device or module
Letter connection can be electrical property, mechanical or other forms.
It is described this module may or may not be physically separated as illustrated by the separation member, as module
The component of display may or may not be physical module, it can and it is in one place, or may be distributed over more
On a network module.Some or all of the modules therein can be selected to realize this embodiment scheme according to the actual needs
Purpose.
It, can also be in addition, each functional module in each embodiment of the present invention can integrate in a processing module
It is that modules physically exist alone, can also be integrated in two or more modules in a module.Above-mentioned integrated mould
Block both can take the form of hardware realization, can also realize in the form of hardware adds software function module.
The above-mentioned integrated module realized in the form of software function module, can store and computer-readable deposit at one
In storage media.Above-mentioned software function module is stored in a storage medium, including some instructions are used so that a computer
It is each that equipment (can be personal computer, server or the network equipment etc.) or processor (processor) execute the present invention
The part steps of embodiment the method.And storage medium above-mentioned includes: USB flash disk, mobile hard disk, read-only memory (Read-
Only Memory, abbreviation ROM), random access memory (Random Access Memory, abbreviation RAM), magnetic disk or light
The various media that can store program code such as disk.
Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention., rather than its limitations；To the greatest extent
Pipe present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that: its according to
So be possible to modify the technical solutions described in the foregoing embodiments, or to some or all of the technical features into
Row equivalent replacement；And these are modified or replaceed, various embodiments of the present invention technology that it does not separate the essence of the corresponding technical solution
The range of scheme.
Claims (14)
1. a kind of date storage method characterized by comprising
Determine that the subtree path in distributed file system, the subtree path refer to the root node of subtree to the root node of system
Path, the subtree path includes the root node of the subtree in system in each back end and the subtree of each back end
Children tree nodes on root node to the path of the root node of system；
The node-home information on the subtree path is obtained, the node-home information indicates the son on the subtree path
The ownership back end of link information and each subtree between tree, the link information packet between subtree on the subtree path
Include the inode number of each children tree nodes on the subtree path, each children tree nodes a upper children tree nodes index
Node number；
Store the node-home information on the subtree path.
2. the method according to claim 1, wherein the node-home letter on the storage subtree path
Breath, comprising:
It generates the subtree path including the node-home information on the subtree path and belongs to table, and store the subtree path and return
Belong to table.
3. according to the method described in claim 2, it is characterized in that, the method also includes:
When the target subtree that Radix Angelicae Sinensis belongs to the first back end needs to move to the second back end, the target subtree is obtained
Inode number；
Inode number identical with the inode number of the target subtree is found out from the subtree path ownership table；
The corresponding ownership back end of the inode number found out in the subtree path ownership table is updated to described
Two back end.
4. according to the method described in claim 2, it is characterized in that, the method also includes:
Receive the subtree takeover request that instruction third back end breaks down；
The subtree takeover request is responded, the subtree of the third back end will be belonged in the subtree path ownership table
Ownership back end is updated to the 4th back end；
It is returned in the ownership table of update foregoing description subtree path to the 4th back end and belongs to the third data
The subtree information of node, so that the 4th back end carries out subtree weight in the data buffer storage of the 4th back end
It builds.
5. a kind of data-storage system characterized by comprising the first back end and central node；Wherein,
First back end, for providing data service for the subtree of first back end；
The central node, for determining the subtree path in the distributed file system where first back end；It obtains
The node-home information on the subtree path is taken, and stores the node-home information on the subtree path；
Wherein, the subtree path refers to that the path of root node of the root node of subtree to system, the subtree path include being
The root node of the subtree of the root node of subtree in system in each back end and each back end to system root node road
Children tree nodes on diameter；The node-home information indicates link information between the subtree on the subtree path and each
The ownership back end of a subtree, the link information between subtree on the subtree path includes each on the subtree path
Inode number, the inode number of a upper children tree nodes for each children tree nodes of children tree nodes.
6. system according to claim 5, which is characterized in that
The central node is also used to generate the subtree path ownership table including the node-home information on the subtree path,
And store the subtree path ownership table.
7. system according to claim 6, which is characterized in that the system also includes: the second back end；Wherein,
First back end is also used to send announcing removal message to the central node, and the announcing removal message refers to
The second back end for needing the inode number of the target subtree migrated and needs to move to is shown；
The central node is also used to find out the inode number with the target subtree from the subtree path ownership table
Identical inode number；By the corresponding ownership back end of the inode number found out in the subtree path ownership table
It is updated to second back end.
8. system according to claim 7, which is characterized in that
Second back end, is also used to construct data buffer storage, and by the ownership data of target subtree described in the caching
Node updates are second back end；
First back end is also used to the ownership number of target subtree described in the data buffer storage of first back end
It is second back end according to node updates.
9. system according to claim 6, which is characterized in that the system also includes: third back end and the 4th number
According to node, the third back end is the back end to break down, and the 4th back end is for taking over described the
The back end of the subtree of three back end；Wherein,
4th back end is used to indicate what the third back end broke down for sending to the central node
Subtree takeover request；
The central node is also used to receive the subtree takeover request, will belong in the subtree path ownership table described
The ownership back end of the subtree of third back end is updated to the 4th back end；And it is returned to the 4th back end
Return the subtree information for updating and belonging to the third back end in the ownership table of foregoing description subtree path；
4th back end is also used to the subtree information based on the third back end in the 4th back end
Subtree reconstruction is carried out in data buffer storage.
10. a kind of data storage device characterized by comprising
Path determination module, for determining that the subtree path in distributed file system, the subtree path refer to the root of subtree
Node to system root node path, the subtree path include the subtree in system in each back end root node and
Children tree nodes on the root node of the subtree of each back end to the path of the root node of system；
First obtains module, and for obtaining the node-home information on the subtree path, the node-home information is indicated
The ownership back end of the link information and each subtree between subtree on the subtree path, on the subtree path
Link information between subtree includes the inode number of each children tree nodes, each children tree nodes on the subtree path
The inode number of a upper children tree nodes；
Memory module, for storing the node-home information on the subtree path.
11. device according to claim 10, which is characterized in that the memory module is specifically used for:
It generates the subtree path including the node-home information on the subtree path and belongs to table, and store the subtree path and return
Belong to table.
12. device according to claim 11, which is characterized in that described device further include:
Second obtains module, when the target subtree for belonging to the first back end for Radix Angelicae Sinensis needs to move to the second back end,
Obtain the inode number of the target subtree；
Searching module, it is identical with the inode number of the target subtree for being found out from the subtree path ownership table
Inode number；
First update module, for by the corresponding ownership data of the inode number found out in the subtree path ownership table
Node updates are second back end.
13. device according to claim 11, which is characterized in that described device further include:
Receiving module, the subtree takeover request to break down for receiving instruction third back end；
Second update module will belong to described for responding the subtree takeover request in the subtree path ownership table
The ownership back end of the subtree of three back end is updated to the 4th back end；
Sending module is belonged to for returning in the ownership table of update foregoing description subtree path to the 4th back end
The subtree information of the third back end, so that the 4th back end is in the data buffer storage of the 4th back end
Carry out subtree reconstruction.
14. a kind of data server characterized by comprising memory and processor, the memory and the processor connect
It connects；Wherein,
The memory is used for storage driving software；
The processor is used to read the drive software from the memory and execute under the action of the drive software:
Determine that the subtree path in distributed file system, the subtree path refer to the root node of subtree to the root node of system
Path, the subtree path includes the root node of the subtree in system in each back end and the subtree of each back end
Children tree nodes on root node to the path of the root node of system；
The node-home information on the subtree path is obtained, the node-home information indicates the son on the subtree path
The ownership back end of link information and each subtree between tree, the link information packet between subtree on the subtree path
Include the inode number of each children tree nodes on the subtree path, each children tree nodes a upper children tree nodes index
Node number；
Store the node-home information on the subtree path.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610866797.2A CN106446197B (en) | 2016-09-30 | 2016-09-30 | A kind of date storage method, apparatus and system |
| PCT/CN2017/082141 WO2018058949A1 (en) | 2016-09-30 | 2017-04-27 | Data storage method, device and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610866797.2A CN106446197B (en) | 2016-09-30 | 2016-09-30 | A kind of date storage method, apparatus and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106446197A CN106446197A (en) | 2017-02-22 |
| CN106446197B true CN106446197B (en) | 2019-11-19 |
Family
ID=58171408
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610866797.2A Active CN106446197B (en) | 2016-09-30 | 2016-09-30 | A kind of date storage method, apparatus and system |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN106446197B (en) |
| WO (1) | WO2018058949A1 (en) |
Families Citing this family (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106446197B (en) * | 2016-09-30 | 2019-11-19 | 华为数字技术（成都）有限公司 | A kind of date storage method, apparatus and system |
| WO2019000388A1 (en) * | 2017-06-30 | 2019-01-03 | Microsoft Technology Licensing, Llc | Staging anchor trees for improved concurrency and performance in page range index management |
| WO2019000386A1 (en) | 2017-06-30 | 2019-01-03 | Microsoft Technology Licensing, Llc | Online schema change of range-partitioned index in distributed storage system |
| CN107798104A (en) * | 2017-10-31 | 2018-03-13 | 郑州云海信息技术有限公司 | A kind of catalog management method, device, equipment and computer-readable recording medium |
| CN108153842B (en) * | 2017-12-18 | 2021-11-26 | 青岛科技大学 | Abstract fault tree-oriented structure synthesis method |
| CN109962797B (en) * | 2017-12-23 | 2022-01-11 | 华为技术有限公司 | Storage system and method for pushing business view |
| CN112181899A (en) * | 2019-07-05 | 2021-01-05 | 中兴通讯股份有限公司 | Metadata processing method and device and computer readable storage medium |
| WO2021017655A1 (en) * | 2019-07-30 | 2021-02-04 | 华为技术有限公司 | Method, apparatus, and computing device for obtaining inode number, and storage medium |
| CN110659249A (en) * | 2019-09-25 | 2020-01-07 | 浪潮电子信息产业股份有限公司 | Metadata subtree migration method, device and equipment and readable storage medium |
| CN111125024B (en) * | 2019-11-29 | 2022-05-24 | 浪潮电子信息产业股份有限公司 | Method, device, equipment and storage medium for deleting distributed system files |
| CN111176916B (en) * | 2019-12-20 | 2023-04-07 | 国久大数据有限公司 | Data storage fault diagnosis method and system |
| CN112131223B (en) * | 2020-09-24 | 2024-02-02 | 曙光网络科技有限公司 | Traffic classification statistical method, device, computer equipment and storage medium |
Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101196929A (en) * | 2007-12-29 | 2008-06-11 | 中国科学院计算技术研究所 | Metadata management method for splitting name space |
| CN101577735A (en) * | 2009-06-24 | 2009-11-11 | 成都市华为赛门铁克科技有限公司 | Method, device and system for taking over fault metadata server |
| CN101692239A (en) * | 2009-10-19 | 2010-04-07 | 浙江大学 | Method for distributing metadata of distributed type file system |
| CN101697168A (en) * | 2009-10-22 | 2010-04-21 | 中国科学技术大学 | Method and system for dynamically managing metadata of distributed file system |
| CN103106207A (en) * | 2011-11-10 | 2013-05-15 | 中国移动通信集团公司 | Method and equipment for metadata distribution in object storing system |
| CN103150394A (en) * | 2013-03-25 | 2013-06-12 | 中国人民解放军国防科学技术大学 | Distributed file system metadata management method facing to high-performance calculation |
| CN103218175A (en) * | 2013-04-01 | 2013-07-24 | 无锡成电科大科技发展有限公司 | Multi-tenant cloud storage platform access control system |
| CN103279568A (en) * | 2013-06-18 | 2013-09-04 | 无锡紫光存储系统有限公司 | System and method for metadata management |
| CN103544322A (en) * | 2013-11-08 | 2014-01-29 | 北京邮电大学 | Hotspot metadata management method based on server cluster |
| CN103685453A (en) * | 2013-09-11 | 2014-03-26 | 华中科技大学 | A method for obtaining metadata in a cloud storage system |
| CN103688257A (en) * | 2012-11-27 | 2014-03-26 | 华为技术有限公司 | Method and device for managing metadata |
| CN103793534A (en) * | 2014-02-28 | 2014-05-14 | 苏州博纳讯动软件有限公司 | Distributed file system and implementation method for balancing storage loads and access loads of metadata |
| CN104272707A (en) * | 2012-04-27 | 2015-01-07 | 交互数字专利控股公司 | Method and apparatus for supporting proximity discovery procedures |
| CN105550371A (en) * | 2016-01-27 | 2016-05-04 | 华中科技大学 | Big data environment oriented metadata organization method and system |
Family Cites Families (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101344882B (en) * | 2007-07-10 | 2010-06-02 | 中国移动通信集团公司 | Data query method, insertion method and deletion method |
| CN102238202B (en) * | 2010-04-23 | 2014-12-03 | 华为技术有限公司 | Method and device for storing and searching index information |
| CN103020315B (en) * | 2013-01-10 | 2015-08-19 | 中国人民解放军国防科学技术大学 | A kind of mass small documents storage means based on master-salve distributed file system |
| CN105981033B (en) * | 2014-02-14 | 2019-05-07 | 慧与发展有限责任合伙企业 | Placement Strategy is distributed into set of segments |
| CN104239528B (en) * | 2014-09-19 | 2017-03-08 | 盛杰 | File storage system and file storage paths record method |
| CN105635310B (en) * | 2016-01-20 | 2019-02-26 | 杭州宏杉科技股份有限公司 | A kind of access method and device of storage resource |
| CN106446197B (en) * | 2016-09-30 | 2019-11-19 | 华为数字技术（成都）有限公司 | A kind of date storage method, apparatus and system |
-
2016
- 2016-09-30 CN CN201610866797.2A patent/CN106446197B/en active Active
-
2017
- 2017-04-27 WO PCT/CN2017/082141 patent/WO2018058949A1/en active Application Filing
Patent Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101196929A (en) * | 2007-12-29 | 2008-06-11 | 中国科学院计算技术研究所 | Metadata management method for splitting name space |
| CN101577735A (en) * | 2009-06-24 | 2009-11-11 | 成都市华为赛门铁克科技有限公司 | Method, device and system for taking over fault metadata server |
| CN101692239A (en) * | 2009-10-19 | 2010-04-07 | 浙江大学 | Method for distributing metadata of distributed type file system |
| CN101697168A (en) * | 2009-10-22 | 2010-04-21 | 中国科学技术大学 | Method and system for dynamically managing metadata of distributed file system |
| CN103106207A (en) * | 2011-11-10 | 2013-05-15 | 中国移动通信集团公司 | Method and equipment for metadata distribution in object storing system |
| CN104272707A (en) * | 2012-04-27 | 2015-01-07 | 交互数字专利控股公司 | Method and apparatus for supporting proximity discovery procedures |
| CN103688257A (en) * | 2012-11-27 | 2014-03-26 | 华为技术有限公司 | Method and device for managing metadata |
| CN103150394A (en) * | 2013-03-25 | 2013-06-12 | 中国人民解放军国防科学技术大学 | Distributed file system metadata management method facing to high-performance calculation |
| CN103218175A (en) * | 2013-04-01 | 2013-07-24 | 无锡成电科大科技发展有限公司 | Multi-tenant cloud storage platform access control system |
| CN103279568A (en) * | 2013-06-18 | 2013-09-04 | 无锡紫光存储系统有限公司 | System and method for metadata management |
| CN103685453A (en) * | 2013-09-11 | 2014-03-26 | 华中科技大学 | A method for obtaining metadata in a cloud storage system |
| CN103544322A (en) * | 2013-11-08 | 2014-01-29 | 北京邮电大学 | Hotspot metadata management method based on server cluster |
| CN103793534A (en) * | 2014-02-28 | 2014-05-14 | 苏州博纳讯动软件有限公司 | Distributed file system and implementation method for balancing storage loads and access loads of metadata |
| CN105550371A (en) * | 2016-01-27 | 2016-05-04 | 华中科技大学 | Big data environment oriented metadata organization method and system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106446197A (en) | 2017-02-22 |
| WO2018058949A1 (en) | 2018-04-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106446197B (en) | A kind of date storage method, apparatus and system | |
| JP6778795B2 (en) | Methods, devices and systems for storing data | |
| CN109241161B (en) | Meteorological data management method | |
| US10331641B2 (en) | Hash database configuration method and apparatus | |
| US9122698B2 (en) | Peer-to-peer redundant file server system and methods | |
| JP4354233B2 (en) | Backup system and method | |
| CN102708165B (en) | Document handling method in distributed file system and device | |
| CN110019004B (en) | Data processing method, device and system | |
| CN105677251B (en) | Storage system based on Redis cluster | |
| CN106462575A (en) | Design and implementation of clustered in-memory database | |
| EP3495981B1 (en) | Directory deletion method and device, and storage server | |
| JP2012256324A (en) | Data management method and hybrid data management system | |
| CN108984686A (en) | A kind of distributed file system indexing means and device merged based on log | |
| KR101078287B1 (en) | Method Recovering Data Server at the Applying Multiple Reproduce Dispersion File System and Metadata Storage and Save Method Thereof | |
| CN103944958A (en) | Wide area file system and implementation method | |
| US20150254320A1 (en) | Using colocation hints to facilitate accessing a distributed data storage system | |
| CN105550371A (en) | Big data environment oriented metadata organization method and system | |
| EP3285186B1 (en) | Methods and procedures for timestamp-based indexing of items in real-time storage | |
| US11151095B2 (en) | Cloud-scale file-based filesystem for hybrid cloud storage | |
| CN107343021A (en) | A kind of Log Administration System based on big data applied in state's net cloud | |
| CN113360456B (en) | Data archiving method, device, equipment and storage medium | |
| CN103916467A (en) | Load transfer method and system in metadata cluster | |
| CN103501319A (en) | Low-delay distributed storage system for small files | |
| CN107302569A (en) | A kind of security monitoring Data acquisition and storage method of facing cloud platform | |
| CN108540510A (en) | A kind of cloud host creation method, device and cloud service system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20220907 Address after: 611731 No. 1899, Xiyuan Avenue, high tech Zone (West District), Chengdu, Sichuan Patentee after: Chengdu Huawei Technologies Co.,Ltd. Address before: 611731 Qingshui River District, Chengdu hi tech Zone, Sichuan, China Patentee before: HUAWEI DIGITAL TECHNOLOGIES (CHENG DU) Co.,Ltd. |