CN111984592A - Method and system for delaying synchronous recovery of metadata operation authority information - Google Patents
Method and system for delaying synchronous recovery of metadata operation authority information Download PDFInfo
- Publication number
- CN111984592A CN111984592A CN202010886946.8A CN202010886946A CN111984592A CN 111984592 A CN111984592 A CN 111984592A CN 202010886946 A CN202010886946 A CN 202010886946A CN 111984592 A CN111984592 A CN 111984592A
- Authority
- CN
- China
- Prior art keywords
- state
- index node
- metadata
- metadata server
- index
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 230000001360 synchronised effect Effects 0.000 title claims abstract description 52
- 238000000034 method Methods 0.000 title claims abstract description 33
- 238000011084 recovery Methods 0.000 title claims abstract description 17
- 230000004044 response Effects 0.000 claims abstract description 14
- 230000008569 process Effects 0.000 claims abstract description 8
- 238000012545 processing Methods 0.000 claims description 12
- 230000003111 delayed effect Effects 0.000 claims description 2
- 239000000725 suspension Substances 0.000 claims description 2
- 238000010586 diagram Methods 0.000 description 3
- 230000003993 interaction Effects 0.000 description 3
- 238000006467 substitution reaction Methods 0.000 description 3
- 230000007246 mechanism Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000012217 deletion Methods 0.000 description 1
- 230000037430 deletion Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/13—File access structures, e.g. distributed indices
- G06F16/134—Distributed indices
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/182—Distributed file systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/60—Protecting data
- G06F21/62—Protecting access to data via a platform, e.g. using keys or access control rules
- G06F21/6218—Protecting access to data via a platform, e.g. using keys or access control rules to a system of files or objects, e.g. local or distributed file system or database
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2221/00—Indexing scheme relating to security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F2221/21—Indexing scheme relating to G06F21/00 and subgroups addressing additional information or applications relating to security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F2221/2141—Access rights, e.g. capability lists, access control lists, access tables, access matrices
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Databases & Information Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Health & Medical Sciences (AREA)
- Bioethics (AREA)
- General Health & Medical Sciences (AREA)
- Computer Hardware Design (AREA)
- Computer Security & Cryptography (AREA)
- Software Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention provides a method and a system for delaying synchronous recovery of metadata operation authority information, wherein the method comprises the following steps: when the metadata server fails and is restarted, the client sends reconnection information containing the number of metadata operation authorities to the metadata server; when the metadata server processes the message sent by the client, judging whether the number of metadata operation authorities is equal to 1 or not; if the number of the metadata operation authorities is equal to 1, creating an empty index node and setting the state of the created index node as an unloaded state; after the metadata server is started, judging the state of the index node when a request sent by a client side for operating the index node is received; and carrying out metadata operation authority synchronization operation according to the state of the index node. And after the metadata operation permission synchronous operation is completed, continuing to execute the request completion response of the client. The efficiency of metadata server failover is improved, and the influence on the use of the user is reduced.
Description
Technical Field
The invention relates to the technical field of metadata access permission, in particular to a method and a system for delaying synchronous recovery of metadata operation permission information.
Background
When a client accesses the file system to perform operations, the MDS (metadata server, which provides various access operations to metadata) assigns a required cap (metadata operation authority) to a client (client). The client and MDS will save the cap in memory. The structure stored in the client is key-value, the key is the number of the MDS, and the value is the authority applied from the MDS. The structure stored in the MDS is key-value, wherein the key is the number value of the client and is the authority allocated to the client.
From the above, it can be seen that the client and the MDS are in a many-to-many relationship, and when the same inode (index node, each file has a corresponding inode, and some information related to the file is contained therein) is operated, cap synchronization is ensured, otherwise, the permissions are inconsistent, which may cause serious problems such as data inconsistency.
When the MDS fails and restarts, the client sends a reconnect message to the MDS, all cap information acquired from the original MDS in the memory is sent to the MDS, and the MDS performs cache reconstruction according to the cap information and restores the cap to the state before the failure. When the client caches too much, the work takes a very long time, and during this time, the user cannot operate the file directory of the metadata on the MDS, which affects the use of the client.
Disclosure of Invention
The invention provides a method and a system for delaying synchronous recovery of metadata operation permission information, aiming at the problems that when clients have a lot of caches, MDS can carry out cache reconstruction according to the cap information, the working process of recovering the cap to the state before failure needs a long time, and in the time, a user cannot operate a file directory of metadata on the MDS, so that the use of the user is influenced.
The technical scheme of the invention is as follows:
on one hand, the technical scheme of the invention provides a method for delaying synchronous recovery of metadata operation authority information, which comprises the following steps:
when the metadata server fails and is restarted, the client sends reconnection information containing the number of metadata operation authorities to the metadata server;
when the metadata server processes the message sent by the client, judging whether the number of metadata operation authorities is equal to 1 or not;
if the number of the metadata operation authorities is equal to 1, creating an empty index node and setting the state of the created index node as an unloaded state;
after the metadata server is started, judging the state of the index node when a request sent by a client side for operating the index node is received;
if the state of the index node is the unloaded state, suspending the request, calling an interface to load index node information from a disk, and performing metadata operation permission synchronous operation on the index node after the loading is finished; restoring the metadata operation authority to the state before the fault;
if the state of the index node is the loaded unsynchronized state, performing metadata operation authority synchronization operation on the index node;
and after the metadata operation permission synchronous operation is completed, continuing to execute the request completion response of the client. The metadata operation authority delay synchronization is simply realized under the condition that the existing mechanism is reserved.
Further, the step of creating an empty inode and setting the state of the created inode to an unloaded state includes:
creating an empty index node and setting the state of the created index node to be an unloaded state;
and storing the created index nodes in the unsynchronized index node set.
Further, if the state of the index node is an unloaded state, suspending the request, calling an interface to load index node information from a disk, and performing metadata operation permission synchronization operation on the index node after the loading is completed comprises the following steps:
if the state of the index node is the unloaded state, suspending the request;
calling an interface to load index node information from a disk;
judging whether the loading is successful;
if not, continuing to execute the steps: calling an interface to load index node information from a disk;
if so, removing the created empty index node, and setting the state of the index node loaded into the state of being not synchronized;
carrying out metadata operation authority synchronization operation on the index nodes;
and after the metadata operation authority synchronization operation is completed, clearing the loaded unsynchronized state of the index node.
Further, if the state of the index node is the loaded unsynchronized state, the step of performing metadata operation permission synchronization operation on the index node comprises:
if the state of the index node is the loaded unsynchronized state, performing metadata operation authority synchronization operation on the index node;
and after the metadata operation authority synchronization operation is completed, clearing the loaded unsynchronized state of the index node. Through the steps, the metadata operation authority is ensured to be synchronous when the index nodes are processed.
Further, the method further comprises:
after the metadata server is started, judging the state of the metadata server, and if the state of the metadata server is in an operating state, executing the following steps: when a request sent by a client side for operating the index node is received, judging the state of the index node; otherwise, ending. And a certain amount of metadata operation permission synchronous operation is carried out, and after a certain time, the metadata operation permission is completely and synchronously completed, so that the influence on the request is reduced.
Further, after the metadata server is started, the step of judging the state of the metadata server specifically includes:
after the metadata server is started, acquiring the state of the metadata server;
judging whether the state of the metadata server is a running state or not;
if not, ending;
if yes, judging whether the number of the index nodes to be processed in the configuration item is larger than 0;
if not, ending;
if so, traversing the unsynchronized index node set to acquire the state of the index node;
judging whether the state of the index node is an unloaded state or a loaded unsynchronized state;
if not, ending;
if yes, performing metadata operation authority synchronization operation according to the state of the index node, and adding one to the count;
judging whether the count is greater than the number of the index nodes to be processed in the configuration item;
if yes, ending;
if not, executing the following steps: and traversing the unsynchronized index node set to acquire the state of the index node. For a small part of metadata operation authorities needing interaction between metadata servers, the existing logic processing is used, and the implementation is simple.
On the other hand, the technical scheme of the invention provides a system for delaying synchronous recovery of metadata operation authority information, which comprises a metadata server and a client;
the metadata server comprises a quantity judgment module, an index node creation module, an index node state judgment module, a synchronous operation module and a request response module;
the client is used for sending reconnection information containing metadata operation authority quantity to the metadata server when the metadata server fails and restarts;
the quantity judging module is used for judging the quantity of the metadata operation permission when processing the message sent by the client;
the index node creating module is used for creating an empty index node and setting the state of the created index node as an unloaded state if the number judging module judges that the number of the metadata operation authority is equal to 1;
the index node state judgment module is used for judging the state of the index node when a request sent by a client side for operating the index node is received after the metadata server is started;
the synchronous operation module is used for carrying out metadata operation authority synchronous operation according to the state of the index node output by the index node state judgment module; the method is specifically used for suspending a request if the state of the index node is an unloaded state, calling an interface to load index node information from a disk, and performing metadata operation permission synchronization operation on the index node after loading is completed; if the state of the index node is the loaded unsynchronized state, performing metadata operation authority synchronization operation on the index node;
and the request response module is used for continuously executing the request completion response service of the client after the metadata operation permission synchronous operation is completed. The time consumption of switching the metadata server is greatly reduced.
Further, the index node creating module comprises a creating and setting unit and a storing unit;
the creating and setting unit is used for creating an empty index node and setting the state of the created index node to be an unloaded state;
and the storage unit is used for storing the created index nodes in the unsynchronized index node set.
Furthermore, the synchronous operation module comprises a suspension unit, a loading state judgment unit, a state setting unit and a synchronization unit;
the suspending unit is used for suspending the request if the state of the index node output by the index node state judging module is an unloaded state;
the loading unit is used for calling an interface to load index node information from a disk;
a loading state judgment unit for judging whether loading is successful;
if not, continuing to execute the steps: calling an interface to load index node information from a disk;
the state setting unit is used for removing the created empty index node and setting the state of the index node loaded into the loaded unsynchronized state if the loading state judgment unit outputs that the loading is successful; the synchronization unit is also used for clearing the loaded unsynchronized state of the index node after completing the metadata operation authority synchronization operation;
and the synchronization unit is used for carrying out metadata operation authority synchronization operation on the index nodes.
Furthermore, the system also comprises a metadata server state judgment module;
and the metadata server state judgment module is used for judging the state of the metadata server after the metadata server is started, and outputting information to the index node state judgment module if the state of the metadata server is in the running state. And the timing is synchronized, so that the influence is further reduced.
According to the technical scheme, the invention has the following advantages: most of the cache in the client can not be reused in an actual use scene. When a request is used, cap synchronization is carried out, and synchronization is carried out in tick messages at fixed time, so that the influence is further reduced. Through the delayed synchronization of the cap, the efficiency of the metadata server fault switching is improved, and the influence on the use of a user is reduced.
In addition, the invention has reliable design principle, simple structure and very wide application prospect.
Therefore, compared with the prior art, the invention has prominent substantive features and remarkable progress, and the beneficial effects of the implementation are also obvious.
Drawings
In order to more clearly illustrate the embodiments or technical solutions in the prior art of the present invention, the drawings used in the description of the embodiments or prior art will be briefly described below, and it is obvious for those skilled in the art that other drawings can be obtained based on these drawings without creative efforts.
FIG. 1 is a schematic flow diagram of a method of one embodiment of the invention.
FIG. 2 is a flowchart illustrating a synchronization operation of an index node in an unloaded state according to an embodiment of the present invention.
FIG. 3 is a flow diagram of a method of one embodiment of the invention.
FIG. 4 is a schematic block diagram of a system of one embodiment of the present invention.
Detailed Description
In order to make those skilled in the art better understand the technical solution of the present invention, the technical solution in the embodiment of the present invention will be clearly and completely described below with reference to the drawings in the embodiment of the present invention, and it is obvious that the described embodiment is only a part of the embodiment of the present invention, and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention. The following explains key terms appearing in the present invention.
And (3) a client: the client sends a request to the file system to complete operations such as file addition, deletion, reading and writing;
MDS: a metadata server providing various access operations of metadata;
cap: the metadata operation authority is distributed to the client by the MDS for use, so that the client can acquire the authority of operating the file;
inode: each file has a corresponding inode, and some information related to the file is contained in the inode;
as shown in fig. 1, the technical solution of the present invention provides a method for delaying synchronous recovery of metadata operation permission information, including the following steps:
s1: when the metadata server fails and is restarted, the client sends reconnection information containing the number of metadata operation authorities to the metadata server;
s2: when the metadata server processes the message sent by the client, judging whether the number of metadata operation authorities is equal to 1 or not; that is, whether the client acquires the authority of the same index node from a plurality of metadata servers is judged;
s3: if the number of the metadata operation authorities is equal to 1, creating an empty index node and setting the state of the created index node as an unloaded state;
s4: after the metadata server is started, judging the state of the index node when a request sent by a client side for operating the index node is received;
s5: if the state of the index node is the unloaded state, suspending the request, calling an interface to load index node information from a disk, and performing metadata operation permission synchronous operation on the index node after the loading is finished; restoring the metadata operation authority to the state before the fault;
s6: if the state of the index node is the loaded unsynchronized state, performing metadata operation authority synchronization operation on the index node;
s7: and after the metadata operation permission synchronous operation is completed, continuing to execute the request completion response of the client. The metadata operation authority delay synchronization is simply realized under the condition that the existing mechanism is reserved.
In some embodiments, the steps of creating an empty inode and setting the state of the created inode to an unloaded state include:
creating an empty index node and setting the state of the created index node to be an unloaded state;
and storing the created index nodes in the unsynchronized index node set.
It should be noted here that when a client processes a certain file, the client sends a request to the metadata server where the metadata is located as much as possible, and there is basically only one metadata operation authority cap data in the client. For the cap recovery of this type of inode, no interaction with other metadata servers is required. Only one of the clients is processed according to the method. When MDS fails and switches, and when the client sends a reconnect message, the size field is increased, and the number of caps in the client is saved; when processing a message sent by a client, judging the size, if the size is equal to 1, creating an empty inode, setting the STATE to be an unloaded STATE _ REJOIN _ DELAY _ NULL (indicating that the inode is not really loaded), and storing the inode in an unsynchronized inode node set cap _ objects _ DELAY.
It should be further noted that, for a small portion of many-to-many data, that is, if it is determined that the number of metadata operation permissions is greater than 1, processing is still performed according to the existing flow. The problems that are not solved by the present application are not described herein.
As shown in fig. 2, in some embodiments, if the state of the inode is an unloaded state, the request is suspended, the interface is called to load inode information from the disk, and the step of performing metadata operation permission synchronization operation on the inode after the loading is completed includes:
s51: if the state of the index node is the unloaded state, suspending the request;
s52: calling an interface to load index node information from a disk;
s53: judging whether the loading is successful;
if not, continue to step S52;
if yes, go to step S54;
s54, removing the created empty index node and setting the state of the index node loaded to be a loaded unsynchronized state;
s55, carrying out metadata operation authority synchronization operation on the index nodes;
and S56, clearing the loaded unsynchronized state of the index node after the metadata operation authority synchronization operation is completed.
It should be noted that after the MDS completes startup, the inode in the cap _ objects _ delay set is not really loaded, and the cap is not synchronized. When the client sends a request to operate the inode, the following processing is performed: if the inode STATE is STATE _ REJOIN _ DELAY _ NULL; suspending the request; calling an interface, and loading inode information from a disk; after the inode is successfully loaded, removing the previously created empty inode, and setting the STATE of the loaded inode as the STATE _ REJOIN _ DELAY _ fed loaded unsynchronized STATE (indicating that the inode is loaded but not yet cap synchronized) to perform cap synchronization operation on the inode. And after the synchronization is completed, clearing the STATE _ REJOIN _ DELAY _ FETCHED STATE, re-executing the client request, and completing the response service.
If the state of the index node is the loaded unsynchronized state, performing metadata operation authority synchronization operation on the index node; and after the metadata operation authority synchronization operation is completed, clearing the loaded unsynchronized state of the index node. Through the steps, the metadata operation authority is ensured to be synchronous when the index nodes are processed. If the inode STATE is STATE _ REJOIN _ DELAY _ FETCHED, the inode is subjected to cap synchronization operation. And after the synchronization is completed, clearing the STATE _ REJOIN _ DELAY _ FETCHED STATE, continuously executing the client request, and completing the responded service.
By the aid of the method, the cap is ensured to be synchronous when the inode is processed. The cap synchronization is time-consuming, and has a certain influence on the time consumption of the request. There is a tick method in the MDS that is executed periodically with a certain time interval (default 5 seconds). Logic is added in the tick, a certain number of cap synchronous operations are carried out, after a certain time, the caps are completely and synchronously completed, and the influence on the request is reduced. That is, in some embodiments, the method further comprises:
after the metadata server is started, judging the state of the metadata server, and if the state of the metadata server is in an operating state, executing the following steps: when a request sent by a client side for operating the index node is received, judging the state of the index node; otherwise, ending. And a certain amount of metadata operation permission synchronous operation is carried out, and after a certain time, the metadata operation permission is completely and synchronously completed, so that the influence on the request is reduced.
As shown in fig. 3, after the metadata server is started, the step of tick determining the state of the metadata server specifically includes:
s41: after the metadata server is started, acquiring the state of the metadata server;
s42: judging whether the state of the metadata server is a running state or not;
if not, ending;
if yes, go to step S43;
s43: judging whether the number of index nodes to be processed in the configuration item is greater than 0;
if not, ending;
if yes, go to step S44;
s44: traversing the unsynchronized index node set to acquire the states of the index nodes;
s45: judging whether the state of the index node is an unloaded state or a loaded unsynchronized state;
if not, ending;
if yes, go to step S46;
s46: performing metadata operation authority synchronous operation according to the state of the index node, and adding one to count;
s47: judging whether the count is greater than the number of the index nodes to be processed in the configuration item;
if yes, ending;
if not, go to step S44. For a small part of metadata operation authorities needing interaction between metadata servers, the existing logic processing is used, and the implementation is simple.
Step S46: the specific steps of performing metadata operation authority synchronization operation according to the state of the index node comprise: if the state of the index node is the unloaded state, suspending the request, calling an interface to load index node information from a disk, and performing metadata operation permission synchronous operation on the index node after the loading is finished; restoring the metadata operation authority to the state before the fault; and if the state of the index node is the loaded unsynchronized state, performing metadata operation authority synchronization operation on the index node.
the processing logic for tick addition is as follows:
judging whether the MDS is in an active state (the MDS is in an operating state and can normally process various services), and only processing the MDS by the active state; and judging whether mds _ delay _ load _ dead _ by _ tick (configuration item, inode number processed each time) is larger than 0 or not, otherwise, not processing, and ending. Circulating the unsynchronized index node set cap _ objects _ DELAY, acquiring the inode, judging whether the inode is in a STATE _ REJOIN _ DELAY _ NULL STATE or a STATE _ REJOIN _ DELAY _ FETCHED STATE, if not, not processing, ending, and adding one to the count; if the inode is STATE _ REJOIN _ DELAY _ NULL, loading from a disk first and then synchronizing; if the STATE _ DELAY _ fed is the STATE _ DELAY _ fed, the cap synchronization is directly performed, and if the count is greater than mds _ DELAY _ load _ deal _ by _ tick, the process is ended.
As shown in fig. 4, the technical solution of the present invention provides a system for delaying synchronous recovery of metadata operation permission information, including a metadata server and a client;
the metadata server comprises a quantity judgment module, an index node creation module, an index node state judgment module, a synchronous operation module and a request response module;
the client is used for sending reconnection information containing metadata operation authority quantity to the metadata server when the metadata server fails and restarts;
the quantity judging module is used for judging the quantity of the metadata operation permission when processing the message sent by the client;
the index node creating module is used for creating an empty index node and setting the state of the created index node as an unloaded state if the number judging module judges that the number of the metadata operation authority is equal to 1;
the index node state judgment module is used for judging the state of the index node when a request sent by a client side for operating the index node is received after the metadata server is started;
the synchronous operation module is used for carrying out metadata operation authority synchronous operation according to the state of the index node output by the index node state judgment module; the method is specifically used for suspending a request if the state of the index node is an unloaded state, calling an interface to load index node information from a disk, and performing metadata operation permission synchronization operation on the index node after loading is completed; if the state of the index node is the loaded unsynchronized state, performing metadata operation authority synchronization operation on the index node;
and the request response module is used for continuously executing the request completion response service of the client after the metadata operation permission synchronous operation is completed. The time consumption of switching the metadata server is greatly reduced.
In some embodiments, the index node creation module comprises a creation setting unit and a saving unit;
the creating and setting unit is used for creating an empty index node and setting the state of the created index node to be an unloaded state;
and the storage unit is used for storing the created index nodes in the unsynchronized index node set.
In some embodiments, the synchronization operation module includes a suspend unit, a load state determination unit, a state setting unit, and a synchronization unit;
the suspending unit is used for suspending the request if the state of the index node output by the index node state judging module is an unloaded state;
the loading unit is used for calling an interface to load index node information from a disk;
a loading state judgment unit for judging whether loading is successful;
if not, continuing to execute the steps: calling an interface to load index node information from a disk;
the state setting unit is used for removing the created empty index node and setting the state of the index node loaded into the loaded unsynchronized state if the loading state judgment unit outputs that the loading is successful; the synchronization unit is also used for clearing the loaded unsynchronized state of the index node after completing the metadata operation authority synchronization operation;
and the synchronization unit is used for carrying out metadata operation authority synchronization operation on the index nodes.
In some embodiments, the system further comprises a metadata server state determination module;
and the metadata server state judgment module is used for judging the state of the metadata server after the metadata server is started, and outputting information to the index node state judgment module if the state of the metadata server is in the running state. And the timing is synchronized, so that the influence is further reduced.
Although the present invention has been described in detail by referring to the drawings in connection with the preferred embodiments, the present invention is not limited thereto. Various equivalent modifications or substitutions can be made on the embodiments of the present invention by those skilled in the art without departing from the spirit and scope of the present invention, and these modifications or substitutions are within the scope of the present invention/any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the appended claims.
Claims (10)
1. A method for delaying synchronous recovery of metadata operation authority information is characterized by comprising the following steps:
when the metadata server fails and is restarted, the client sends reconnection information containing the number of metadata operation authorities to the metadata server;
when the metadata server processes the message sent by the client, judging whether the number of metadata operation authorities is equal to 1 or not;
if the number of the metadata operation authorities is equal to 1, creating an empty index node and setting the state of the created index node as an unloaded state;
after the metadata server is started, judging the state of the index node when a request sent by a client side for operating the index node is received;
if the state of the index node is the unloaded state, suspending the request, calling an interface to load index node information from a disk, and performing metadata operation permission synchronous operation on the index node after the loading is finished;
if the state of the index node is the loaded unsynchronized state, performing metadata operation authority synchronization operation on the index node;
and after the metadata operation permission synchronous operation is completed, continuing to execute the request completion response of the client.
2. The method of claim 1, wherein the step of creating an empty inode and setting the state of the created inode to an unloaded state comprises:
creating an empty index node and setting the state of the created index node to be an unloaded state;
and storing the created index nodes in the unsynchronized index node set.
3. The method of claim 2, wherein if the state of the inode is an unloaded state, suspending the request, calling an interface to load inode information from a disk, and performing metadata operation permission synchronization operation on the inode after the loading is completed comprises:
if the state of the index node is the unloaded state, suspending the request;
calling an interface to load index node information from a disk;
judging whether the loading is successful;
if not, continuing to execute the steps: calling an interface to load index node information from a disk;
if so, removing the created empty index node, and setting the state of the index node loaded into the state of being not synchronized;
carrying out metadata operation authority synchronization operation on the index nodes;
and after the metadata operation authority synchronization operation is completed, clearing the loaded unsynchronized state of the index node.
4. The method as claimed in claim 3, wherein if the state of the inode is a loaded non-synchronous state, the step of performing the metadata operation permission synchronization operation on the inode comprises:
if the state of the index node is the loaded unsynchronized state, performing metadata operation authority synchronization operation on the index node;
and after the metadata operation authority synchronization operation is completed, clearing the loaded unsynchronized state of the index node.
5. The method for delaying synchronous recovery of metadata operation authority information as claimed in claim 4, wherein the method further comprises:
after the metadata server is started, judging the state of the metadata server, and if the state of the metadata server is in an operating state, executing the following steps: when a request sent by a client side for operating the index node is received, judging the state of the index node; otherwise, ending.
6. The method for delayed synchronous recovery of metadata operation authority information according to claim 5, wherein the step of judging the state of the metadata server after the metadata server is started comprises:
after the metadata server is started, acquiring the state of the metadata server;
judging whether the state of the metadata server is a running state or not;
if not, ending;
if yes, judging whether the number of the index nodes to be processed in the configuration item is larger than 0;
if not, ending;
if so, traversing the unsynchronized index node set to acquire the state of the index node;
judging whether the state of the index node is an unloaded state or a loaded unsynchronized state;
if not, ending;
if yes, performing metadata operation authority synchronization operation according to the state of the index node, and adding one to the count;
judging whether the count is greater than the number of the index nodes to be processed in the configuration item;
if yes, ending;
if not, executing the following steps: and traversing the unsynchronized index node set to acquire the state of the index node.
7. A system for delaying synchronous recovery of metadata operation authority information is characterized by comprising a metadata server and a client;
the metadata server comprises a quantity judgment module, an index node creation module, an index node state judgment module, a synchronous operation module and a request response module;
the client is used for sending reconnection information containing metadata operation authority quantity to the metadata server when the metadata server fails and restarts;
the quantity judging module is used for judging the quantity of the metadata operation permission when processing the message sent by the client;
the index node creating module is used for creating an empty index node and setting the state of the created index node as an unloaded state if the number judging module judges that the number of the metadata operation authority is equal to 1;
the index node state judgment module is used for judging the state of the index node when a request sent by a client side for operating the index node is received after the metadata server is started;
the synchronous operation module is used for carrying out metadata operation authority synchronous operation according to the state of the index node output by the index node state judgment module; the method is specifically used for suspending a request if the state of the index node is an unloaded state, calling an interface to load index node information from a disk, and performing metadata operation permission synchronization operation on the index node after loading is completed; if the state of the index node is the loaded unsynchronized state, performing metadata operation authority synchronization operation on the index node;
and the request response module is used for continuously executing the request completion response service of the client after the metadata operation permission synchronous operation is completed.
8. The system for delaying synchronous recovery of metadata operation authority information as claimed in claim 7, wherein the index node creation module includes a creation setting unit and a saving unit;
the creating and setting unit is used for creating an empty index node and setting the state of the created index node to be an unloaded state;
and the storage unit is used for storing the created index nodes in the unsynchronized index node set.
9. The system for delaying synchronous recovery of metadata operation authority information as claimed in claim 8, wherein the synchronous operation module includes a suspension unit, a loading state judgment unit, a state setting unit and a synchronization unit;
the suspending unit is used for suspending the request if the state of the index node output by the index node state judging module is an unloaded state;
the loading unit is used for calling an interface to load index node information from a disk;
a loading state judgment unit for judging whether loading is successful;
if not, continuing to execute the steps: calling an interface to load index node information from a disk;
the state setting unit is used for removing the created empty index node and setting the state of the index node loaded into the loaded unsynchronized state if the loading state judgment unit outputs that the loading is successful; the synchronization unit is also used for clearing the loaded unsynchronized state of the index node after completing the metadata operation authority synchronization operation;
and the synchronization unit is used for carrying out metadata operation authority synchronization operation on the index nodes.
10. The system for delaying synchronous recovery of metadata operation authority information as claimed in claim 7, further comprising a metadata server status judging module;
and the metadata server state judgment module is used for judging the state of the metadata server after the metadata server is started, and outputting information to the index node state judgment module if the state of the metadata server is in the running state.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010886946.8A CN111984592B (en) | 2020-08-28 | 2020-08-28 | Method and system for delayed synchronous recovery of metadata operation authority information |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010886946.8A CN111984592B (en) | 2020-08-28 | 2020-08-28 | Method and system for delayed synchronous recovery of metadata operation authority information |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111984592A true CN111984592A (en) | 2020-11-24 |
| CN111984592B CN111984592B (en) | 2022-07-19 |
Family
ID=73440162
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010886946.8A Active CN111984592B (en) | 2020-08-28 | 2020-08-28 | Method and system for delayed synchronous recovery of metadata operation authority information |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111984592B (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103179185A (en) * | 2012-12-25 | 2013-06-26 | 中国科学院计算技术研究所 | Method and system for creating files in cache of distributed file system client |
| CN108880906A (en) * | 2018-07-06 | 2018-11-23 | 郑州云海信息技术有限公司 | A kind of fault recovery method of Metadata Service, server, client and system |
| CN108900337A (en) * | 2018-06-29 | 2018-11-27 | 郑州云海信息技术有限公司 | A kind of fault recovery method of Metadata Service, server, client and system |
| CN109165112A (en) * | 2018-08-16 | 2019-01-08 | 郑州云海信息技术有限公司 | A kind of fault recovery method, system and the associated component of metadata cluster |
-
2020
- 2020-08-28 CN CN202010886946.8A patent/CN111984592B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103179185A (en) * | 2012-12-25 | 2013-06-26 | 中国科学院计算技术研究所 | Method and system for creating files in cache of distributed file system client |
| CN108900337A (en) * | 2018-06-29 | 2018-11-27 | 郑州云海信息技术有限公司 | A kind of fault recovery method of Metadata Service, server, client and system |
| CN108880906A (en) * | 2018-07-06 | 2018-11-23 | 郑州云海信息技术有限公司 | A kind of fault recovery method of Metadata Service, server, client and system |
| CN109165112A (en) * | 2018-08-16 | 2019-01-08 | 郑州云海信息技术有限公司 | A kind of fault recovery method, system and the associated component of metadata cluster |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111984592B (en) | 2022-07-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US5923832A (en) | Method and apparatus for checkpointing in computer system | |
| WO2019154394A1 (en) | Distributed database cluster system, data synchronization method and storage medium | |
| US20150213100A1 (en) | Data synchronization method and system | |
| CN110247984B (en) | Service processing method, device and storage medium | |
| CN107919977B (en) | Online capacity expansion and online capacity reduction method and device based on Paxos protocol | |
| CN107623703B (en) | Synchronization method, device and system for Global Transaction Identifier (GTID) | |
| WO2015081470A1 (en) | Data processing device and data processing method | |
| CN110768873B (en) | Distributed heartbeat detection method, system and device and computer equipment | |
| CN105069152B (en) | data processing method and device | |
| US20120278429A1 (en) | Cluster system, synchronization controlling method, server, and synchronization controlling program | |
| WO2022174735A1 (en) | Data processing method and apparatus based on distributed storage, device, and medium | |
| CN103220183A (en) | Implement method of Hadoop high-availability system based on double-main-engine warm backup | |
| US11748215B2 (en) | Log management method, server, and database system | |
| CN111061431B (en) | Distributed storage method, server and client | |
| CN114900532A (en) | Power data disaster tolerance method, system, device, computer equipment and storage medium | |
| CN110351313B (en) | Data caching method, device, equipment and storage medium | |
| CN111666266A (en) | Data migration method and related equipment | |
| CN107357800A (en) | A kind of database High Availabitity zero loses solution method | |
| US20100274758A1 (en) | Data processing method, computer, and data processing program | |
| CN110377664B (en) | Data synchronization method, device, server and storage medium | |
| CN112052230A (en) | Multi-machine room data synchronization method, computing equipment and storage medium | |
| CN109726211B (en) | Distributed time sequence database | |
| CN114443364A (en) | Distributed block storage data processing method, device, equipment and storage medium | |
| CN108616598B (en) | Data synchronization method and device and distributed storage system | |
| CN107943615B (en) | Data processing method and system based on distributed cluster |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |