WO2016141702A1

WO2016141702A1 - Distributed system metadata migration method and device

Info

Publication number: WO2016141702A1
Application number: PCT/CN2015/089679
Authority: WO
Inventors: 姬庆发
Original assignee: 中兴通讯股份有限公司
Priority date: 2015-03-10
Filing date: 2015-09-15
Publication date: 2016-09-15
Also published as: CN106034080A

Abstract

Provided are a distributed system metadata migration method and device. The method comprises: receiving a notification message from a distributed system, wherein the notification message is used to instruct a metadata distribution routing rule to change; and comparing, according to the notification message, a metadata distribution routing rule before an adjustment and that after an adjustment to obtain a migration data stream of the metadata between nodes of a metadata server cluster, wherein the migration data stream is used to instruct a metadata migration process. The present invention addresses a problem in the related art in which upon adjusting a metadata routing rule it is necessary to split or migrate part of the metadata to one or more other metadata server clusters and synchronously modify the affected metadata routing rule.

Description

Metadata migration method and device in distributed system

Technical field

The present invention relates to the field of communications, and in particular to a method and apparatus for migrating metadata in a distributed system.

Background technique

After the border gateway that processes the client request message in the distributed system where the client requests the message arrives, the border gateway needs to first locate the metadata server according to a certain method, and then the metadata server queries the database to route the message to the next-level metadata server or Routed to a server that stores real data. In the process of locating the metadata server, two technologies are generally used in the related art: pre-configuring the routing information of the metadata or the system to create an internal database to maintain the routing relationship between the query condition and the metadata server. However, with the increase of stored data, the operation of querying the comparison relationship will become the bottleneck of the system, and this solution will no longer be feasible. Therefore, in the scheme of large-capacity database, the scheme of pre-configuring metadata routing information is adopted.

For the scheme of pre-configuring metadata routing, the related problem in the related art is: in this scheme, once the routing rule is determined, the distribution of the metadata is bound to the routing information of the metadata. Therefore, when the system needs to adjust the metadata routing rules, it is necessary to first migrate the metadata from one of the existing metadata server clusters to another cluster, and ensure that the control system switches to the new metadata distribution routing scheme after the metadata migration is completed. For example, when the system needs to modify the routing information of a user number segment to a new metadata server cluster, or when the growth of metadata in a certain metadata server cluster reaches a system bottleneck, the system needs to split part of the metadata into One or more other metadata server clusters and synchronously modify the affected metadata routing rules.

When adjusting the metadata routing rules in the related art, it is necessary to split or migrate part of the metadata to another one or more metadata server clusters, and simultaneously modify the affected metadata routing rules. The industry has not proposed an effective solution. Program.

Summary of the invention

The main purpose of the embodiments of the present invention is to provide a method and an apparatus for migrating metadata in a distributed system, so as to solve the problem of adjusting metadata routing rules in the related art, part of the metadata needs to be split or migrated to one or more other ones. The metadata server clusters and synchronizes the issue of modifying the affected metadata routing rules.

According to an aspect of an embodiment of the present invention, a method for migrating metadata in a distributed system includes: receiving a notification message from a distributed system, wherein the notification message is used to indicate a metadata distribution routing rule And changing the metadata distribution rule before and after the adjustment according to the notification message, to obtain a migration data flow of the metadata between nodes of the metadata server cluster, where the migration data The flow is used to indicate the migration process of the metadata, where the migration data flow includes: an migration node that migrates the metadata in the first server cluster, and an migration node that migrates the metadata in the second server cluster, And the content of the metadata is sent to the egress node to copy the content of the metadata to the embedding node according to the migrated data stream.

Optionally, before the migrating data stream is notified to the egress node to copy the content of the metadata to the migrating node, the method further includes: recording migration startup information of migrating the metadata The startup information includes: a startup time, a last write operation information of the distributed system before starting data migration, wherein the write operation information includes: a time when the write operation occurs, a content of the write operation, and a write The serial number of the operation; or, the notification, is sent to the eviction node to apply for the first lock, wherein the first lock is used to lock the write operation of all the migrated data in the migrating node.

Optionally, after the migrating data stream is started, and before copying the metadata to the migrating node, the method further includes: after the migrating startup information of the metadata is recorded, All write operation information before copying the metadata to the move-in node is written to the synchronization queue; or, after notifying the move-out node to apply for the first lock, and copying the metadata to the Before moving to the node, the method further includes: prohibiting the egress node from performing the write operation.

Optionally, after the migrating data stream is notified to the migrating node to copy the content of the metadata to the migrating node, the method includes: notifying the migrating node to apply for a second lock, where a second lock is configured to lock a write operation of the migrated metadata; and send the write operation information in the synchronization queue to the move node; after the content copy of the metadata is completed, notify Deleting the metadata that has been copied in the migrating node, and releasing the second lock; or notifying, deleting the metadata that has been copied in the migrating node, and releasing the first lock.

Optionally, the deleting the metadata that is copied in the migrating node includes: notifying, when the migrating node and the migrating node complete data replication, the current data migration stream replication of the distributed system. Completing, and receiving the notification to delete the indication of the copied metadata in the migrating node; or sending a migration status for querying the migrating data stream to the migrating node and the migrating node And the response instruction sent by the migration node and the migration node in response to the query instruction, wherein the response instruction is used to indicate the migration data flow state; When the migration of the migration data stream is successful, the distributed system is notified to enable a new metadata distribution routing rule, and the mobile node is notified to delete the content of the metadata.

Optionally, the method further includes: when the query instruction indicates that the migration data flow is abnormal, sending a fallback request to the migrating node and the migrating node, where the back request is As stated in the instructions The eviction node and the migrating node terminate the copy operation of the migrating stream data, and notify the migrating node to delete the content of the copied metadata.

Optionally, the first service cluster or the second service cluster may serve as an inbound and outbound service cluster of multiple migrated data streams at the same time.

According to another aspect of the embodiments of the present invention, there is provided a metadata migration apparatus in a distributed system, comprising: a receiving module configured to receive a notification message from a distributed system, wherein the notification message is used to indicate metadata The distribution routing rule is changed; the comparison module is configured to compare the pre-adjusted and adjusted metadata distribution routing rules according to the notification message, to obtain a migration data flow of the metadata between nodes of the metadata server cluster. The migration data stream is used to indicate a migration process of the metadata, where the migration data flow includes: an migration node in the first server cluster that migrates the metadata, and a migration in the second server cluster The migration node of the metadata and the content of the metadata; the first replication module is configured to notify the migration node to copy the content of the metadata to the migration node according to the migration data flow.

Optionally, before the migrating node copies the content of the metadata to the migrating node, the device further includes: a recording module, configured to record migration startup information of migrating the metadata, The startup information includes: a startup time, and a last write operation information of the distributed system before starting the data migration, where the write operation information includes: a time when the write operation occurs, a content of the write operation, and a write operation. Or the first notification module is configured to notify the migrating node to apply for the first lock, where the first lock is used to lock a write operation of all migrated data in the migrating node.

Optionally, after the migrating data stream is started, and before copying the metadata to the migrating node, the device further includes: a second copying module, configured to record the metadata in the After the migration startup information, all the write operation information before copying the metadata to the migration node is written to the synchronization queue; or, after notifying the migration node to apply for the first lock, and copying Before the metadata is sent to the ingress node, the device further includes: a prohibiting module, configured to prohibit the ejecting node from performing the writing operation.

Optionally, after the migrating node copies the content of the metadata to the migrating node, the device further includes: a second notification module, configured to notify the migrating node to apply for the second lock The second lock is used to lock a write operation of the migrated metadata; and the write operation information in the synchronization queue is sent to the move node; the content of the metadata After the copying is completed, the notification deletes the copied metadata in the migrating node, and releases the second lock; or the third notification module is configured to notify the deletion of the copied copy in the migrating node. Metadata and release the first lock.

Optionally, the second notification module includes: a first notification unit, configured to notify the distributed system of completion of current data migration flow replication when the migration node and the migration node complete data replication; a first receiving unit, configured to receive the indication to delete the copied metadata in the migrating node; The third notification module includes: a first sending unit, configured to send, to the egress node and the ingress node, a query instruction for querying a migration status of the migrated data stream; and the second receiving unit is configured to receive the And a response instruction sent by the migrating node and the migrating node in response to the query instruction, where the response instruction is used to indicate the migration data migration state; and the second notification unit is configured to be in the response When the instruction indicates that the migration data stream is successfully migrated, the distributed system is notified to enable a new metadata distribution routing rule, and the mobile node is notified to delete the content of the metadata.

Optionally, the third notification module further includes: a second sending unit, configured to send a fallback to the migrating node and the migrating node when the query instruction indicates that the migration data flow is abnormal a request, wherein the fallback request is used to indicate that the migrating node and the migrating node terminate the copy operation of the migrated stream data, and notify the migrating node to delete the copied metadata. content.

According to the embodiment of the present invention, after the distribution routing rule of the metadata received in the distributed system is changed, the metadata distribution rule in the metadata server cluster is obtained by comparing the pre-adjusted and adjusted metadata distribution routing rules. The migration data stream, according to the migration data flow, informs the migration node to copy the content of the metadata to the migration node, that is, the metadata that needs to be migrated is copied by the mobile data flow for indicating the migration process of the metadata. To the nodes that need to be moved, the metadata migration is realized, and the routing rules of other metadata are not changed at the same time. When the metadata routing rules are adjusted in the related art, some metadata needs to be split or migrated to one or more other ones. The metadata server clusters, and the problem of modifying the affected metadata routing rules is synchronously modified, thereby achieving the effect of smooth movement of metadata.

DRAWINGS

The drawings described herein are intended to provide a further understanding of the invention, and are intended to be a part of the invention. In the drawing:

1 is a flow chart of a method for migrating metadata in a distributed system according to an embodiment of the present invention;

2 is a structural block diagram of a metadata migration apparatus in a distributed system according to an embodiment of the present invention;

3 is a block diagram 1 of an optional structure of a metadata migration apparatus in a distributed system according to an embodiment of the present invention;

4 is a block diagram 2 of an optional structure of a metadata migration apparatus in a distributed system according to an embodiment of the present invention;

5 is a block diagram 3 of an optional structure of a metadata migration apparatus in a distributed system according to an embodiment of the present invention;

6 is an execution sequence of metadata redistribution in a metadata server cluster in accordance with an alternate embodiment of the present invention. schematic diagram;

7 is a flow chart of a metadata adjustment method between two metadata clusters in accordance with an alternative embodiment of the present invention;

8 is a flow chart of a method for metadata redistribution in accordance with an alternate embodiment of the present invention;

9 is a flowchart of a method for copying metadata to be migrated from a migration cluster to an migration cluster according to an alternative embodiment of the present invention;

10 is a flow chart of a method for completing replication of data redistribution in accordance with an alternative embodiment of the present invention;

11 is a flowchart of a fallback processing method according to an alternative embodiment of the present invention;

FIG. 12 is a flowchart of a method for processing migration completion of metadata according to an alternative embodiment of the present invention.

detailed description

It should be noted that the embodiments in the present application and the features in the embodiments may be combined with each other without conflict. The invention will be described in detail below with reference to the drawings in conjunction with the embodiments.

The steps illustrated in the flowchart of the figures may be performed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, may differ from this The steps shown are performed in the order shown or described.

This embodiment provides a method for migrating metadata in a distributed system. FIG. 1 is a flowchart of a method for migrating metadata in a distributed system according to an embodiment of the present invention. As shown in FIG. 1 , the process includes the following steps. :

Step S102: Receive a notification message from a distributed system.

The notification message is used to indicate that the metadata distribution routing rule has changed;

Step S104: Comparing the pre-adjusted and adjusted metadata distribution routing rules according to the notification message to obtain a migration data flow of metadata between nodes of the metadata server cluster;

The migration data flow is used to indicate the migration process of the metadata, and the migration data flow includes: the migration node that migrates the metadata in the first server cluster, the migration node that migrates the metadata in the second server cluster, and the element involved in the migration. The content of the data;

Step S106: The migration node is notified according to the migration data stream to copy the content of the metadata to the migration node.

In this embodiment, after the distribution routing rule of the metadata received in the distributed system is changed, the metadata is distributed between the nodes of the metadata server cluster by comparing the pre-adjusted and adjusted metadata distribution routing rules. The migrated data stream, according to the migrated data stream, informs the evicted node to copy the contents of the metadata to the migrated The way in the node, that is, through the mobile data flow for indicating the migration process of the metadata, the metadata that needs to be migrated is copied to the node that needs to be migrated, the metadata is migrated, and the routing rules of other metadata are not changed. When the adjustment of the metadata routing rule in the related art is solved, the partial metadata needs to be split or migrated to another one or more metadata server clusters, and the affected metadata routing rules are synchronously modified, thereby achieving the meta The effect of smooth data movement.

In an optional embodiment of the present embodiment, before the migrating data stream is notified to the migrating node to copy the content of the metadata to the migrating node according to the migrating data stream, the embodiment may further include: Before the egress node copies the content of the metadata to the embedding node, the embodiment may include the following two methods:

Method 1: Recording the migration startup information of the migrated metadata, where the startup information includes: a startup time, a last write operation information of the distributed system before starting the data migration, wherein the write operation information includes: a time when the write operation occurs, The content of the write operation, the serial number of the write operation;

In addition, after the migration data stream is started, and before the metadata is copied to the migration node, the method further includes: after recording the migration startup information of the metadata, writing all the write operation information before copying the metadata to the migration node. Enter the synchronization queue;

Manner 2: The mobile node is notified to apply for the first lock, where the first lock is used to lock the write operation of all the migrated data in the migrated node.

In addition, after notifying the migrating node to apply for the first lock, and before copying the metadata to the migrating node, the method further includes: prohibiting the escaping node from performing a write operation.

In an optional implementation manner of this embodiment, the method in this embodiment may further include: notifying the migration after the content of the metadata is copied to the migration node according to the migration data flow. The exit node applies for the second lock, wherein the second lock is used to lock the write operation of the migrated metadata; and the write operation information in the synchronization queue is sent to the migrated node; after the content copy of the metadata is completed, the notification is deleted. Migrate the copied metadata in the node and release the second lock.

The method of this embodiment may further include: notifying the deletion of the copied metadata in the migrated node, and releasing the first, after the second embodiment is configured to notify the migrating node to copy the content of the metadata to the migrating node according to the migrating data stream. A lock.

That is, in the above manner, after the metadata that needs to be migrated is synchronized, the write operation of the migrated metadata after the synchronization is locked, or the write operation of all the migrated data of the migrated node is locked by the above manner, thereby preventing the migration. The node continues to process writes for the metadata to be migrated or all of the data. That is to say, before the migration, the write operation of the metadata to be migrated or the write operation of all the migrated data of the migrated node is locked, thereby achieving a smoother migration of the metadata. It should be noted that the above two modes are merely optional embodiments of the embodiments of the present invention, and do not constitute a limitation of the present invention.

The manner of deleting the content of the copied metadata in the migrating node in the foregoing embodiment may be implemented in the following manner. The following two exemplary implementation manners in this embodiment are used to illustrate:

Optional Embodiment 1: When the mobile node and the egress node complete data replication, notify the distributed system of the completion of the current data migration flow replication, and receive an indication to delete the copied metadata in the migrated node.

Optional implementation manner 2: sending, to the migrating node and the migrating node, a query instruction for querying a migration state of the migrated data stream; receiving a response instruction sent by the migrating node and the migrating node in response to the query instruction, where the response instruction It is used to indicate the migration data flow migration state; when the response instruction indicates that the migration data flow migration is successful, the distributed system is notified to enable the new metadata distribution routing rule, and the migration node is notified to delete the content of the metadata.

It should be noted that, in the foregoing two optional implementation manners, the optional implementation manner 1 is a method for actively informing the replication completion after the data migration and the migration node completes the data replication; the optional implementation manner is completed by the two-dimensional system active query replication. The way the state is.

In addition, in the foregoing optional embodiment 2, when the query instruction indicates that the migration data stream migration is abnormal, the method in this embodiment further includes: sending a fallback request to the egress node and the ingress node, where the back request is used for Instructing the eviction node and the migrating node to terminate the copy operation of the migrating stream data, and notifying the migrating node to delete the content of the copied metadata.

Optionally, the first service cluster or the second service cluster involved in this embodiment may serve as an inbound and outbound service cluster of multiple migrated data flows at the same time.

In this embodiment, a device for migrating metadata in a distributed system is also provided. The device is used to implement the foregoing embodiments and preferred embodiments, and details are not described herein. As used below, the term "module" "unit" may implement a combination of software and/or hardware of a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in software, hardware, or a combination of software and hardware, is also possible and contemplated.

2 is a structural block diagram of a metadata migration apparatus in a distributed system according to an embodiment of the present invention. As shown in FIG. 2, the apparatus includes: a receiving module 22 configured to receive a notification message from a distributed system, where the notification is The message is used to indicate that the metadata distribution routing rule has changed; the comparing module 24 is coupled to the receiving module 22, and is configured to compare the pre-adjusted and adjusted metadata distribution routing rules according to the notification message to obtain metadata in the metadata server cluster. The migrated data stream between the nodes, wherein the migrated data stream is used to indicate the migration process of the metadata, and the migrated data stream includes: the migrated node that migrates the metadata in the first server cluster, and the migrated metadata in the second server cluster The content of the migration node and the metadata; the first replication module 26 is coupled to the comparison module 24, and is configured to notify the migration node to copy the content of the metadata to the migration node according to the migration data flow.

3 is a block diagram of an optional structure of a metadata migration apparatus in a distributed system according to an embodiment of the present invention. As shown in FIG. 3, before the migration node copies the content of the metadata to the migration node, the apparatus further Can include: The recording module 32 is coupled to the comparison module 24 and configured to record migration boot information of the migrated metadata, where the startup information includes: a startup time, and a last write operation information of the distributed system before starting the data migration, where The operation information includes: a time when the write operation occurs, a content of the write operation, and a sequence number of the write operation; after the migration of the data stream is started, and before the metadata is copied to the migrating node, the device may further include: a second copy module 34, Coupled with the recording module 32, it is set to write all the write operation information before copying the metadata to the migrating node to the synchronization queue after recording the migration start information of the metadata.

In this embodiment, after the migrating node copies the content of the metadata to the migrating node, the device includes: a second notification module 36 coupled to the second replication module 34 and coupled to the first replication module. Set to notify the eviction node to apply for the second lock, wherein the second lock is used to lock the write operation of the migrated metadata; and send the write operation information in the synchronization queue to the migrating node; the content of the metadata is copied. After that, the notification deletes the copied metadata in the migrated node and releases the second lock.

Optionally, the second notification module 36 includes: a first notification unit, configured to notify the distributed system of completion of current data migration flow replication when the migration node and the migration node complete data replication; the first receiving unit, setting An indication to delete the copied metadata in the evicted node in order to receive the notification;

In another optional structural block diagram of the embodiment, FIG. 4 is a block diagram 2 of an optional structure of a metadata migration apparatus in a distributed system according to an embodiment of the present invention. As shown in FIG. 4, the metadata is removed from the node. Before the content is copied to the migrating node, the device may further include: a first notification module 42 coupled to the comparison module 24, configured to notify the escaping node to apply for the first lock, wherein the first lock is used to lock the migrating The write operation of all the migrated data in the egress node; after notifying the egress node to apply for the first lock, and before copying the metadata to the embedding node, the device may further include: a disabling module 44 coupled to the first notification module 42 , set to prohibit the move out node from performing write operations.

After the migrating node copies the content of the metadata to the migrating node, the device may further include: a third notification module 46 coupled to the forbidden module 44, configured to notify the deletion of the copied metadata in the migrating node, and Release the first lock.

FIG. 5 is a block diagram 3 of an optional structure of a metadata migration apparatus in a distributed system according to an embodiment of the present invention. As shown in FIG. 5, the third notification module 46 includes: a first sending unit 52, configured to move to and from the node. The migrating node sends a query instruction for querying the migration status of the migrating data stream; the second receiving unit 54 is coupled to the first sending unit 52, and is configured to receive the response sent by the migrating node and the migrating node in response to the query instruction. An instruction, wherein the response instruction is used to indicate a migration data flow migration state; the second notification unit 56 is coupled to the second receiving unit 54 and configured to notify the distributed system to enable the new one when the response instruction indicates that the migration data flow migration is successful. Metadata distributes routing rules and notifies the evicted node to delete the contents of the metadata.

When the query instruction indicates that the migration data flow is abnormal, the third notification module 46 may further include: The sending unit 58 is coupled to the second receiving unit 54 and configured to send a fallback request to the migrating node and the migrating node when the query instruction indicates that the migration data flow is abnormal, wherein the back request is used to indicate the notification migrating The node and the ingress node terminate the copy operation of the migrated stream data, and notify the inmigrated node to delete the contents of the copied metadata.

It should be noted that, for the first service cluster or the second service cluster involved in the migration apparatus of this embodiment, the migration and migration service clusters of multiple migration data flows can be simultaneously performed.

The invention is illustrated by the following figures in conjunction with the accompanying drawings and alternative embodiments.

The present embodiment provides a method for adjusting metadata distribution in a distributed system based on a plan. FIG. 6 is a schematic diagram of an execution sequence of metadata redistribution in a metadata server cluster according to an alternative embodiment of the present invention, as shown in FIG. As shown, the order of the methods is in accordance with the order of execution of the business logic:

Step S1: preparation after redistribution of the metadata route;

Step S2: copying the metadata to be migrated from the migrated cluster to the migrated cluster;

Step S3, the data reproduction of the metadata redistribution is completed;

Step S4, completion of metadata redistribution;

Step S5: When an abnormal process or manual intervention may occur in the above steps, the system is triggered to perform a rollback operation.

The following steps are explained for the above steps S1 to S5. First, the process of preparing the metadata redistribution in the alternative embodiment is explained correspondingly;

The preparation of metadata redistribution can be automatically/manually triggered each time the administrator adjusts the metadata distribution rules, or can be automatically executed by the administrator when the execution plan is scheduled.

In the optional embodiment, after the metadata distribution rule is changed, there is a metadata redistribution control device (hereinafter referred to as a device) in the distributed system, and the device compares the pre-change and changed metadata distribution routing rules. Obtaining a collection of metadata flows between metadata server clusters, each metadata migration data flow is used to define a metadata migration process, including: metadata migration out of the server cluster, migration into the server cluster And the content of the migration metadata. Since the adjustment of metadata distribution may involve moving part of the data and migrating data to a cluster of metadata servers at the same time, a cluster of metadata servers may serve as both a migration server cluster and another migration data stream for one migration data stream. Move out of the server cluster.

After determining the migration to and from the metadata cluster, the device selects a normal server that can handle the write operation from the migrated cluster as the migration node for each migrated data stream, and selects one that can handle the write operation from the migrated cluster. The server acts as an ingress node.

After all the migrated nodes and the migrated nodes of the migrated data stream are determined, the device controls the inbound and outbound nodes of each metadata migration flow to start the data migration process, or cancels the operation according to the operator's instructions.

For the process of copying the metadata to be migrated from the migrated cluster to the migrated cluster in this alternative embodiment:

Each metadata migration data stream, when the metadata redistribution control device notifies the migration and migration server to start the metadata distribution adjustment, in order to achieve smooth migration of the system, the metadata migration server can record the start state of the startup data replication. For example, the start time, start the log sequence number and other information, and record all the write operations for the data to be migrated or the local node that occurred after the start of the operation to the synchronization queue. After that, the metadata will continue to flow from the migrating node to the migrating node. When the metadata to be migrated is copied to the migrating node and saved successfully, the migrating node will automatically synchronize the changed data to the cluster through the data synchronization function deployed in the cluster. Other server nodes within;

After the migration server determines that all the data to be migrated is copied, the system may apply for a write operation lock, lock the migrated data or the write operation of the entire node, and prevent the migrated metadata server from continuing to process the metadata or all the data to be migrated. Write operation, then start synchronizing the operation information cached in the synchronization queue.

After the migration and migration server determines that the synchronization queue synchronization in the migration server is completed, the migration server reports the currently processed metadata migration data flow to the control device for redistributing the metadata. The migration metadata server releases the write operation lock applied by the server, and reports the metadata migration data stream currently processed to the control device for redistributing the metadata.

The process of data replication completion of the metadata to be migrated in this alternative embodiment is interpreted accordingly:

The device updates the state of the metadata migration data stream after receiving the metadata server notification. In order to avoid message loss due to transient failure of the network, the metadata redistribution control device may initiate a query mechanism to query the involved metadata server for the migration status of each migrated data stream.

After the metadata redistribution control device confirms that the metadata migration data flow involved in the adjustment is successfully migrated, it can be concluded that the data replication of the metadata to be migrated is completed. When any one of the metadata migration data streams performs abnormal termination of the metadata, the control device determines that the current metadata adjustment fails, and the system may enter the return flow, or wait for manual intervention and resolve the failure to continue the data replication process.

The data migration completion implementation process for the metadata redistribution involved in the alternative embodiment is interpreted accordingly:

After the metadata redistribution control device confirms that all metadata migration data flows involved in this adjustment are completed, the system performs the following operations:

Step A: Send a notification to the gateway or client responsible for metadata routing in the system to notify the new metadata distribution and routing rules.

Step B: Send a notification message to the data migration out cluster to notify the migrated metadata cluster to delete the migrated metadata that has been copied and is to be deleted.

The gateway that processes the metadata routing, the client, and the control device that redistributes the metadata after the data is deleted by the migration server confirms the processing result, and the current metadata distribution adjustment ends correctly.

To provide system stability, the device can initiate a timing query mechanism to query the processing status of the gateway, client, and eviction server that handles metadata routing.

The fallback implementation process of the metadata redistribution in the alternative embodiment is interpreted accordingly:

In the process of redistribution of metadata, the rollback operation can be triggered in the following scenarios, and the condition is judged in no particular order:

Scenario 1: During the metadata replication process to be migrated, it is impossible to find a suitable migration and migration metadata server for the metadata migration data stream.

Scenario 2: During the metadata replication process to be migrated, data replication fails.

Scenario 3: The metadata redistribution control device loses communication with the at least one migrated, migrated metadata server cluster during the metadata redistribution process.

Scene 4: Manual operation setting operation is rolled back.

The execution of the rollback operation can support the following methods: (1): the system executes immediately; (2): applying for manual intervention, manually selecting to perform the rollback or selecting to continue the attempt until the fault is eliminated.

After the rollback operation is started, the metadata redistribution control device sends a fallback request to the inbound and outbound servers involved in each metadata migration data flow, and moves into and out of the server after receiving the fallback request to perform the following operations. :

Move out of the server: terminate the replication process of the corresponding metadata migration data stream as required, and clear the synchronization queue. If you have applied for a write lock, release the operation lock.

Move to the server: Stop the corresponding metadata migration data stream replication process as required, and delete the copied data.

With the optional embodiment, when the system needs to adjust the distribution rule of the metadata, the metadata server cluster that needs to perform data migration is determined based on the change of the metadata routing rules before and after the comparison update, and the collaboration and control metadata are in the metadata cluster. The migration between the metadata enables smooth distribution of the metadata distribution in the system as the routing information is adjusted. At the same time, after the control system adjustment is completed, it can automatically switch to the new metadata distribution rule to avoid business interruption.

The optional embodiment will be described in detail below with reference to the accompanying drawings and specific embodiments;

Embodiment 1

7 is a flowchart of a metadata adjustment method between two metadata clusters according to an alternative embodiment of the present invention. As shown in FIG. 7, a system based on two metadata clusters is used as an example, and a metadata adjustment method based on a plan is used. The steps include:

Steps S702-S704: The client or the gateway responsible for distributing the client message accesses the server node in the metadata cluster 1 and the metadata cluster 2 according to the rules of metadata distribution.

Step S706: The system administrator adjusts the distribution rule of the metadata, and according to the manner triggered by the system: automatic, manual, and timing are triggered; after the distribution rule is adjusted, the metadata redistribution request is initiated to the metadata redistribution device in the system.

Step S708: The metadata redistribution device analyzes the pre-adjusted and adjusted metadata distribution rules to obtain the changed content of the current adjustment: confirming that the metadata in the partial metadata server cluster 1 is adjusted into the metadata server cluster 2. At the same time, the metadata redistribution device selects the metadata server 1-1 as the migration server according to the state of the server, and the metadata server 2-1 serves as the migration server.

Step S710: The metadata re-distribution device requests the migration server of the metadata to perform a data migration operation;

Step S712: The metadata re-distribution device requests the migration server of the metadata to perform a data migration operation;

Step S714: The metadata starts migration, and the migration and migration server will copy the metadata to be migrated from the migration server to the migration server according to the instruction given by the metadata redistribution device;

Step S716: After the migrated metadata enters the migration server and is successfully saved, the metadata migration server synchronizes the migrated data to other nodes of the cluster through the data synchronization mechanism deployed in the cluster;

Step S718: The metadata migration server confirms that the metadata to be migrated has been copied to the migration meta-server, and reports the completion status to the metadata redistribution device.

Step S720: The metadata migration server confirms that the data to be migrated has been successfully saved and reports the completion status to the metadata redistribution device;

Step S722: The metadata redistribution device confirms that the metadata migration operations involved in the adjustment have been completed according to the confirmation message of the migration and migration server, and is responsible for routing the client request to the gateway (or the client of the metadata server). When the client is responsible for data routing, the metadata distribution rules are synchronized.

Step S724: The gateway or the client of the metadata server that requests the client request takes effect a new metadata distribution rule;

Step S726: The gateway or client of the metadata server that requests the client request confirms to the metadata redistribution device that the new metadata distribution rule takes effect;

Step S728: The metadata redistribution device notifies the migration server to delete the migrated data, and the number of the migrated server executions Delete the migrated metadata according to the delete operation;

Step S730: After the metadata migration server removes the metadata, the metadata synchronization mechanism deployed in the cluster is used to delete the migrated metadata of other servers in the cluster.

In the first embodiment, the metadata redistribution device can trigger the rollback process by the following steps:

Manner 1: In step S708:, the metadata redistribution device cannot select a migration or migration server that is in a normal state from the system.

Manner 2: In step S710, in S712, the migration or migration server confirms that data migration cannot be performed.

Manner 3: The metadata redistribution device loses communication with the migrating or migrating server in step S714.

Manner 4: In step S718, in S720, the migration server reports the data migration or migration abnormality.

Mode 5: Before step S720, the operator submits a fallback request.

The processing manner of the fallback process related to the first embodiment is: A: the metadata redistribution device requests the metadata migration server to stop the data migration; B: the metadata redistribution device requests the metadata migration server to stop the data migration. The migrated server will delete the copied metadata after receiving the request, and the delete operation will be synchronized to other servers in the cluster through the synchronization mechanism deployed in the cluster.

FIG. 8 is a flowchart of a method for redistributing metadata according to an alternative embodiment of the present invention. As shown in FIG. 8, the steps of preparing for metadata redistribution in this embodiment include:

Step S802: Modify a metadata distribution routing rule.

Step S804: confirming the modification to the distributed system and setting the execution mode of the system;

Wherein, the execution mode can be executed immediately: the system will immediately perform preparation for metadata redistribution; or, timing execution: the administrator wants the system to execute at some point in the future;

Step S806: setting the execution mode, if it is executed immediately, proceeding to step S810, otherwise proceeding to step S808;

Step S808: The system waits for the time set by the administrator to arrive;

Step S810: The metadata redistribution control device compares the metadata routing rules before and after the change, obtains the metadata server cluster information that needs to be moved out and moved in, and the metadata information that needs to be adjusted;

Wherein, due to the adjustment of the distribution rules of the metadata, a metadata node may have a metadata redistribution control device in the distributed system after the metadata distribution rule changes in the metadata distribution adjustment process. The metadata distribution rules before and after the change will be compared to obtain the migrated data stream of the metadata between the metadata server clusters, and the migration data stream for each metadata, including the cluster of the migrated server And the cluster that moved into the server.

Step S812: The metadata redistribution control device checks whether the current adjustment needs to perform data migration, and if necessary, performs step S814; otherwise, the execution ends;

Step S814: Select a server capable of processing a write operation from each of the migrated and migrated metadata server clusters involved in each migrated data stream;

The metadata redistribution control device selects a normal server that can handle the write operation from the migrated cluster as the migration node for each migrated data stream, and selects a normal server that can handle the write operation from the migrated cluster as the migration. Into the node.

Step S816: Check all the migrated data streams are successfully operated; if successful, proceed to step S818; if not, proceed to step S820;

Step S818: the metadata to be migrated is copied from the migrated cluster to the migrated cluster, and the execution ends;

Step S820: The system enters a fallback process judgment.

9 is a flowchart of a method for copying metadata to be migrated from an eviction cluster to an migrating cluster according to an alternative embodiment of the present invention. As shown in FIG. 9, the steps of the method include:

Step S902: The metadata redistribution control device initiates a data migration request to the migration and migration server of each migration data stream;

Step S904: the migration to the server records the data migration start, after which all the write operations or all write operations involving the data to be migrated will be recorded in the synchronization queue;

Step S906: The migration server sends the metadata to be migrated to the migration server according to the indication;

After the data migration is started, the metadata will continue to be copied from the migrating node to the migrating node. When the copied metadata reaches the migrating node and is saved successfully, the migrating node will automatically pass the existing data synchronization function in the cluster. Synchronize changes to other server nodes within the cluster. The system client does not take effect immediately after the data migration is completed. To ensure the continuity of the service, the metadata to be migrated cannot be deleted immediately after being copied from the migrated meta-server cluster to the migrated meta-server cluster. You must obtain confirmation from the metadata redistribution control device or manually confirm and move out of the metabase server to perform the delete operation.

Step S908: The egress node determines whether all the data to be migrated is copied to the migration server, and if yes, execute S910, otherwise proceed to step S906;

Step S910: The migration operation locks the write operation of the metadata to be migrated, and synchronously caches the operation in the synchronization queue to the migration server;

Step S912: The migration server determines whether all the operations in the synchronization queue are copied to the migration server, and if yes, execute S914: otherwise, continue to execute S910;

Step S914: The migration and migration server reports to the "metadata redistribution control device" that the currently processed metadata migration data stream is completed.

Step S916: Enter the “data migration completion of metadata redistribution” process, and wait for the system to determine that the data migration is completed;

FIG. 10 is a flowchart of a method for completing replication of data redistribution according to an alternative embodiment of the present invention. As shown in FIG. 10, the steps of the method include:

Step S1002: The control device for metadata redistribution determines the type of event inside the device:

Wherein, A, if it is the status information or status query response message reported by the migrating and migrating server, step S1004 is performed;

B, if it is a timing check task inside the control device of metadata redistribution, then step S1006;

Step S1004: The control device of the metadata redistribution updates the state of the corresponding metadata migration data stream, and the process ends;

Step S1006: The control device for metadata redistribution needs to obtain a list of metadata migration data streams;

Step S1008: The metadata redistribution control device checks each migrated data flow state in the metadata migration data flow list and checks whether there is a data flow in which the execution state is abnormal;

A, if yes, proceeds to the fallback process of step S1010;

B, if not, proceed to step S1012 to continue execution;

Step S1010: The device performs a rollback process;

Step S1012: The control device for metadata redistribution continues to check whether all the migrated data streams are successfully completed, and judges according to the check result: if all the execution is completed, the process proceeds to step S1014; if there is an uncompleted migrated data stream, Go to step S1016 to execute;

Step S1014: Enter the process of “data migration completion of metadata redistribution”, refer to the description of the “data migration completion of metadata redistribution” process of the present invention, and the process ends;

Step S1016: Check whether there is a migration and migration server of the metadata migration data flow failed to report the status to the control device redistributed at the appointed time: if yes, proceed to step S1018; if not, proceed to step S1020;

Step S1018: The control device for redistributing the metadata initiates a status query request to the migration and migration server of the migrated data stream and waits for the response, and the process ends;

Step S1020: The control device for metadata redistribution enters a waiting state, waiting for the next check whether the metadata migration data stream is all ended.

It should be noted that the fallback processing procedure mentioned above is all related to the backoff processing procedure in FIG.

FIG. 11 is a flowchart of a method for processing a rollback according to an alternative embodiment of the present invention. As shown in FIG. 11, the steps of the method for processing the rollback include:

Step S1102: After the rollback process is started, it is first checked whether the current fallback process is a manual intervention rollback: if yes, the device will immediately perform the rollback operation, and proceeds to step S1104; if not, the device proceeds to S1104 to perform the setting according to the system. The decision is to roll back immediately or require manual confirmation before the rollback can be performed.

Step S1104: The device migrates the data stream for each metadata, and sends a fallback request to the inbound and outbound server nodes therein;

Step S1106: After the migration server receives the fallback request, the replication queue is cleared, and the transmission of the replication data is stopped. If the operation lock has been applied, the lock release operation needs to be performed. After the migration server receives the fallback request, the copied metadata is deleted, and the process execution ends;

Step S1108: The device needs to check the processing mode of the system setting: if the system is set to automatically roll back, the process proceeds to step S1104; if the system is set to require manual intervention to confirm whether to roll back, step S1110 is performed;

Step S1110: Check the result of the manual intervention: if the manual confirmation result is the execution of the retreat, the process proceeds to step S1104; if the manual confirmation result is that the retreat is not performed, the device judgment ends, and the device waits for the failure through the timing detection process in FIG. Continue the data migration after elimination.

FIG. 12 is a flowchart of a method for processing migration completion of metadata according to an alternative embodiment of the present invention. As shown in FIG. 12, the steps of the method for processing metadata migration completion in this embodiment include:

Step S1202: The device determines whether it is a timing check message:

If yes, proceed to step S1216 to periodically check the status of the message distribution gateway/client and the involved migration metadata server in the system using the metadata distribution rules.

If no, the process proceeds to step S1204 to notify the system to complete the metadata migration operation.

Step S1204: The device sends a notification to the metadata routing gateway/system client, requesting it to switch to a new metadata routing rule;

Step S1206: The device determines whether a confirmation message returned by the metadata routing gateway/system client is received: If yes, go to step S1208; if no, go to step S1214 to continue waiting;

Step S1208: Send a request to the migration server in each metadata migration stream to delete the migration data.

Step S1210: The device determines whether the migration server finishes execution and returns a processing result to the device: if yes, step S1212 is performed, and if no, step S1214 is performed;

Step S1212: the update metadata operation state is completed, and the process ends;

Step S1214: setting a timing status query, querying an execution status of the migration server and the metadata distribution gateway/client, and the process ends;

Step S1216: determining that the timer is operating, the device queries the metadata distribution gateway/client and the migration metadata server for the result of performing the operation;

Step S1218: Check whether the metadata distribution gateway/client completes the routing rule switching: if yes, execute step S1220; if not, the processing flow ends;

Step S1220: It is checked whether the migration out server completes the migration data deletion: if yes, step S1212 is performed; if not, the flow processing ends.

The above is only an alternative embodiment of the present invention, and is not intended to limit the present invention, and various modifications and changes can be made to the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and scope of the present invention are intended to be included within the scope of the present invention.

Industrial applicability

According to the present invention, after the distribution routing rule of the metadata received in the distributed system is changed, the metadata is distributed between the nodes of the metadata server cluster by comparing the pre-adjusted and adjusted metadata distribution routing rules. Migrating the data stream, according to the way in which the migrated data stream notifies the evicted node to copy the content of the metadata to the embedding node, that is, the metadata that needs to be migrated is copied to the required manner by the mobile data stream for indicating the migration process of the metadata. The migrated node realizes the migration of metadata and does not need to change the routing rules of other metadata. It solves the problem of splitting or migrating part of the metadata to other one or more elements when adjusting the metadata routing rules in the related art. The data server clusters and synchronizes the problem of modifying the affected metadata routing rules, thereby achieving the effect of smooth data movement.

Claims

A method for migrating metadata in a distributed system, comprising:

Receiving a notification message from a distributed system, wherein the notification message is used to indicate that a metadata distribution routing rule has changed;

Comparing the pre-adjusted and adjusted metadata distribution routing rules according to the notification message, to obtain a migration data flow of the metadata between nodes of the metadata server cluster, wherein the migration data flow is used to indicate the a migration process of the metadata, the migration data stream includes: an migration node that migrates the metadata in the first server cluster, an migration node that migrates the metadata in the second server cluster, and the metadata content;

Notifying the egress node to copy the content of the metadata to the embedding node according to the migrated data stream.
The method according to claim 1, wherein, before the migrating data stream is notified to the migrating node to copy the content of the metadata to the migrating node, the method further includes:

Recording the migration startup information of the metadata, wherein the startup information includes: a startup time, a last write operation information of the distributed system before starting the data migration, wherein the write operation information includes: writing The time when the operation occurred, the content of the write operation, the serial number of the write operation; or,

Notifying the vacating node to apply for the first lock, wherein the first lock is used to lock a write operation of all migrated data in the migrating node.
The method of claim 2, wherein

After the migrating data stream is started, and before copying the metadata to the migrating node, the method further includes: after the migrating boot information of the metadata is recorded, at the copying station All write operation information before the metadata is forwarded to the synchronization node is written to the synchronization queue; or

After notifying the vacating node to apply for the first lock, and before copying the metadata to the migrating node, the method further includes: prohibiting the escaping node from performing the write operation.
The method of claim 3, wherein after the migrating data stream is notified to the migrating node to copy the content of the metadata to the migrating node, the method comprises:

Notifying the vacating node to apply for a second lock, wherein the second lock is used to lock a write operation of the migrated metadata; and send the write in the synchronization queue to the migrating node After the content of the metadata is copied, the metadata of the copied node is deleted and the second lock is released; or

Notifying the deletion of the metadata copied in the migrating node and releasing the first lock.
The method according to claim 4, wherein the notifying the deletion of the metadata copied in the migrating node comprises:

Notifying the completion of the current data migration flow replication of the distributed system when the migration node and the migration node complete data replication, and receiving the notification to delete the copied metadata in the migration node Instructions; or,

Sending, to the egress node and the embedding node, a query instruction for querying a migration status of the migrated data stream; receiving a response sent by the egress node and the emigment node in response to the query instruction An instruction, wherein the response instruction is used to indicate the migration data flow migration state; and when the response instruction indicates that the migration data flow migration is successful, notifying the distributed system to enable a new metadata distribution routing rule, and Notifying the migrating node to delete the content of the metadata.
The method of claim 5 wherein the method further comprises:

When the query instruction indicates that the migration data flow is abnormal, sending a fallback request to the migrating node and the migrating node, where the back request is used to notify the migrating node and the The moving in node terminates the copying operation of the migrated stream data, and notifies the ingress node to delete the content of the copied metadata.
The method of claim 1, wherein the first service cluster or the second service cluster can simultaneously serve as an inbound and outbound service cluster of a plurality of migrated data streams.
A metadata migration device in a distributed system, comprising:

a receiving module, configured to receive a notification message from the distributed system, where the notification message is used to indicate that the metadata distribution routing rule has changed;

a comparison module, configured to compare the pre-adjusted and adjusted metadata distribution routing rules according to the notification message, to obtain a migration data flow of the metadata between nodes of the metadata server cluster, where the migration data The flow is used to indicate the migration process of the metadata, where the migration data flow includes: an migration node that migrates the metadata in the first server cluster, and an migration node that migrates the metadata in the second server cluster, The content of the metadata;

And the first copying module is configured to notify the ejecting node to copy the content of the metadata to the embedding node according to the migrated data stream.
The apparatus of claim 8, wherein before the migrating node copies the content of the metadata into the migrating node, the apparatus further comprises:

a recording module, configured to record migration startup information for migrating the metadata, where the startup information includes: a startup time, a last write operation information of the distributed system before starting data migration, wherein the writing The operation information includes: the time when the write operation occurs, the content of the write operation, the serial number of the write operation; or

The first notification module is configured to notify the migrating node to apply for the first lock, where the first lock is used to lock a write operation of all migrated data in the migrating node.
The apparatus according to claim 9, wherein

After the migrating data stream is started, and before copying the metadata to the migrating node, the apparatus further includes: a second copying module, configured to start the migration in the recording the metadata After the information, all the write operation information before copying the metadata to the migrating node is written to the synchronization queue; or

After notifying the migrating node to apply for the first lock, and before copying the metadata to the migrating node, the apparatus further includes: a prohibiting module, configured to prohibit the escaping node from performing the write operation .
The apparatus of claim 10, wherein after the migrating node copies the content of the metadata to the migrating node, the apparatus further comprises:

a second notification module, configured to notify the migrating node to apply for a second lock, where the second lock is used to lock a write operation of the migrated metadata; and send the The write operation information in the synchronization queue; after the content copy of the metadata is completed, notifying the deletion of the copied metadata in the migrating node, and releasing the second lock; or

The third notification module is configured to notify the deletion of the copied metadata in the migrating node, and release the first lock.
The apparatus according to claim 11, wherein

The second notification module includes: a first notification unit, configured to notify the distributed system of completion of current data migration flow replication when the migration node and the migration node complete data replication; and the first receiving unit Providing, by receiving the notification, an indication to delete the copied metadata in the migrating node;

The third notification module includes: a first sending unit, configured to send, to the egress node and the ingress node, a query instruction for querying a migration status of the migrated data stream; and the second receiving unit is configured to Receiving, by the migrating node and the migrating node, a response instruction in response to the query instruction, where the response instruction is used to indicate the migration data flow transition state; and the second notification unit is configured to When the response instruction indicates that the migration data stream is successfully migrated, the distributed system is notified to enable a new metadata distribution routing rule, and the migration node is notified to delete the content of the metadata.
The apparatus of claim 12, wherein the third notification module further comprises:

a second sending unit, configured to send a fallback request to the migrating node and the migrating node when the query instruction indicates that the migrating data stream is abnormal, wherein the back request is used to indicate a notification The migrating node and the migrating node terminate the copy operation of the migrating stream data, and notify the migrating node to delete the content of the copied metadata.
The apparatus according to claim 8, wherein the first service cluster or the second service cluster can simultaneously serve as an inbound and outbound service cluster of a plurality of migrated data streams.