TWI734744B - Method, device and system for synchronizing routing table - Google Patents

Abstract

The invention discloses a method, device and system for synchronizing a routing table. Wherein, the method includes: if it is detected that the routing table in the first data management group is modified, obtaining the modified routing table data; and synchronizing the modified routing table data to any one or more other non-default routing table data The second data management group is used to update the routing table in the non-default second data management group; wherein, a unique first data management group and at least one second data management group are deployed in the distributed file system. The invention solves the technical problem that the routing table in the distributed file system in the prior art is a static table. Therefore, when any routing table in the distributed file system changes, the technical problem that the routing table in the system will be inconsistent.

Description

Method, device and system for synchronizing routing table

The present invention relates to the field of data synchronization, and in particular to a method, device and system for synchronizing routing tables.

Data plays a vital role in the application of computers and the Internet. With the advent of the era of explosive growth of information, users have increasingly higher requirements for data storage and read and write speeds. The traditional Increasing the number of hard disks to expand the storage capacity of a computer file system cannot meet the needs of users in terms of capacity, capacity growth, data read and write speed, and data security. For this reason, there are many existing Internet industries. Use a distributed file system to store and manage data.

In Internet applications, the routing table can be stored in the address resolution server of the distributed file system, and the data can be cached through the dispatch module, because the routing table is used as a storage table stored on routers or other Internet devices , The specific path to reach a specific network interruption is stored, and the router finds the best transmission path among the specific paths stored for each data packet passing through the router through the multiple specific paths stored in the routing table. At present, the routing tables commonly used in distributed file systems are mostly static routing tables. The actual transmission path is modified due to changes. Therefore, when the actual transmission path changes, the data packet cannot be transmitted normally.

Aiming at the technical problem that the tables in the distributed file system in the prior art are static tables, and the tables in any data management group are changed, the tables in the other data management groups cannot be changed synchronously, so far no effective solution has been proposed. .

The embodiment of the present invention provides a method, device and system for synchronizing routing tables, to at least solve the problem that when the tables in the distributed file system in the prior art are static tables, the rest of the data is managed when the tables in any data management group are changed. The technical problem that the tables in the group cannot be changed synchronously.

According to one aspect of the embodiment of the present invention, a method for synchronizing a routing table is provided, the method includes: if it is detected that the routing table in the first data management group is modified, obtaining the modified routing table data; The modified routing table data is synchronized to any one or more non-default second data management group to update the routing table in the non-default second data management group; among them, the only one deployed in the distributed file system A first data management group and at least one second data management group.

According to another aspect of the embodiments of the present invention, a device for synchronizing a routing table is also provided. The device includes: a first acquisition module for detecting that the routing table in the preset first data management group is modified To obtain the modified routing table data; the synchronization module is used to synchronize the modified routing table data to any other one or more non-default second data management groups to update A new non-default routing table in the second data management group; wherein, a unique first data management group and at least one second data management group are deployed in the distributed file system.

According to another aspect of the embodiment of the present invention, there is also provided a routing table synchronization system, the system includes: a first data management server, if it is detected that the routing table in the local storage area is modified, obtain The modified routing table data, and the modified routing table data is synchronized, wherein the first data management server is any server included in the preset first data management group in the distributed file system; Or a plurality of second data management servers, which have a communication relationship with the first data management server, and are used to receive modified routing table data synchronized by the first data management server to update the routing table in the second data management group, Wherein, the second data management server is any server included in the non-default second data management group in the distributed file system.

What needs to be explained here is that the main purpose of the default first data management group in the routing table synchronization method is to detect the change of the routing table in the distributed file system, that is, whether the routing table in the distributed file system is Change, and in the case of detecting a change in the routing table, obtain the latest routing table after the change. Then synchronize the latest routing table to the rest of the second data management group. After updating the routing tables of multiple data management groups in the distributed file system, the interface of the client (user terminal) can confirm the routing table of each data management group in the distributed file system to ensure that each The routing tables in the data management group are all the latest routing tables.

Therefore, the solution of the above-mentioned embodiment provided by the present invention solves the problem of the prior art The table in the intra-operative distributed file system is a static table. When the table in any data management group is changed, the other data management group tables cannot be changed synchronously.

10‧‧‧Computer terminal

102‧‧‧Processor

104‧‧‧Memory

106‧‧‧Transmission Module

S21-S66‧‧‧Step

120‧‧‧First acquisition module

122‧‧‧Sync Module

80‧‧‧Registration Module

90‧‧‧Second acquisition module

92‧‧‧Return Module

100‧‧‧Determine the module

101‧‧‧Execution Module

103‧‧‧Third acquisition module

110‧‧‧First Data Management Server

112‧‧‧Second Data Management Server

1201‧‧‧Processor

1203‧‧‧Memory

1205‧‧‧Transmission device

1207‧‧‧Request terminal

1209‧‧‧Data Storage Server

The drawings described here are used to provide a further understanding of the present invention and constitute a part of the present invention. The exemplary embodiments and descriptions of the present invention are used to explain the present invention, and do not constitute an improper limitation of the present invention. In the drawings: FIG. 1 is a block diagram of the hardware structure of a computer terminal according to a method for synchronizing a routing table according to Embodiment 1 of the present invention; FIG. 2 is a diagram of an optional method for synchronizing a routing table according to Embodiment 1 of the present invention Flow chart; Figure 3 is a schematic diagram of an optional routing table synchronization method according to Embodiment 1 of the present invention; Figure 4 is a schematic diagram of a requesting terminal according to Embodiment 1 of the present invention to send a routing table request to any data management group; Figure 5 It is an optional flow chart of sending a routing table request to the data management group according to Embodiment 1 of the present invention; Figure 6 is an optional requesting terminal to send a routing table request to the data storage server according to Embodiment 1 of the present invention Figure 7 is a schematic diagram of an optional routing table synchronization device according to Embodiment 2 of the present invention; Figure 8 is a synchronization of an optional routing table according to Embodiment 2 of the present invention Figure 9 is a schematic diagram of an optional routing table synchronization device according to Embodiment 2 of the present invention; Figure 10 is a schematic diagram of an optional routing table synchronization device according to Embodiment 2 of the present invention; 11 is a schematic structural diagram of a routing table synchronization system according to Embodiment 2 of the present invention; and FIG. 12 is a structural block diagram of a computer terminal according to Embodiment 2 of the present invention.

In order to enable those with ordinary knowledge in the technology to better understand the solutions of the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described implementation The examples are only a part of the embodiments of the present invention, not all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by a person with ordinary knowledge in the field without creative work shall fall within the protection scope of the present invention.

It should be noted that the terms "first" and "second" in the specification and patent application scope of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific sequence or sequence. It should be understood that the materials used in this way can be interchanged under appropriate circumstances so that the embodiments of the present invention described herein can be implemented in a sequence other than those illustrated or described herein. In addition, the terms "include" and "have" and any variations of them It is intended to cover non-exclusive inclusions. For example, a process, method, system, product, or device that includes a series of steps or units need not be limited to those clearly listed steps or units, but may include Or other steps or units inherent in these processes, methods, products or equipment.

First of all, some terms or terms appearing in the process of describing the embodiments of the present invention are applicable to the following explanations: Federation: The namespace is stored in a distributed manner to form a system cluster, thereby realizing scalability in a distributed system.

Volume: In the Federation system, after namespace grouping, it represents the name of each group of namespaces.

MountTable, routing table, in the Pangu distributed file system, the data in different directories in the system can be placed in the routing table on different volumes.

Pangu Master: The data management group included in the Pangu distributed file system. The structure of the data management group is the structure of data management and data storage. The data management group Master mainly records the attribute meta information of the data, and the data storage group is mainly used for data data Storage.

PanguDefaultVolume: After the introduction of the Federation mode in the Pangu distributed file storage system, multiple data management groups pangu master groups can be divided into the same server cluster. Each group of data management servers constitutes a data management group. Can be mapped to a volume, in which a group of data management groups is the default for the distributed file system , And the only master data management group PanguDefaultVolume.

Example 1

According to the embodiment of the present invention, there is also provided an embodiment of a task transmission method based on sub-library and sub-table. It should be noted that the steps shown in the schematic flowchart can be implemented in a computer system such as a set of computer-executable instructions. Execution, and although a logical sequence is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than here.

The method embodiment provided in the first embodiment of the present invention may be executed in a mobile terminal, a computer terminal, or a similar computing device. Taking running on a computer terminal as an example, FIG. 1 is a block diagram of the hardware structure of a computer terminal according to a method for synchronizing a routing table according to Embodiment 1 of the present invention. As shown in FIG. 1, the computer terminal 10 may include one or more (only one is shown in the figure) processor 102 (the processor 102 may include, but is not limited to, a microprocessor MCU or a field programmable logic gate array FPGA, etc.) Device), a memory 104 for storing data, and a transmission module 106 for communication. Those with ordinary knowledge in the art can understand that the structure shown in FIG. 1 is only for illustration, and does not limit the structure of the above electronic device. For example, the computer terminal 10 may also include more or fewer elements than those shown in FIG. 1, or have a configuration different from that shown in FIG.

The memory 104 can be used to store software programs and modules of application software, such as the routing table synchronization method and the corresponding program instructions/modules of the device in the embodiment of the present invention. The processor 102 is stored in the memory 104 by running The software programs and modules in the software program can execute various functional applications and data processing, that is, the synchronization method of the routing table of the above-mentioned application program is realized. The memory 104 may include a high-speed random memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory provided remotely with respect to the processor 102, and these remote storages may be connected to the computer terminal 10 via a network. Examples of the aforementioned networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

The transmission device 106 is used to receive or send data via a network. The above-mentioned specific examples of the network may include a wireless network provided by the communication provider of the computer terminal 10. In one example, the transmission device 106 includes a network interface controller (NIC), which can be connected to other network devices through a base station to communicate with the Internet. In one example, the transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet in a wireless manner.

In the above operating environment, the present invention provides a method for synchronizing the routing table as shown in FIG. 2. Fig. 2 is a flowchart of an optional routing table synchronization method according to Embodiment 1 of the present invention. The method shown in Fig. 2 may include the following steps:

In step S21, if it is detected that the routing table in the preset first data management group is modified, obtain the modified routing table data.

In the above steps, the preset first data management group can be distributed For any data management group in the file system, in the routing table synchronization method, the first data management group is used to synchronize the received latest routing table to other data management groups deployed in the distributed file system.

In an alternative embodiment, the preset first data management group may be the data management group storing the most important data in the distributed file system, or the data management group with the largest data storage capacity, but in practical applications, Specifically speaking, which data management group in the distributed file system serves as the default first data management group is not limited here.

What needs to be explained here is that the main purpose of the default first data management group in the routing table synchronization method is to detect the change of the routing table in the distributed file system, that is, whether the routing table in the distributed file system is Change, and in the case of detecting a change in the routing table, obtain the latest routing table after the change.

It should also be noted here that in the distributed file system involved in the present invention (such as the Pangu pangu distributed file system), the formation of a data management group can usually be realized by introducing the Federation mode, that is, the Federation The decentralized partitioning mode of the data namespace (namespace) adopted in the NAS introduces the mode of data management group, so that it forms a collection group consisting of one or more data management servers, and realizes the distributed system. Storage mode. After the namespace is grouped through Federation in the distributed file system, multiple master groups are formed, each of which is a data management group, and among the multiple data management groups in the above-mentioned distributed file system, there are and Only one data management group is the default first data management group.

It should also be noted here that when the distributed file system is a Master/Chunk server structure, the data management server Master is used to store the meta (tag) information in the distributed file system, that is, the attribute information of the data, for example ：Data index, web page data title, keywords and other information, Chunk server provides data storage service. In the above method for synchronizing the routing table, the routing table is stored in the Master of the distributed file system, and the above-mentioned data management group is a group of data management groups obtained by grouping the Master.

Step S23, synchronizing the modified routing table data to any one or more other non-predetermined second data management groups to update the routing table in the non-predetermined second data management group.

Among them, a unique first data management group and at least one second data management group are deployed in the distributed file system.

In an alternative embodiment, taking the distributed file system as Alibaba's Pangu as an example, Pangu, as a distributed file system that provides data storage services for Alibaba's cloud computing platform, includes multiple data management groups (for example, Taobao, Taodiandian, Laixiang, etc.), one of the data management groups is preset as PanguDefaultVolume (that is, the first data management group is preset), and the rest of the data management groups in the distributed file system are the second data management groups, for example , Taobao’s data management group is preset as the first data management group, and the rest of the data management groups in the distributed file system (such as Taodiandian, Laixiang, etc.) are all the second data management group; when the routing table is generated When changing, the data management group of Taobao (the default first data management group) receives the latest routing table first, and the latest routing table is obtained from the first data management group After that, the latest routing table is synchronized to the rest of the second data management group, that is, the modification of the routing tables in the remaining data management group (second data management group) is completed through the default first data management group .

After updating the routing tables of multiple data management groups in the distributed file system, the interface of the client (user terminal) can confirm the routing table of each data management group in the distributed file system to ensure that each The routing tables in the data management group are all the latest routing tables.

3 is a schematic diagram of an optional routing table synchronization method according to Embodiment 1 of the present invention. As shown in FIG. 3, when the routing table changes due to user modification and other factors, the first data management in the distributed file system The group first receives the revised new routing table, and then synchronizes the new routing table to the rest of the second data management group in the distributed file system. The above description does not limit the number of second data management groups in practical applications, that is, when there are multiple second data management groups in the distributed file system, the new routing table is synchronized through the first data management group.

Therefore, the solution of the above-mentioned embodiment provided by the present invention solves the problem that the tables in the distributed file system in the prior art are static tables. When the tables in any data management group are changed, the tables in the other data management groups cannot be synchronized. The technical issue of making the change.

In the above-mentioned embodiment of the present invention, when a new second data management group is deployed in the distributed file system, the data management server in the new second data management group is registered to the first data management group, and the The routing table in the first data management group is synchronized to the new second data management group.

In the above steps, the new second data management group is in the original distributed file There is no data management group in the case system. After adding a new second data management group, in order to enable the new second data management group to synchronize the latest routing table in the first data management group, first add the second data management group The data management server in is registered to the default first data management group.

Optionally, the aforementioned data management server may be any data management server Master in the second data management group.

In an alternative embodiment, still taking the above-mentioned distributed file system as Alibaba's Pangu as an example, the preset first data management group is Taobao's data management group by default. If necessary, the Pangu distributed file system Add the data management group that is not in the original distributed file system, and register the data management server of the future data management group to Taobao's data management group. The above-mentioned data management server can be the data management group. The Master of the management group, after the future data management server is registered to Taobao's data management group, it ensures the communication between the data management group and Taobao's data management group, so that the data management composition of the exchange is the distributed file The second data management group that is the same as the rest of the second data management group in the system can perform routing table synchronization steps for the data management group to and from, so that the data management group includes the latest routing table.

The method provided by the foregoing embodiment of the present invention solves the technical problem that the routing table is difficult to be unified after the new data management group is added to the distributed file system.

In the foregoing embodiment of the present invention, the foregoing method further includes the following steps:

Step S23: After any data management group in the distributed file system receives the routing table request initiated by the requesting terminal, it obtains the routing table version information in the requesting terminal from the routing table request.

Step S25, if the version information of the routing table is inconsistent with the version information of the routing table in any data management group, any data management group returns the routing table of the local storage area to the requesting terminal, or the requesting terminal distributes through access The data storage server deployed in the distributed file system to obtain the latest routing table.

Fig. 4 is a schematic diagram of a requesting terminal sending a routing table request to an arbitrary data management group according to Embodiment 1 of the present invention. As shown in FIG. 4, in an alternative embodiment, still taking the distributed file system of Alibaba's Pangu as an example, the preset first data management group is Taobao's data management group by default, and the terminal is requested Send a request to update the routing table to any data management group in the distributed file system, and any data management group obtains the routing table version information of the requesting terminal that initiated the routing table request, and manages the version information and the data that receives the routing table request When the version information of the routing table in the group is inconsistent, the stored routing table is sent to the requesting terminal.

In the above steps, the version information of the routing table may be information used to indicate the new routing table and the old routing table, so that the data management group can identify the latest routing table.

It should be noted here that any of the above-mentioned data management groups can be randomly selected data management groups in the distributed file system, but in the preferred case, any data management group is the first data management group in the distributed file system Since the first data management group is the first data management group in the distributed file system that receives the latest routing table, in order to ensure that the routing table received by the requesting terminal is also the latest routing table in the distributed file system, the preferred method Is any of the above data management groups as the first in a distributed system Data management team.

In the above embodiment of the present invention, in step S23, before obtaining the routing table version information in the requesting terminal from the routing table request, the above method further includes:

Step S231: Determine the permission of the file or directory to be operated according to the information of the file or directory to be operated carried in the routing table request.

In step S233, if the permission is the permitted operation, continue to perform the step of obtaining the routing table version information in the requesting terminal from the routing table request.

In step S235, if the permission is forbidden to operate, then obtaining the latest routing table fails.

In the above steps, the file or directory to be operated in the routing table may be the file name or file directory of the data management group that initiated the routing table request.

In an optional embodiment, still taking the above-mentioned distributed file system as Alibaba's Pangu as an example, the request terminal sends a routing table request to the first data management group in the distributed file system, and the first data management group passes The middle file or directory information of the routing table determines whether the requesting terminal has permission to obtain the latest routing table.

The above-mentioned steps of the present invention set a preset limit for obtaining the routing table in the distributed file system, so that the requesting terminal with the permission to obtain the latest routing table can obtain the latest routing table from the distributed file system.

In another optional embodiment, after the information management group that has received the routing table request obtains the routing table version of the requesting terminal, it is confirmed by comparison that the routing table version of the requesting terminal is the latest routing table version, then it can be considered The request permission of the requesting terminal has been tampered with, so an error is returned to the requesting terminal Code. After receiving the error code, the requesting terminal will retry the request at a preset time until the routing table version of the requesting terminal is not the same as the routing table of the data management group receiving the routing table request.

Fig. 5 is an optional flowchart of sending a routing table request to a data management group according to Embodiment 1 of the present invention, including the following steps:

Step S51, request the terminal to initiate a routing table request.

In the above steps, the request terminal sends a routing table request to any data management group in the distributed file system.

Step S52: Determine the permission of the file or directory to be operated.

If the file or directory to be operated has the operation permission, go to step S53, otherwise go to step S55.

Step S53: It is judged whether the routing table of the requesting terminal is consistent with the routing table of any data management group.

In the above steps, if the requesting terminal is consistent with the routing table of any data management group, then step S55 is entered, otherwise step S54 is entered.

Step S54: Send the routing table to the requesting terminal.

If the version information of the routing table is inconsistent with the version information of the routing table in any data management group, any data management group returns the routing table of the local storage area to the requesting terminal, or the requesting terminal accesses the distributed file system The data storage server deployed in to get the latest routing table.

Step S55, an error code is returned.

In the case that the requesting terminal has not obtained the latest routing table, the error code returned by the data management group is received.

In the foregoing embodiment of the present invention, before any data management group in the distributed file system receives the routing table request initiated by the requesting terminal, the method further includes:

Step S237: After calling the access interface, the request terminal parses to obtain the access information of each data management group in the distributed file system.

In step S239, the requesting terminal sends a routing table request to any data management group. If the requesting terminal fails to obtain the latest routing table from any data management group, it sends the routing table request to other data management groups in the distributed file system. Until the latest routing table is obtained.

Specifically, in the above steps, the access interface may be an API interface, and the access information of each data management group in the distributed file system may be the master information of the data management group.

In an alternative embodiment, still taking the above-mentioned distributed file system as Alibaba's Pangu as an example, the user calls API (Application Programming Interface) and then uses Alibaba's address resolver (address resolver). Analysis tools, such as nuwa) parse out the address information of all data management groups in the distributed file system (for example: tcp/ip address), and after analyzing the address information of all data management groups, randomly send Any data management group sends a request to obtain a routing table. If the request fails, the above process is repeated and retryed.

What needs to be explained here is that in the case that the request of the requesting terminal fails, it is necessary to determine whether the current time is within the preset time range. If the current time is within the preset time range, re-initiate the acquisition to other data management groups. In the case of obtaining the routing table, if the current time has exceeded the preset time range, stop requesting the terminal to continue sending the request for obtaining the routing table.

In the above embodiment of the present invention, in step S237, before the access information of each data management group in the distributed file system is obtained by analyzing, the above method further includes:

Step S2371, request the terminal to analyze and obtain the access information of the data storage server deployed in the distributed file system.

Step S2373: The requesting terminal sends a routing table request to the data storage server. If the requesting terminal fails to obtain the latest routing table from the data storage server, or the routing table stored in the data storage server is not the latest routing table, then the request The terminal enters the step of analyzing the access information of each data management group in the distributed file system.

In an alternative embodiment, the above-mentioned distributed file system is a Master, Slave structure, and the above-mentioned data storage server may be a chunk server. The requesting terminal first sends a routing table request to the chunk server. When the request to the chunk server fails, or chunk When the version information of the routing table in the server is not the latest version, request the terminal to send a routing table request to any data management group in the distributed file system.

In the foregoing embodiment of the present invention, in step S23, before any data management group in the distributed file system receives the routing table request initiated by the requesting terminal, the method further includes:

Step S25: After calling the access interface, the request terminal parses to obtain the address information of the first data management group preset in the distributed file system.

Step S27, request the terminal to send a routing table request to the first data management group Wherein, if the requesting terminal fails to obtain the latest routing table from the first data management group, it will send a routing table request to any other one or more second data management groups in the distributed file system until the latest routing table is obtained .

In an optional embodiment, still taking the above-mentioned distributed file system as Alibaba's Pangu as an example, the preset first data management group is the data management group of Taobao by default, and the terminal is requested to inform Taobao's data management The group sends a request to update the routing table. After receiving the routing table request, Taobao’s data management group obtains the version information of the routing table in the requesting terminal. When the version information is inconsistent, the stored routing table is sent to the requesting terminal.

In the above embodiment of the present invention, the above method further includes: before the address information of the first data management group preset in the distributed file system is obtained by analyzing, the method further includes:

Step S251, request the terminal to analyze and obtain the access information of the data storage server deployed in the distributed file system.

Step S253 requests the terminal to send a routing table request to the data storage server. If the request terminal fails to obtain the latest routing table from the data storage server, or the routing table stored in the data storage server is not the latest routing table, the request terminal Enter the step of analyzing the access information of the first data management group in the distributed file system.

In an optional embodiment, the above-mentioned distributed file system is a Master, Slave structure, and the request terminal first sends a routing table request to the Master. When the request fails, the error code returned by the routing table is When the version information is incorrect, request the terminal to send a routing table request to any data management group in the distributed file system.

Fig. 6 is a flowchart of an optional requesting terminal to send a routing table request to a data storage server according to Embodiment 1 of the present invention, including the following steps:

S61: Obtain access information of the data storage server.

S62: Send a routing table request to the data storage server.

S63, whether the request fails.

S64: Whether the routing table in the data storage server is the latest routing table.

S65: Send the routing table to the requesting terminal.

S66: Parse the access information of the first data management group.

In the above embodiment of the present invention, the above method further includes: in the process of requesting the terminal to initiate the routing table request, if it is detected that the routing table in the first data management group has been modified, revisiting any one of the distributed file systems Data management team to get the latest routing table.

In the process of initiating the routing table request by the requesting terminal, it is detected that the routing table in the first data management group is modified. If you continue to access any of the distributed data management groups, the latest routing table in the possible part of the routing table will be obtained. Therefore, when the above steps of the present invention detect that the routing table in the first data management group is modified during the process of initiating the routing table request by the requesting terminal, it revisits any data management group in the distributed file system so that the requesting terminal can obtain the information obtained Is the latest routing table.

In the above embodiment of the present invention, the above method further includes: distributed files The first data management group and the second data management group deployed in the project system are set with read and write request thresholds, where the read and write request thresholds are used to characterize the maximum access times of the data management server in the corresponding data management group. If the number of accesses to the management server exceeds the threshold for read and write requests, the access to the data management server is terminated.

In an alternative embodiment, taking the online production clusters AY75C and AY88B as an example, when there are 5000 sets of data management servers, the average number of reads and writes is 10,000, and the peak number of reads and writes is 100,000. When the number of Masters is 3, the average data management server needs to withstand a maximum of 33,000 reads and writes, which does not exceed the read and write request thresholds set by the first data management group and the second data management group.

In the above embodiment of the present invention, the above method further includes: performing persistence processing on the data management server included in the first data management group, and after the first data management group has performed the persistence processing, send at least one second data The management group sends a remote program call agreement for modifying the routing table, so that the second data management group performs persistence processing.

In the above steps, the persistent processing of the data management server can be to store the terminal data of the data management server in a device that can be permanently stored.

What needs to be explained here is that since the updates of all routing tables in the distributed file system depend on the update of the first data management group, the second data management group has a great dependence on the first data management group. If the data management group fails, you need to rely on the first data management group to repair it. If the first data management group also fails, you need to After the restoration is completed, the management group relies on the first data management group to perform the restoration, which slows down the restoration progress of the second data management group. Therefore, the data in the second data management group is sent to the second data management group by a remote call protocol. The server performs persistence processing, so that when the first data management group fails, the second data management group also fails. Since the second data management group has performed persistent processing on the routing table, the recovery of the second data management group does not depend on the first data management group, thereby improving the damage repair efficiency of the second data management group.

It should be noted that for the foregoing method embodiments, for the sake of simple description, they are all expressed as a series of action combinations, but those with ordinary knowledge in the art should know that the present invention is not subject to the described sequence of actions. The limitation is because according to the present invention, certain steps can be performed in other order or simultaneously. Secondly, those with ordinary knowledge in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present invention.

Through the description of the above embodiments, those with ordinary knowledge in the field can clearly understand that the task transmission method based on the sub-library and sub-table according to the above embodiments can be implemented by means of software plus the necessary general hardware platform. Of course, hardware can also be used, but in many cases the former is a better implementation. Based on this understanding, the technical solution of the present invention essentially or the part that contributes to the existing technology can be embodied in the form of a software product, which is stored in a storage medium (such as ROM/RAM, magnetic disk, optical disk) ), including several instructions to make a terminal device (can be a mobile phone, computer, server, or network device Etc.) Perform the methods described in the various embodiments of the present invention.

Example 2

According to an embodiment of the present invention, a device for synchronizing a routing table is also provided. As shown in FIG. 7, the device includes: a first obtaining module 120 and a synchronization module 122.

The first obtaining module 120 is used to obtain the modified routing table data if it is detected that the routing table in the first data management group is modified; the synchronization module 122 is used to synchronize the modified routing table data to other Any one or more non-default second data management groups to update the routing table in the non-default second data management group; wherein, a unique first data management group and at least one first data management group are deployed in the distributed file system 2. Data management team.

In an alternative embodiment, the preset first data management group may be the data management group storing the most important data in the distributed file system, or the data management group with the largest data storage capacity, but in practical applications, Specifically, which data management group in the distributed file system is used as the default first data management group is not limited here.

What needs to be explained here is that the main purpose of the default first data management group in the routing table synchronization method is to detect the change of the routing table in the distributed file system, that is, whether the routing table in the distributed file system is Change, and in the case of detecting a change in the routing table, obtain the latest routing table after the change.

It should also be noted here that in a distributed file system, the Through the Federation, the namespace of the data is distributed, so that it forms a federation, and then realizes the storage mode in the distributed system. After the distributed file system groups the namespaces through the Federation, it will form There are multiple master groups, each of which is a data management group, and among the multiple data management groups in the above-mentioned distributed file system, there is one and only one data management group as the default first data management group.

It should also be noted here that when the distributed file system is a Master/Chunk server structure, the Master is used to store meta (tag) information in the distributed file system, that is, the attribute information of the data, such as the index of the data , Web page data titles, keywords and other information, Chunk server provides data storage services. In the above method for synchronizing the routing table, the routing table is stored in the Master of the distributed file system, and the above-mentioned data management group is a group of data management groups obtained by grouping the Master.

In an alternative embodiment, taking the distributed file system as Alibaba's Pangu as an example, Pangu, as a distributed file system that provides data storage services for Alibaba's cloud computing platform, includes multiple data management groups (for example, Taobao, Taodiandian, Laixiang, etc.), one of the data management groups is preset as PanguDefaultVolume (default first data management group), and the rest of the data management groups in the distributed file system are second data management groups, for example, By default, Taobao’s data management group is the first data management group, and the rest of the data management groups in the distributed file system (such as Taodiandian, Laixiang, etc.) are the second data management group; when the routing table is changed When, first Taobao’s data management group (the default first data management group) receives the latest routing table, and after the first data management group obtains the latest routing table, the latest routing table is synchronized to the rest of the second data management In the group, that is, the modification of the routing table in the remaining data management group (the second data management group) is completed through the preset first data management group.

After updating the routing tables of multiple data management groups in the distributed file system, the interface of the client (user terminal) can confirm the routing table of each data management group in the distributed file system to ensure that each The routing tables in the data management group are all the latest routing tables.

Therefore, the solution of the above-mentioned embodiment provided by the present invention solves the problem that the tables in the distributed file system in the prior art are static tables. When the tables in any data management group are changed, the tables in the other data management groups cannot be synchronized. The technical issue of making the change.

In the above-mentioned embodiment of the present invention, as shown in FIG. 8, the above-mentioned device further includes: a registration module 80, The registration module 80 is used for registering the data management server in the new second data management group to the first data management group when the new second data management group is deployed in the distributed file system, and the first data The routing table in the management group is synchronized to the new second data management group.

In the above device, the new second data management group does not exist in the original distributed file system. After adding the new second data management group, in order to enable the new second data management group to synchronize the first data The latest routing table in the management group first registers the data management server in the second data management group to the preset first data management group.

Optionally, the aforementioned data management server may be the Master of the second data management group.

In an alternative embodiment, still taking the above-mentioned distributed file system as Alibaba's Pangu as an example, the preset first data management group is Taobao's data management group by default. If necessary, the Pangu distributed file system Add the data management group that is not in the original distributed file system, and register the data management server of the future data management group to Taobao's data management group. The above-mentioned data management server can be the data management group. The Master of the management group, after the future data management server is registered to Taobao's data management group, it ensures the communication between the data management group and Taobao's data management group, so that the data management composition of the exchange is the distributed file The second data management group that is the same as the rest of the second data management group in the system can perform routing table synchronization steps for the data management group to and from, so that the data management group includes the latest routing table.

The method provided by the foregoing embodiment of the present invention solves the technical problem that the routing table is difficult to be unified after the new data management group is added to the distributed file system.

In the above-mentioned embodiment of the present invention, as shown in FIG. 9, the above-mentioned device further includes: a second obtaining module 90 and a returning module 92, The second obtaining module 90 is used to obtain the routing table version information in the requesting terminal from the routing table request after any data management group in the distributed file system receives the routing table request initiated by the requesting terminal; return module 92 If the version information of the routing table is inconsistent with the version information of the routing table in any data management group, any data management group returns the routing table of the local storage area to the requesting terminal, or the requesting terminal Get the latest routing table by accessing the data storage server deployed in the distributed file system.

As shown in FIG. 4, in an alternative embodiment, still taking the distributed file system of Alibaba's Pangu as an example, the preset first data management group is Taobao's data management group by default, and the terminal is requested Send a request to update the routing table to any data management group in the distributed file system, and any data management group obtains the routing table version information of the requesting terminal that initiated the routing table request, and manages the version information and the data that receives the routing table request When the version information of the routing table in the group is inconsistent, the stored routing table is sent to the requesting terminal.

In the above device, the version information of the routing table may be information used to indicate the new routing table and the old routing table, so that the data management group can identify the latest routing table.

It should be noted here that any of the above-mentioned data management groups can be randomly selected data management groups in the distributed file system, but in the preferred case, any data management group is the first data management group in the distributed file system Since the first data management group is the first data management group in the distributed file system that receives the latest routing table, in order to ensure that the routing table received by the requesting terminal is also the latest routing table in the distributed file system, the preferred method Any one of the above-mentioned data management groups is the first data management group in the distributed system.

In the foregoing embodiment of the present invention, as shown in FIG. 10, the foregoing apparatus further includes: a determination module 100, an execution module 101, and a third acquisition module 103.

The determining module 100 is used to determine the permission of the file or directory to be operated according to the information of the file or directory to be operated carried in the routing table request; the execution module 101 is used to continue the execution if the permission is allowed to operate The step of obtaining the version information of the routing table in the requesting terminal from the routing table request; the third obtaining module 103 is configured to fail to obtain the latest routing table if the permission is forbidden to operate.

In the above embodiment of the present invention, the first data management group and the second data management group deployed in the distributed file system are set with read and write request thresholds, where the read and write request thresholds are used to characterize the data in the corresponding data management group The maximum number of visits to the management server. If the number of visits to the data management server exceeds the threshold for read and write requests, the access to the data management server is terminated.

In the above-mentioned embodiment of the present invention, the persistence processing module is used to perform persistence processing on the data management server included in the first data management group, and after the first data management group executes the persistence processing, it sends at least A second data management group sends a remote program call protocol for modifying the routing table, so that the second data management group performs persistence processing.

In the above-mentioned embodiment of the present invention, the above-mentioned device further includes: a parsing module for requesting the terminal to parse and obtain the address information of the first data management group preset in the distributed file system after invoking the access interface; and a third sending module The group requesting terminal sends a routing table request to the first data management group, where if the requesting terminal fails to obtain the latest routing table from the first data management group, it will send to any one or more of the other second data management in the distributed file system The group sends routing table requests until the latest routing table is obtained.

In the above-mentioned embodiment of the present invention, the above-mentioned device further includes: the second processing module is used to request the terminal to analyze and obtain the access information of the data storage server deployed in the distributed file system; the fourth sending module is used to request the terminal to send the data to the data storage server. The storage server sends a routing table request. If the requesting terminal fails to obtain the latest routing table from the data storage server, or the routing table stored in the data storage server is not the latest routing table, the requesting terminal enters the analysis distributed file system Steps of accessing information in the first data management group.

In the above-mentioned embodiment of the present invention, the above-mentioned device further includes: an access module for re-visiting the distributed file if it is detected that the routing table in the first data management group has been modified during the process of the requesting terminal initiating the routing table request Any data management group in the system to obtain the latest routing table.

In the above embodiment of the present invention, the first data management group and the second data management group deployed in the distributed file system are provided with read and write request thresholds, where the read and write request thresholds are used to characterize the corresponding data management group. The maximum number of visits to the data management server. If the number of visits to the data management server exceeds the read and write request threshold, the access to the data management server is terminated.

In the above-mentioned embodiment of the present invention, the above-mentioned device further includes: a persistence module, configured to perform persistence processing on the data management server included in the first data management group, after the first data management group executes the persistence processing , Send the remote program call protocol for modifying the routing table to at least one second data management group, so that the second data management group executes the support Long treatment.

Example 3

According to an embodiment of the present invention, there is also provided a routing table synchronization system. As shown in FIG. 11, the system includes: a first data management server 110 and one or more second data management servers 112, wherein the first The data management server 110 is configured to, if it is detected that the routing table in the local storage area is modified, obtain the modified routing table data, and synchronize the modified routing table data, where the first data management server is Any server included in the preset first data management group in the distributed file system; one or more second data management servers 112, which have a communication relationship with the first data management server, and are used to receive the first data management The server synchronizes the modified routing table data to update the routing table in the second data management group, where the second data management server is any of the non-default second data management groups included in the distributed file system A server.

In an optional embodiment, the preset first data management group may be any data management group in the distributed file system. In the routing table synchronization method, the first data management group is used to implement The latest routing table is synchronized to other data management groups deployed in the distributed file system.

It should also be noted here that in the distributed file system involved in the present invention (such as the Pangu pangu distributed file system), the formation of a data management group can usually be realized by introducing the Federation mode, that is, the Federation The namespace for the data used in Name space) carries out distributed division mode and introduces the mode of data management group, so that it forms a set group consisting of one or more data management servers, and realizes the storage mode in the distributed system. After the namespace is grouped through Federation in the distributed file system, multiple master groups are formed, each of which is a data management group, and among the multiple data management groups in the above-mentioned distributed file system, there are and Only one data management group is the default first data management group.

It should also be noted here that when the distributed file system is a Master/Chunk server structure, the data management server Master is used to store the meta (tag) information in the distributed file system, that is, the attribute information of the data, for example ：Data index, web page data title, keywords and other information, Chunk server provides data storage service. In the above method for synchronizing the routing table, the routing table is stored in the Master of the distributed file system, and the above-mentioned data management group is a group of data management groups obtained by grouping the Master.

In an alternative embodiment, taking the distributed file system as Alibaba's Pangu as an example, Pangu, as a distributed file system that provides data storage services for Alibaba's cloud computing platform, includes multiple data management groups (for example, Taobao, Taodiandian, Laixiang, etc.), one of the data management groups is preset as PanguDefaultVolume (default first data management group), and the rest of the data management groups in the distributed file system are second data management groups, for example, By default, Taobao’s data management group is the first data management group, and the rest of the data management groups in the distributed file system (such as Taodiandian, Laixiang, etc.) are the second data management group; when Taobao’s data Management group (pre The first data management group) detects that the routing table in the local storage area is modified, the Taobao data management group receives the latest routing table, and synchronizes the received latest routing table to the remaining second data management In the group, the remaining second data management group receives the modified routing table synchronized by Taobao’s data management group, so that the local routing table stored in one or more second data management groups is changed to the modified routing table, That is, the modification of the routing tables in the remaining data management group (the second data management group) is completed through the preset first data management group.

After updating the routing tables of multiple data management groups in the distributed file system, the interface of the client (user terminal) can confirm the routing table of each data management group in the distributed file system to ensure that each The routing tables in the data management group are all the latest routing tables.

As shown in Figure 3, when the routing table changes due to user modification and other factors, the first data management group in the distributed file system first receives the modified routing table, and then synchronizes the modified routing table to the distributed file The rest of the second data management group in the system. The foregoing embodiment does not limit the number of second data management groups in practical applications, that is, when there are multiple second data management groups in the distributed file system, the new routing table is synchronized through the first data management group.

Therefore, the solution of the above-mentioned embodiment provided by the present invention solves the problem that the tables in the distributed file system in the prior art are static tables. When the tables in any data management group are changed, the tables in the other data management groups cannot be synchronized. The technical issue of making the change.

In the foregoing embodiment of the present invention, the foregoing system may further include: request The terminal has a communication relationship with the first data management server and one or more second data management servers, and is used to send a routing table request to any data management group in the distributed file system, and receive any data management group return Or, request the terminal to obtain the latest routing table by accessing the data storage server deployed in the distributed file system.

In the above system, when the requesting terminal sends a routing table request to any data management group in the distributed file system, the data management group receiving the routing table request can obtain the routing table version information in the routing table request. When the routing table version information in the routing table is inconsistent with the routing table version information of the data management group receiving the routing table request, the data management group receiving the routing table request returns the routing table in the local storage area to the requesting terminal.

In an alternative embodiment, still taking the above-mentioned distributed file system as Pangu of Alibaba as an example, the preset first data management group is the data management group of Taobao by default, and the terminal is requested to send information to the distributed file system. Any data management group sends a request to update the routing table, and any data management group obtains the routing table version information of the requesting terminal that initiated the routing table request. When the version information is inconsistent, the stored routing table is sent to the requesting terminal.

In the above embodiment, the version information of the routing table may be information used to indicate the new routing table and the old routing table, so that the data management group can identify the latest routing table.

It should be noted here that any of the above-mentioned data management groups can be randomly selected data management groups in the distributed file system, but because the first data management group is the first to receive the latest routing table data in the distributed file system Therefore, in order to ensure that the routing table received by the requesting terminal is also the latest routing table in the distributed file system, the preferred method is that any one of the above-mentioned data management groups is the first data management group in the distributed system.

In the above embodiment of the present invention, the requesting terminal is also used to parse the access information of each data management group in the distributed file system after invoking the access interface, and send a routing table request to any data management group; wherein, If the requesting terminal fails to obtain the latest routing table from any data management group, it will send a routing table request to other data management groups in the distributed file system until the latest routing table is obtained.

Specifically, in the aforementioned system, the aforementioned access interface can be an API (Application Programming Interface) interface, and the access information of each data management group in the distributed file system can be the master information of the data management group.

In the above-mentioned embodiment of the present invention, the request terminal is also used to analyze and obtain access to the data storage server deployed in the distributed file system when the access information of any data management group in the distributed file system cannot be obtained. Information, and send a routing table request to the data storage server; among them, if the requesting terminal fails to obtain the latest routing table from the data storage server, or the routing table stored in the data storage server is not the latest routing table, the request terminal starts Analyze the access information of each data management group in the distributed file system.

In an alternative embodiment, the aforementioned distributed file system is a Master, Slave structure, any of the aforementioned data management groups can be a Master, and the aforementioned data storage server can be a chunk server. Ask the terminal to parse the access information of the chunk server and send a routing table request to the chunk server when it cannot parse the access information of Maste.

In the above-mentioned embodiment of the present invention, the requesting terminal is also used to analyze and obtain the address information of the first data management group preset in the distributed file system after invoking the access interface, and send a routing table request to the first data management group ; Wherein, if the requesting terminal fails to obtain the latest routing table from the first data management group, it sends a routing table request to any other one or more second data management groups in the distributed file system until the latest routing table is obtained.

In an alternative embodiment, still taking the above-mentioned distributed file system as Alibaba's Pangu as an example, after the user calls the API, the user uses Alibaba's address resolver (address resolution tool, such as nuwa) to resolve the distributed file system. The address information of all data management groups in the file system (for example: tcp/ip address), after analyzing the address information of all data management groups, randomly send a request to any one of the data management groups to obtain the routing table Request, if the request fails, repeat the above process and try again until the latest routing table is successfully obtained.

What needs to be explained here is that in the case that the request of the request terminal fails, it is necessary to determine whether the current time is within the preset time range. If the current time is within the preset time range, re-initiate the acquisition route to other data management groups In the case of a table, if the current time has exceeded the preset time range, stop requesting the terminal to continue sending the request for obtaining the routing table.

In the above-mentioned embodiment of the present invention, the requesting terminal is also used to obtain access information of the first data management group preset in the distributed file system if it cannot be parsed. In the case of, the access information of the data storage server deployed in the distributed file system is parsed, and a routing table request is sent to the data storage server; among them, if the request terminal fails to obtain the latest routing table from the data storage server, or If the routing table stored in the data storage server is not the latest routing table, the terminal is requested to start analyzing the access information of the first data management group in the distributed file system.

In an optional embodiment, the distributed file system is a Master, Slave structure, the preset first data management group may be the Master in the preset first data management group, and the data storage server may be Chunk server, when the requesting terminal cannot parse the access information of Maste, it parses the access information of the chunk server and sends a routing table request to the chunk server.

In the above embodiment of the present invention, the requesting terminal is also used for initiating the routing table request, if it is detected that the routing table in the first data management group has been modified, then revisit any data management group in the distributed file system To get the latest routing table.

In the process of initiating the routing table request by the requesting terminal, it is detected that the routing table in the first data management group has been modified. If you continue to access any data management group in the distributed file system, the latest route in the possible part of the routing table will be obtained. Therefore, when the above steps of the present invention detect that the routing table in the first data management group is modified during the process of initiating the routing table request by the requesting terminal, it revisits any data management group in the distributed file system so that the requesting terminal can obtain What you get is the latest routing table.

Example 4

The embodiments of the present invention may provide a computer terminal, and the computer terminal may be any computer terminal device in a computer terminal group. Optionally, in this embodiment, the above-mentioned computer terminal can also be replaced with a terminal device such as a mobile terminal.

Optionally, in this embodiment, the above-mentioned computer terminal may be located in at least one network device among a plurality of network devices in a computer network.

In this embodiment, the aforementioned computer terminal can execute the code of the following steps in the routing table synchronization method: if it is detected that the routing table in the preset first data management group is modified, obtain the modified routing table data; The modified routing table data is synchronized to any one or more non-default second data management group to update the routing table in the non-default second data management group; among them, the only one deployed in the distributed file system A first data management group and at least one second data management group.

Optionally, FIG. 12 is a structural block diagram of a computer terminal according to an embodiment of the present invention. As shown in FIG. 12, the computer terminal A may include: one or more (only one shown in the figure) processor 1201, memory 1203, transmission device 1205, request terminal 1207, and data storage server 1209.

Among them, the memory can be used to store software programs and modules, such as the routing table synchronization method and the program instructions/modules corresponding to the device in the embodiment of the present invention. The processor runs the software programs and modules stored in the memory, So as to perform various functional applications and data processing, that is, to achieve the above-mentioned synchronization method of the routing table. The memory can include high-speed random memory, but also It may include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory may further include a memory provided remotely with respect to the processor, and these remote storages may be connected to the terminal A via a network. Examples of the aforementioned networks include, but are not limited to, the Internet, corporate intranet, local area network, mobile communication network, and combinations thereof.

The processor can call the information and application programs stored in the memory through the transmission device to perform the following steps: if it is detected that the routing table in the first data management group has been modified, obtain the modified routing table data; The subsequent routing table data is synchronized to any other one or more non-default second data management group to update the routing table in the non-default second data management group; among them, the only second data management group is deployed in the distributed file system A data management group and at least one second data management group.

Optionally, the above-mentioned processor may also execute the code of the following steps: when a new second data management group is deployed in the distributed file system for the synchronization method of the routing table, the synchronization method of the routing table is changed to a new second data management group Register the data management server in the routing table synchronization method to the first data management group, and synchronize the routing table in the first data management group of the routing table synchronization method to the synchronization method of the routing table new second data management group middle.

Optionally, the above-mentioned processor may also execute the program code of the following steps: after any data management group in the distributed file system of the routing table synchronization method receives the routing table request initiated by the requesting terminal, the synchronization method from the routing table The synchronization method of obtaining the routing table in the routing table request requests the routing table version information in the terminal; if the routing table synchronization method is the routing table version data If the version information of the routing table in any data management group is inconsistent with the synchronization method of the routing table, any data management group of the routing table synchronization method requests the terminal to return the routing table in the local storage area to the synchronization method of the routing table. Or, the synchronization method of the routing table requests the terminal to obtain the latest routing table by accessing the data storage server deployed in the distributed file system of the synchronization method of the routing table.

Optionally, the above-mentioned processor may also execute the program code of the following steps: according to the information of the file or directory to be operated carried in the routing table request for the synchronization method of the routing table, determine the file or directory to be operated by the synchronization method of the routing table Permission; if the synchronization method permission of the routing table is allowed to operate, continue to execute the step of obtaining the routing table synchronization method requesting the routing table version information in the terminal from the routing table synchronization method routing table request; if the routing table synchronization If the method permission is forbidden to operate, the synchronization method for obtaining the latest routing table of the routing table fails.

Optionally, the above-mentioned processor may also execute the program code of the following steps: the synchronization method of the routing table requests the terminal to parse the access information of each data management group in the distributed file system after invoking the access interface. ; The routing table synchronization method requests the terminal to send a routing table synchronization method routing table request to any data management group The synchronization method of obtaining the routing table The latest routing table fails, the synchronization method routing table request of the routing table is sent to the other data management group in the distributed file system of the synchronization method of the routing table until the synchronization method of the routing table is obtained the latest routing table .

Optionally, the above-mentioned processor may also execute the program code of the following steps: request the terminal to parse and obtain the access information of the data storage server deployed in the distributed file system; request the terminal to send a routing table request to the data storage server, where, if If the requesting terminal fails to obtain the latest routing table from the data storage server, or the routing table stored in the data storage server is not the latest routing table, request the terminal to enter and analyze the access information of each data management group in the distributed file system step.

Optionally, the above-mentioned processor may also execute the program code of the following steps: before any data management group in the distributed file system receives the routing table request initiated by the requesting terminal, the method further includes: requesting the terminal to call the access interface after , Parse and obtain the address information of the first data management group preset in the distributed file system; request the terminal to send a routing table request to the first data management group, wherein, if the request terminal obtains the latest routing table from the first data management group If it fails, send a routing table request to any other one or more second data management groups in the distributed file system until the latest routing table is obtained.

Optionally, the above-mentioned processor may also execute the program code of the following steps: before the address information of the first data management group preset in the distributed file system is obtained by parsing, the method further includes: requesting the terminal to analyze and obtain the distributed file system Access information of the data storage server deployed in the data storage server; the requesting terminal sends a routing table request to the data storage server, where if the requesting terminal fails to obtain the latest routing table from the data storage server, or the routing table stored in the data storage server If it is not the latest routing table, request the terminal to enter the step of parsing the access information of the first data management group in the distributed file system Sudden.

Optionally, the above-mentioned processor may also execute the program code of the following steps: in the process of requesting the terminal to initiate the routing table synchronization method routing table request, if the routing table synchronization method is detected, the routing table in the first data management group is modified , Then revisit any data management group in the distributed file system of the synchronization method of the routing table to obtain the latest routing table of the synchronization method of the routing table.

Optionally, the above-mentioned processor may also execute the code of the following steps: the synchronization method of the routing table, the synchronization method of the routing table deployed in the distributed file system, the first data management group and the synchronization method of the routing table, the second data management group setting There is a read and write request threshold. The read and write request threshold of the synchronization method of the routing table is used to characterize the maximum access times of the data management server in the corresponding data management group. If the synchronization method of the routing table has more than the access times of the data management server The synchronization method of the routing table reads and writes the request threshold, then the access to the synchronization method data management server of the routing table is terminated.

Optionally, the above-mentioned processor may also execute the code of the following steps: the data management server included in the first data management group performs the persistence processing, and after the synchronization method of the routing table, the first data management group performs the persistence processing. , To send a remote program call protocol for modifying the routing table to the second data management group of the synchronization method of at least one routing table, so that the second data management group of the synchronization method of the routing table executes persistence processing.

Therefore, the solution of the above-mentioned embodiment provided by the present invention solves the problem that the tables in the distributed file system in the prior art are static tables. When the tables in any data management group are changed, the tables in the other data management groups cannot be the same. Technical issues for step-by-step changes.

Those with ordinary knowledge in the field can understand that the structure shown in Figure 12 is only for illustration, and the computer terminal can also be a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a palmtop computer, and a mobile Internet device (Mobile Internet Devices, MID), PAD and other terminal devices. FIG. 12 does not limit the structure of the above electronic device. For example, the computer terminal 10 may also include more or fewer components (such as a network interface, a display device, etc.) than that shown in FIG. 12, or may have a different configuration from that shown in FIG.

Those with ordinary knowledge in the art can understand that all or part of the steps in the various methods of the above-mentioned embodiments can be completed by instructing the hardware related to the terminal device through a program. The program can be stored in a computer-readable storage medium. The medium may include: a flash memory disk, a read-only memory (Read-Only Memory, ROM), a random access memory (Random Access Memory, RAM), a floppy disk or an optical disk, etc.

Example 5

The embodiment of the present invention also provides a storage medium. Optionally, in this embodiment, the storage medium described above may be used to store the program code executed by the task transmission method based on sub-library and sub-table provided in the first embodiment.

Optionally, in this embodiment, the above-mentioned storage medium may be located in any computer terminal in the computer terminal group in the computer network, or located in any mobile terminal in the mobile terminal group.

Optionally, in this embodiment, the storage medium is configured to store The code that executes the following steps: If it is detected that the routing table in the first data management group is modified by default, obtain the modified routing table data; synchronize the modified routing table data to any one or more other non-prepared A second data management group is set to update the routing table in the non-preset second data management group; wherein, a unique first data management group and at least one second data management group are deployed in the distributed file system.

Optionally, in this embodiment, the storage medium is configured to store the code used to perform the following steps: when the routing table synchronization method deploys a new second data management group in the distributed file system, the routing table Synchronization method The data management server in the new second data management group is registered to the first data management group of the synchronization method of the routing table, and the routing table in the first data management group is synchronized to the routing table of the synchronization method of the routing table The synchronization method is in the new second data management group.

Optionally, in this embodiment, the storage medium is configured to store the code used to perform the following steps: any data management group in the distributed file system of the routing table synchronization method receives the routing table initiated by the requesting terminal After the request, obtain the synchronization method of the routing table from the routing table request of the routing table and request the version information of the routing table in the terminal; if the synchronization method of the routing table, the routing table version information and the synchronization method of the routing table are in any data management group If the version information of the routing table in the routing table is inconsistent, any data management group of the routing table synchronization method requests the terminal to return the routing table in the local storage area to the routing table synchronization method request terminal, or the routing table synchronization method requests the terminal to access the routing Table synchronization method The data storage server deployed in the distributed file system obtains the latest routing table.

Optionally, in this embodiment, the storage medium is configured to store the code used to perform the following steps: according to the routing table synchronization method routing table request, the information of the file or directory to be operated carried in the routing table is determined, The synchronization method is the permission of the file or directory to be operated; if the synchronization method permission of the routing table is allowed to operate, continue to execute the synchronization method of obtaining the routing table from the routing table synchronization method routing table request. Request the routing table version in the terminal Information step; if the synchronization method permission of the routing table is forbidden operation, then obtaining the latest routing table of the synchronization method of the routing table fails.

Optionally, in this embodiment, the storage medium is configured to store the code used to perform the following steps: the synchronization method of the routing table requests the terminal to parse the synchronization method of the routing table after invoking the access interface in the distributed file system The access information of each data management group; the routing table synchronization method request terminal sends a routing table synchronization method routing table request to any data management group of the routing table synchronization method, where, if the routing table synchronization method requests the terminal from the routing table Table synchronization method Any data management group obtains the synchronization method of the routing table. If the latest routing table fails, the synchronization method of the routing table is sent to other data management groups in the distributed file system. The routing table request is sent until it is obtained. The synchronization method of the routing table is the latest routing table.

Optionally, in this embodiment, the storage medium is set to store: request the terminal to parse and obtain access information of the data storage server deployed in the distributed file system; request the terminal to send a routing table request to the data storage server, where, If the request terminal fails to obtain the latest routing table from the data storage server, or the routing table stored in the data storage server is not the latest In the routing table, the terminal is requested to enter the step of parsing the access information of each data management group in the distributed file system.

Optionally, in this embodiment, the storage medium is set to store: before any data management group in the distributed file system receives the routing table request initiated by the requesting terminal, the method further includes: requesting the terminal to call the access interface After that, the address information of the first data management group preset in the distributed file system is obtained by analysis; the request terminal sends a routing table request to the first data management group, wherein, if the request terminal obtains the latest route from the first data management group If the table fails, a routing table request is sent to any other one or more second data management groups in the distributed file system until the latest routing table is obtained.

Optionally, in this embodiment, the storage medium is set to store: request the terminal to parse and obtain access information of the data storage server deployed in the distributed file system; request the terminal to send a routing table request to the data storage server, where, If the requesting terminal fails to obtain the latest routing table from the data storage server, or the routing table stored in the data storage server is not the latest routing table, the requesting terminal enters to analyze the access information of each data management group in the distributed file system A step of.

Optionally, in this embodiment, the storage medium is set to store: before any data management group in the distributed file system receives the routing table request initiated by the requesting terminal, the method further includes: requesting the terminal to call the access interface After that, the address information of the first data management group preset in the distributed file system is obtained by analysis; the request terminal sends a routing table request to the first data management group, wherein, if the terminal is requested to obtain the latest information from the first data management group If the routing table fails, a routing table request is sent to any other one or more second data management groups in the distributed file system until the latest routing table is obtained.

Optionally, in this embodiment, the storage medium is set to store: before the address information of the first data management group preset in the distributed file system is obtained by analyzing, the method further includes: requesting the terminal to analyze and obtain the distributed file Access information of the data storage server deployed in the system; the requesting terminal sends a routing table request to the data storage server, where if the requesting terminal fails to obtain the latest routing table from the data storage server, or the route stored in the data storage server If the table is not the latest routing table, the terminal is requested to enter the step of parsing the access information of the first data management group in the distributed file system.

Optionally, in this embodiment, the storage medium is set to store: request the terminal to parse and obtain access information of the data storage server deployed in the distributed file system; request the terminal to send a routing table request to the data storage server, where, If the requesting terminal fails to obtain the latest routing table from the data storage server, or the routing table stored in the data storage server is not the latest routing table, the requesting terminal enters to analyze the access information of the first data management group in the distributed file system A step of.

Optionally, in this embodiment, the storage medium is set to store: during the routing table synchronization method requesting the terminal to initiate the routing table synchronization method routing table request process, if the routing table synchronization method is detected, the first data management If the routing table in the group is modified, the synchronization method of the routing table is revisited to any data management group in the distributed file system to obtain the routing table Synchronize the latest routing table.

Optionally, in this embodiment, the storage medium is set to store: the synchronization method of the routing table, the synchronization method of the routing table deployed in the distributed file system, the first data management group and the synchronization method of the routing table, the second data management group A read and write request threshold is set. The read and write request threshold of the synchronization method of the routing table is used to represent the maximum access times of the data management server in the corresponding data management group. If the synchronization method of the routing table is the access times of the data management server If the synchronization method read and write request threshold of the routing table is exceeded, the access to the synchronization method data management server of the routing table is terminated.

Optionally, in this embodiment, the storage medium is set to store: the data management server included in the first data management group of the synchronization method of the routing table is persisted, and the first data management of the synchronization method of the routing table is After the group performs the persistence processing, the remote program call protocol for modifying the routing table is sent to the second data management group of the synchronization method of at least one routing table, so that the second data management group of the synchronization method of the routing table executes the persistence processing.

The sequence numbers of the foregoing embodiments of the present invention are only for description, and do not represent the superiority or inferiority of the embodiments.

In the above-mentioned embodiments of the present invention, the description of each embodiment has its own focus. For parts that are not described in detail in an embodiment, reference may be made to related descriptions of other embodiments.

In the several embodiments provided by the present invention, it should be understood that the disclosed technical content can be implemented in other ways. Wherein, the device embodiment described above is only illustrative, for example, the division of the unit, It is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units or elements may be combined or integrated into another system, or some features may be ignored or not implemented. In addition, the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, units or modules, and may be in electrical or other forms.

The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed to multiple network units. . Some or all of the units may be selected according to actual needs to achieve the objectives of the solutions of the embodiments.

In addition, the functional units in the various embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit. The above-mentioned integrated unit can be realized either in the form of hardware or in the form of software functional unit.

If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on this understanding, the technical solution of the present invention essentially or the part that contributes to the existing technology or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium. It includes a number of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the method described in each embodiment of the present invention. And the aforementioned storage The medium includes: U disk, Read-Only Memory (ROM), Random Access Memory (RAM), mobile hard disk, floppy disk or CD-ROM and other media that can store program codes.

The above are only the preferred embodiments of the present invention. It should be pointed out that for those skilled in the art, without departing from the principle of the present invention, several improvements and modifications can be made. These improvements and Retouching should also be regarded as the protection scope of the present invention.

Claims (22)

A method for synchronizing a routing table, which is characterized in that it includes: if it is detected that the routing table in the first data management group preset is modified, obtaining the modified routing table data; and synchronizing the modified routing table data to other Any one or more non-default second data management groups to update the routing table in the non-default second data management group; wherein, the only first data management group and at least one first data management group are deployed in the distributed file system The second data management group, wherein the first data management group and the second data management group deployed in the distributed file system are set with read and write request thresholds, wherein the read and write request thresholds are used to characterize corresponding data The maximum number of accesses to the data management server in the management group. If the number of accesses of the data management server exceeds the threshold of the read and write request, the access to the data management server is terminated. The method according to claim 1, wherein when a new second data management group is deployed in the distributed file system, the data management server in the new second data management group is registered to the first data management And synchronize the routing table in the first data management group to the new second data management group. The method according to claim 1 or 2, wherein the method further comprises: after any one of the data management groups in the distributed file system receives the routing table request initiated by the requesting terminal, obtaining the routing table request from the routing table request Request the routing table version information in the terminal; In the case that the version information of the routing table is inconsistent with the version information of the routing table in any one of the data management groups, the any one of the data management groups returns the routing table of the local storage area to the requesting terminal, or the requesting terminal passes through Visit the data storage server deployed in the distributed file system to obtain the latest routing table. The method according to claim 3, wherein, before obtaining the routing table version information in the requesting terminal from the routing table request, the method further includes: according to the file or directory to be operated carried in the routing table request Information, determine the permission of the file or directory to be operated; if the permission is allowed, continue to perform the step of obtaining the routing table version information in the requesting terminal from the routing table request; if the permission is forbidden Operation, then obtaining the latest routing table fails. The method according to claim 3, wherein, before any data management group in the distributed file system receives the routing table request initiated by the requesting terminal, the method further includes: after the requesting terminal invokes the access interface, parsing Obtain the access information of each data management group in the distributed file system; the requesting terminal sends the routing table request to any one of the data management groups, wherein, if the requesting terminal obtains the latest data from any one of the data management groups If the routing table fails, the routing table request is sent to other data management groups in the distributed file system until the latest routing table is obtained. The method according to claim 5, wherein the score is obtained after parsing Before the access information of each data management group in the distributed file system, the method further includes: the requesting terminal parses to obtain the access information of the data storage server deployed in the distributed file system; and the requesting terminal stores the data to the data The server sends the routing table request, where, if the requesting terminal fails to obtain the latest routing table from the data storage server, or the routing table stored in the data storage server is not the latest routing table, the requesting terminal Enter the step of analyzing the access information of each data management group in the distributed file system. The method according to claim 3, wherein, before any data management group in the distributed file system receives the routing table request initiated by the requesting terminal, the method further includes: after the requesting terminal invokes the access interface, parsing Obtain the address information of the preset first data management group in the distributed file system; the requesting terminal sends the routing table request to the first data management group, wherein, if the requesting terminal is from the first data management group If obtaining the latest routing table fails, the routing table request is sent to any other one or more second data management groups in the distributed file system until the latest routing table is obtained. The method according to claim 7, wherein, before the address information of the preset first data management group in the distributed file system is obtained by analyzing, the method further includes: the requesting terminal analyzes to obtain the distributed file system Access information of the data storage server deployed in; The requesting terminal sends the routing table request to the data storage server, where if the requesting terminal fails to obtain the latest routing table from the data storage server, or the routing table stored in the data storage server is not the latest Routing table, the requesting terminal enters the step of analyzing the access information of the first data management group in the distributed file system. The method according to claim 3, wherein, in the process of the requesting terminal initiating the routing table request, if it is detected that the routing table in the first data management group has been modified, revisiting the distributed file system Any data management group or the first data management group can obtain the latest routing table. The method according to claim 1, wherein the data management server included in the first data management group is subjected to persistence processing, and after the first data management group executes the persistence processing, it sends at least one second data management group to at least one of the second data management servers. The data management group sends a remote program call protocol for modifying the routing table, so that the second data management group executes persistence processing. A device for synchronizing a routing table is characterized by comprising: a first obtaining module for obtaining modified routing table data if it is detected that the routing table in the preset first data management group is modified; and the synchronization module , Used for synchronizing the modified routing table data to any one or more non-default second data management group to update the routing table in the non-default second data management group; wherein, the distributed file The only first data management group and at least one second data management group deployed in the system, wherein the first data management group deployed in the distributed file system And the second data management group is set with a read and write request threshold, where the read and write request threshold is used to characterize the maximum number of accesses of the data management server in the corresponding data management group, if the number of accesses of the data management server exceeds The read and write request threshold will terminate the access to the data management server. The device according to claim 11, wherein the device further includes: a registration module for when a new second data management group is deployed in the distributed file system, the new second data management group The data management server is registered to the first data management group, and the routing table in the first data management group is synchronized to the new second data management group. The device according to claim 11 or 12, wherein the device further comprises: a second acquisition module, which is used to, after any data management group in the distributed file system receives a routing table request initiated by the requesting terminal, Obtain the routing table version information in the request terminal from the routing table request; the return module is used if the routing table version information is inconsistent with the version information of the routing table in any one of the data management groups, the arbitrary A data management group returns the routing table of the local storage area to the requesting terminal, or the requesting terminal obtains the latest routing table by accessing the data storage server deployed in the distributed file system. The device according to claim 13, wherein the device further includes: a determining module for determining the permission of the file or directory to be operated based on the information of the file or directory to be operated carried in the routing table request ； The execution module is used to continue to execute the step of obtaining the routing table version information in the requesting terminal from the routing table request if the permission is allowed operation; the third obtaining module is used to if the permission is forbidden Operation, then obtaining the latest routing table fails. The device according to claim 11, wherein the device further comprises: a persistence processing module, configured to perform persistence processing on the data management server included in the first data management group, and perform the persistence processing in the first data management group. After the group executes the persistence processing, the remote program call protocol for modifying the routing table is sent to at least one of the second data management groups, so that the second data management group executes the persistence processing. A routing table synchronization system, which is characterized by comprising: a first data management server, which is used to obtain the modified routing table data if it is detected that the routing table in the local storage area is modified, and the modified routing Table data is synchronized, wherein the first data management server is any server included in the first data management group preset in the distributed file system; one or more second data management servers are connected to the The first data management server has a communication relationship and is used to receive the modified routing table data synchronized by the first data management server to update the routing table in the second data management group, wherein the second data management server The server is any server included in the non-default second data management group in the distributed file system. The system according to claim 16, wherein the system further comprises: a requesting terminal, which has a communication relationship with the first data management server and the one or more second data management servers, and is used to communicate with the distributed file system Any data management group in the data management group sends a routing table request and receives the routing table returned by any data management group, or the requesting terminal obtains the latest routing table by accessing the data storage server deployed in the distributed file system . The system according to claim 17, wherein the requesting terminal is also used to parse the access information of each data management group in the distributed file system after invoking the access interface, and send it to any one of the data management groups The routing table request; wherein, if the requesting terminal fails to obtain the latest routing table from any one of the data management groups, the routing table request is sent to other data management groups in the distributed file system until the latest one is obtained Routing table. The system according to claim 18, wherein the requesting terminal is also used to parse the distributed file system to obtain the access information of any data management group in the distributed file system. The access information of the data storage server, and the routing table request is sent to the data storage server; wherein, if the requesting terminal fails to obtain the latest routing table from the data storage server, or the data storage server stores If the routing table of is not the latest routing table, the requesting terminal starts to analyze the access information of each data management group in the distributed file system. The system according to claim 17, wherein the requesting terminal is further configured to parse and obtain the address information of the preset first data management group in the distributed file system after invoking the access interface, and send the address information to the first data management group. The data management group sends the routing table request; wherein, if the requesting terminal fails to obtain the latest routing table from the first data management group, it will send to any other one or more second data management groups in the distributed file system Send the routing table request until the latest routing table is obtained. The system according to claim 20, wherein the requesting terminal is further configured to obtain the distributed file system by analyzing the access information of the preset first data management group in the distributed file system. The access information of the data storage server deployed in, and send the routing table request to the data storage server; wherein, if the requesting terminal fails to obtain the latest routing table from the data storage server, or the data storage server If the routing table stored in the device is not the latest routing table, the requesting terminal starts to analyze the access information of the first data management group in the distributed file system. The system according to claim 17, wherein the requesting terminal is also used to revisit the distributed file if it is detected that the routing table in the first data management group has been modified during the process of initiating the routing table request Any data management group in the system can obtain the latest routing table.

Images