CN115495432A

CN115495432A - Method, device and equipment for supporting multiple instances

Info

Publication number: CN115495432A
Application number: CN202211216141.8A
Authority: CN
Inventors: 刘俊峰; 姚文辉
Original assignee: Alibaba China Co Ltd
Current assignee: Alibaba China Co Ltd
Priority date: 2022-09-30
Filing date: 2022-09-30
Publication date: 2022-12-20

Abstract

The embodiment of the application provides a method, a device and equipment for supporting multiple instances, wherein the method comprises the following steps: receiving an instance creation request, the instance creation request including an identification of an instance; determining the identifier of a directory tree corresponding to the instance according to the instance creating request; and creating a directory tree in the equipment group and generating a mapping relation between the directory tree and the equipment group. Because a plurality of directory trees can be created in any equipment group, compared with the situation that only one directory tree is arranged in one equipment group, the flexibility of file storage in a file system during distribution is improved.

Description

Method, device and equipment for supporting multiple instances

Technical Field

The present application relates to the field of computer technologies, and in particular, to a method, an apparatus, and a device for supporting multiple instances.

Background

In the field of computer technology, user files may be stored in a plurality of storage servers in a Distributed manner, and a File directory may be stored in a plurality of management devices, and a System formed by the plurality of storage servers and the plurality of management devices may be referred to as a Distributed File System (DFS).

In the related art, for any one management device in a distributed file system, a directory tree may be stored in the management device. The directory tree may correspond to a plurality of user files. After a directory tree is created in the management device, the directory tree monopolizes the resources of the management device, and when the directory tree occupies fewer resources, the resource waste may be caused. When the directory tree occupies more resources, the resources may be insufficient. From the above, the flexibility of file storage in the existing distributed file system is low.

Disclosure of Invention

Aspects of the present application provide a method, apparatus, and device for supporting multiple instances, so as to improve flexibility of file storage in a distributed file system.

In a first aspect, an embodiment of the present application provides a method for supporting multiple instances, including:

receiving an instance creation request, the instance creation request including an identification of an instance;

determining the identifier of the directory tree corresponding to the instance according to the instance creating request;

creating the directory tree in a device group and generating a mapping relation between the directory tree and the device group.

In one possible embodiment, the method further comprises:

receiving a data access request sent by a client, wherein the data access request comprises a file name;

determining a directory tree and target file nodes corresponding to the file names according to the data access request, wherein the target file nodes are included in a plurality of file nodes corresponding to the directory tree;

and determining the storage position of the file to be accessed in a storage server according to the directory tree and the target file node, and sending the storage position to the client.

In one possible implementation, the method is applied to a distributed file system, wherein the distributed file system comprises a first device group and a second device group; the method further comprises the following steps:

determining a first directory tree to be migrated in a plurality of directory trees of the first device group, wherein the first device group comprises a plurality of first devices, and the plurality of first devices respectively comprise the plurality of directory trees;

setting a second device in the second device group to be in a virtual state, and acquiring a first operation log of the first directory tree from the first device group, wherein the second device group comprises a plurality of second devices;

and when a first operation log corresponding to the first directory tree in the second device is the same as a first operation log corresponding to the first directory tree in the first device, switching an authority device group of the first directory tree to the second device group.

In one possible implementation, the obtaining, by a second device in the second device group, a first operation log of the first directory tree from the first device group includes:

receiving the first operation log sent by a first master device, wherein the first master device is a master device in the first device group;

and storing the first operation log to a first storage space corresponding to the first directory tree.

In one possible implementation, receiving the first operation log sent by the first master device includes:

receiving a historical first operation log corresponding to the first directory tree and sent by the first main device;

storing the historical first operation log and sending a storage success response message to the first main device;

receiving a latest first operation log corresponding to the first directory tree sent by the first master device, and storing the latest first operation log, wherein the first operation log comprises the historical first operation log and the latest first operation log.

In one possible implementation, when a first operation log corresponding to the first directory tree in the second device is the same as a first operation log corresponding to the first directory tree in the first device, switching the authorized device group of the first directory tree to the second device group includes:

when a first operation log corresponding to the first directory tree in the second device is the same as a first operation log corresponding to the first directory tree in the first device, sending first indication information to the first master device, where the first indication information is used to indicate that the first operation log is synchronized;

receiving a switching log sent by the first master device, wherein the switching log is used for indicating that the permission device group of the first directory tree is switched to the second device group;

and switching the authority equipment group of the first directory tree into the second equipment group according to the switching log.

In a possible implementation manner, the first operation log includes an identifier of the first directory tree and identifiers of N second directory trees, where N is an integer; the method further comprises the following steps:

setting the states of the N second directory trees to be null;

switching the state of the second device from the virtual state to a running state.

In one possible embodiment, the method further comprises:

the second device obtains a device identifier of the second device from the first operation log;

and the second equipment elects main equipment in the second equipment group according to the equipment identification so as to determine second main equipment in the second equipment group.

In a possible implementation manner, the second device elects a master device in the second device group according to the device identifier, so that after determining a second master device in the second device group, the method further includes:

and sending second indication information to the first master device, wherein the second indication information comprises the identifier of the first directory tree, and the second indication information is used for indicating the first master device to delete the first directory tree.

In a possible embodiment, the number of said first directory trees is greater than 1; the method further comprises the following steps:

generating a second operation log corresponding to each first directory tree;

and storing the second operation log corresponding to each first directory tree into a corresponding log directory, wherein the log directories corresponding to the first directory trees are positioned under the same parent directory.

In one possible embodiment, the method further comprises:

determining an identifier of a second storage space occupied by the first directory tree in a storage server, wherein the identifier of the second storage space is allocated to a root server;

determining a storage file corresponding to the first directory tree in the storage server according to the identifier of the second storage space;

and updating the operation log and/or the storage file according to the operation log and the storage file corresponding to the first directory tree so as to enable the operation log and the storage file to be consistent.

In one possible embodiment, the method further comprises:

determining a target directory tree in the first directory tree;

according to the identification of the target directory tree, at least one of the following treatments is carried out on the target directory tree: probe processing, delete processing, or copy processing.

In a second aspect, an embodiment of the present application provides a distributed file system, which includes a client, a plurality of management devices, and at least one storage server, wherein,

the management device is configured to store a directory tree and execute the method of any of the first aspects to implement migration of the directory tree in different device groups, and the management device is located in one of the device groups;

the storage device is used for storing the files corresponding to the directory tree;

the client is used for accessing the file in the storage device through the directory tree in the management device.

In a third aspect, an embodiment of the present application provides an apparatus for supporting multiple instances, including: a first receiving module, a first determining module, and a creating module, wherein,

the first receiving module is used for receiving an instance creating request, and the instance creating request comprises an identifier of an instance;

the first determining module is used for determining the identifier of the directory tree corresponding to the instance according to the instance creating request;

the creating module is configured to create the directory tree in a device group, and generate a mapping relationship between the directory tree and the device group.

In a possible embodiment, the apparatus further comprises: a second determination module to determine,

determining a directory tree and target file nodes corresponding to the file names according to the data access request, wherein the plurality of file nodes corresponding to the directory tree comprise the target file nodes;

In a possible embodiment, the apparatus further comprises: a third determining module, a setting module, an obtaining module and a switching module, wherein,

the third determining module is configured to determine a first directory tree to be migrated from a plurality of directory trees of the first device group, where the first device group includes a plurality of first devices, and the plurality of first devices respectively include the plurality of directory trees;

the setting module is used for setting the second equipment in the second equipment group to be in a virtual state;

the obtaining module is configured to obtain a first operation log of the first directory tree from the first device group, where the second device group includes a plurality of second devices;

the switching module is configured to switch the permission device group of the first directory tree to the second device group when a first operation log corresponding to the first directory tree in the second device is the same as a first operation log corresponding to the first directory tree in the first device.

In a possible implementation manner, the obtaining module is specifically configured to:

In a possible implementation manner, the switching module is specifically configured to:

when a first operation log corresponding to the first directory tree in the second device is the same as a first operation log corresponding to the first directory tree in the first device, sending first indication information to a first master device, where the first indication information is used to indicate that the first operation log is synchronized;

In a possible implementation manner, the first operation log includes an identifier of the first directory tree and identifiers of N second directory trees, where N is an integer; the setting module is also used for setting the data,

setting the states of the N second directory trees to be null;

in one possible embodiment, the switching module is further configured to,

In one possible embodiment, the apparatus further comprises: an election module to select, from the plurality of candidate devices,

the second device acquires the device identifier of the second device from the first operation log;

In a possible embodiment, the apparatus further comprises: a sending module for sending the data to the receiving module,

the sending module is configured to send second indication information to a first master device, where the second indication information includes an identifier of the first directory tree, and the second indication information is used to indicate the first master device to delete the first directory tree.

In a possible embodiment, the number of said first directory trees is greater than 1; the device further comprises: a storage module to store, in a memory module,

generating a second operation log corresponding to each first directory tree;

and storing the second operation log corresponding to each first directory tree into a corresponding log directory, wherein the log directories corresponding to the plurality of first directory trees are positioned under the same parent directory.

In a possible embodiment, the apparatus further comprises: an update module to update the data stored in the memory,

In a possible embodiment, the apparatus further comprises: a processing module for, in response to a request from a user,

determining a target directory tree in the first directory tree;

according to the identification of the target directory tree, performing at least one of the following processing on the target directory tree: a probe process, a delete process, or a copy process.

In a fourth aspect, an embodiment of the present application provides an electronic device, including: a memory and a processor;

the memory stores computer execution instructions;

the processor executes computer-executable instructions stored by the memory to cause the processor to perform the method of any of the first aspects of supporting multiple instances.

In a fifth aspect, embodiments of the present application provide a computer-readable storage medium, in which computer-executable instructions are stored, and when the computer-executable instructions are executed by a processor, the method for supporting multiple instances according to any one of the first aspect is implemented.

In a sixth aspect, the present application provides a computer program product, including a computer program, which when executed by a processor, implements the method for supporting multiple instances shown in any one of the first aspects.

The embodiment of the application provides a method, a device and equipment for supporting multiple instances. The management device may determine, according to the instance creation request, an identifier of a directory tree corresponding to the instance, create the directory tree in the device group, and generate a mapping relationship between the directory tree and the device group. Because a plurality of directory trees can be created in any equipment group, compared with the situation that only one directory tree is arranged in one equipment group, the flexibility of file storage in a file system during distribution is improved.

Drawings

The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the application and together with the description serve to explain the application and not to limit the application. In the drawings:

FIG. 1 is a block diagram illustrating an architecture of a distributed file system according to an exemplary embodiment of the present application;

FIG. 2A is a diagram illustrating a centralized metadata service provided in an exemplary embodiment of the present application;

fig. 2B is a schematic diagram of a distributed metadata service provided in an exemplary embodiment of the present application;

fig. 3 is a schematic diagram of an application scenario provided in an exemplary embodiment of the present application;

FIG. 4 is a flowchart illustrating a method for supporting multiple instances in accordance with an exemplary embodiment of the present application;

FIG. 5 is a flowchart illustrating a data access method according to an exemplary embodiment of the present application;

fig. 6 is a flowchart illustrating a directory tree migration method according to an exemplary embodiment of the present application;

FIG. 7 is a diagram illustrating a directory tree provided in an exemplary embodiment of the present application;

FIG. 8 is a flowchart illustrating another method for directory tree migration according to an embodiment of the present application;

FIG. 9 is a first schematic diagram of a multi-instance capable device according to an exemplary embodiment of the present disclosure;

FIG. 10 is a second schematic structural diagram of a device supporting multiple instances according to an exemplary embodiment of the present application;

fig. 11 is a schematic structural diagram of an electronic device according to an exemplary embodiment of the present application.

Detailed Description

In order to make the objects, technical solutions and advantages of the present application more apparent, the technical solutions of the present application will be described in detail and completely with reference to the following specific embodiments of the present application and the accompanying drawings. It should be apparent that the described embodiments are only a few embodiments of the present application, and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

In the following, in order to facilitate understanding of the technical solution of the present application, with reference to fig. 1, a general architecture of a distributed file system in the present application is first described.

Fig. 1 is a schematic diagram of an architecture of a distributed file system according to an exemplary embodiment of the present application. Referring to fig. 1, the system includes a client, a device and a storage server.

A plurality of managing devices may be included in the distributed file system. In fig. 1, one management apparatus is illustrated as an example. As shown in fig. 1, the management device may include a plurality of first storage spaces 1, \8230 \ 8230;, first storage space m, respectively. For any one of the first storage spaces, a directory tree may be stored in the first storage space. For example, the first storage space 1 may store the directory tree 1 therein, and the first storage space m may store the directory tree m therein. The first storage space may also be referred to as a storage container (volume).

Optionally, the management device may include a default tree, and the temporary data may be uploaded through the default tree. When data is uploaded through the default tree, the form of the directory tree does not need to be specified, and therefore compatibility problems can be solved through the default tree.

The management device may store a file node list (FileNodeTable), and the file node list may include a plurality of file nodes (filenodes). Each directory tree may have its corresponding file node list, or multiple directory trees may share one file node list, which may include the following two cases:

case 1, one directory tree corresponds to one file node list.

In this case, the file node list includes file nodes corresponding to the files in the directory tree. The file node includes an Identifier (ID) of the file and metadata information, which may include a file size, a modification time, a file storage location, and the like.

For example, the file node list may be as shown in Table 1:

TABLE1

In case 2, a plurality of directory trees share one file node list, that is, one file node list corresponds to a plurality of directory trees.

In this case, the file node list includes file nodes corresponding to the files in the directory trees. The file node includes an Identifier (ID) of the file, a directory tree corresponding to the file, and metadata information, where the metadata information may include a file size, a modification time, a file storage location, and the like.

For example, the file node list may be as shown in Table 2:

TABLE 2

A plurality of storage service (ChunkServer) servers may be included in the distributed file system. The storage server may be configured to store files corresponding to the directory tree. As shown in fig. 1, storage server 2, \8230 \ 8230;, storage server q may be included in the distributed file system. For any one storage server, a plurality of second storage spaces may be included in the storage server. For example, the storage server 1 may include a second storage space 1, \8230;, a second storage space n.

When the client needs to access the file, a request may be sent to the management device, where the request may include a file name. This file name may also be referred to as mount (mount) point. After receiving the request, the management device can perform matching in the directory tree according to the file name in the request and the prefix longest matching principle, and then can find the corresponding file node in the corresponding file node list. For example, if the file name included in the request is "project a/", the management device may match the file name in the directory tree 1 according to the prefix longest matching rule, and further may determine the FileNode information corresponding to the file name "project a/" in the FileNode table1 corresponding to the directory tree 1. The management device may send the FileNode information to the client.

After the client determines the corresponding FileNode information, because the FileNode information includes the ID of the file, the size of the file, the modification time, the storage location of the file, etc., the client can determine the storage location of the file in the storage server according to the FileNode information, thereby accessing the corresponding file.

It should be noted that the service for metadata can be divided into a centralized metadata service and a distributed metadata service.

The centralized metadata service and the distributed metadata service are described below with reference to fig. 2A-2B, respectively.

Fig. 2A is a schematic diagram of a centralized metadata service provided in an exemplary embodiment of the present application. Referring to fig. 2A, the centralized metadata service refers to a meta service (MetaServer), a namespace service (namespace server), and an RGServer, all of which are within one process. The MetaServer is responsible for managing the meta information of the file, is responsible for the security of data and is responsible for a holder of a file lock; the NameSpaceServer is responsible for directory tree management work of the file system; the RGServer is responsible for data replication and garbage collection.

Fig. 2B is a schematic diagram of a distributed metadata service according to an exemplary embodiment of the present application. Referring to fig. 2A, the distributed metadata service refers to that a metasserver, a namespaced server, and an RGServer are respectively located in different independent processes, and interaction between services is performed through Remote Procedure Call (RPC).

Fig. 3 is a schematic diagram of an application scenario provided in an exemplary embodiment of the present application. As shown in fig. 3, a first device group and a second device group may be included in the distributed file system.

The first device group may include a plurality of first devices. For example, first device group may include first device 1, first device 2, \8230;, first device n. For any one first device, the first device may include a plurality of directory trees therein. For example, directory trees 1-1, 1-2, \8230;, 1-x may be included in the first device 1. The directory tree included in each of the first devices is the same. The first device group may also be referred to as a Raft group, and each first device in the Raft group is in a redundant backup relationship, and at the same time, one first master device exists in the Raft group, and the other devices are first slave devices.

Likewise, a plurality of second devices may be included in the second device group. The status of the plurality of second devices is empty, i.e., no directory tree is stored in the plurality of second devices.

If the directory trees to be migrated in the first device group are the directory tree 1-1 and the directory tree 1-2, and the second device 1 in the second device group is in a virtual state, the operation logs corresponding to the directory tree 1-1 and the directory tree 1-2 may be migrated to the second device 1, so as to migrate the directory tree 1-1 and the directory tree 1-2 to the second device 1.

After the migration of the directory tree 1-1 and the directory tree 1-2 is completed, the second device 2, \8230 \ second device m in the second device group can also be subjected to redundant backup, so that the directory tree 1-1 and the directory tree 1-2 are stored in any one second device in the second device group.

It should be noted that the first device and the second device may be management devices in the embodiment in fig. 1.

In the related art, for any one management device in a distributed file system, a directory tree may be stored in the management device. The directory tree may correspond to a plurality of user files. After a directory tree is created in the management device, the directory tree monopolizes the resources of the management device, and when the directory tree occupies fewer resources, resource waste may be caused; when the directory tree occupies more resources, the resources may be insufficient. From the above, the flexibility of file storage in the existing distributed file system is low.

In the embodiment of the application, for any one management device in the distributed file system, the management device can store a plurality of directory trees, and can migrate one or more directory trees in any one management device, so that the flexibility of file storage in the file system during distribution is improved.

Hereinafter, the technical means shown in the present application will be described in detail by specific examples. It should be noted that the following embodiments may exist alone or in combination with each other, and description of the same or similar contents is not repeated in different embodiments.

In the technical scheme of the application, 3 processes are mainly involved, namely a process of creating a directory tree, a process of accessing data and a process of migrating the directory tree. Next, a process of creating a directory tree will be described with reference to fig. 4.

Fig. 4 is a flowchart illustrating a method for supporting multiple instances according to an exemplary embodiment of the present application. Referring to fig. 4, the method may include:

s401, an instance creation request is received.

For any management device, the management device may receive an instance creation request, where the instance creation request may include an identifier of the instance. For example, the instance creation request may include the identification ID of instance 1.

S402, according to the instance creating request, determining the identifier of the directory tree corresponding to the instance.

Since the instance creation request may include the identifier of the instance, the management device may determine, according to the identifier of the instance, the identifier of the directory tree corresponding to the instance. For example, the management device may determine the identity of directory tree 1 based on the identity of instance 1.

S403, creating a directory tree in the device group, and generating a mapping relation between the directory tree and the device group.

For any one device group, a plurality of devices may be included in the device group. The plurality of devices may be the management devices shown in fig. 1.

In an alternative embodiment, the management device may determine a device group with more resources among the plurality of device groups, and create a directory tree on the devices in the device group. Optionally, the device group may include a plurality of created directory trees, or may not include a directory tree.

For example, if the directory tree to be created is directory tree 1, and if there are 10 device groups, where the device group 2 has more resources, then directory tree 1 may be created in device group 2. If device group 2 includes 2 directory trees that have been created, device group 2 may include 3 directory trees after directory tree 1 is created in device group 2.

Alternatively, after the directory tree is successfully created, a mapping relationship between the directory tree and the device group may be generated.

For example, if directory tree 1 is created in device group 2, the mapping relationship generated may be: directory tree 1 corresponds to device group 2.

In an embodiment of the present application, a management device may receive an instance creation request, where the instance creation request includes an identification of an instance. The management device may determine, according to the instance creation request, an identifier of a directory tree corresponding to the instance, create the directory tree in the device group, and generate a mapping relationship between the directory tree and the device group. Because a plurality of directory trees can be created in any equipment group, compared with the situation that only one directory tree is arranged in one equipment group, the flexibility of file storage in a file system during distribution is improved.

Next, a procedure of data access will be described with reference to fig. 5.

Fig. 5 is a schematic flowchart of a data access method according to an exemplary embodiment of the present application. Referring to fig. 5, the method includes:

s501, receiving a data access request sent by a client.

The client may send a data access request to the management device to cause the management device to receive the data access request. The data access request includes a file name. For example, the file name included in the data access request may be "project a/".

S502, determining a directory tree and a target file node corresponding to the file name according to the data access request.

For any file node (FileNode), the file node may include an Identifier (ID) of the file, a directory tree corresponding to the file, and metadata information, where the metadata information may include a file size, modification time, a file storage location, and the like.

Alternatively, the directory tree may have a corresponding list of file nodes. The file node list may include a plurality of file nodes, and the plurality of file nodes may include a target file node.

In an optional embodiment, after receiving the data access request, the management device may perform matching in the directory tree according to the file name in the data access request and according to the prefix longest matching principle, so as to determine the directory tree corresponding to the file name, and further may determine the target file node corresponding to the file name in a file node list (FileNodeTable) corresponding to the directory tree.

For example, if the file name included in the data access request is "project a/", and if the management device can match the file name in directory tree 1 according to the prefix longest matching rule, the directory tree corresponding to the file name "project a/" may be determined to be directory tree 1. Further, the FileNode information corresponding to the file name "project a/", i.e. the target file node, may be determined in the FileNode table1 corresponding to the directory tree 1.

S503, determining the storage position of the file to be accessed in the storage server according to the directory tree and the target file node, and sending the storage position to the client.

Optionally, after the management device determines the directory tree and the target file node, since the target file node includes a file storage location, the management device may determine, according to the target file node, a storage location of the file to be accessed in the storage server. Alternatively, the management device may send the target file node information to the client, that is, send the storage location of the file to be accessed to the client.

Optionally, after the client determines the corresponding target file node information, since the target file node information includes an ID of the file, a file size, modification time, a file storage location, and the like, the client may determine the file storage location in the storage server according to the target file node information, so as to access the corresponding file.

In the embodiment of the application, the management device may receive a data access request sent by a client, determine a directory tree and a file node corresponding to a file name according to the data access request, further determine a storage location of a file to be accessed in the storage server according to the directory tree in which the directory tree and a target file node are located, and send the storage location to the client. The storage position of the file to be accessed in the storage server can be determined through the target file node, so that the accuracy of data access is improved.

Next, a process of directory tree migration will be described with reference to fig. 6.

Fig. 6 is a flowchart illustrating a directory tree migration method according to an exemplary embodiment of the present application. Referring to fig. 6, the method may include:

s601, determining a first directory tree to be migrated in a plurality of directory trees of a first device group.

The execution subject of the embodiment of the present application may be the second device, and may also be a device supporting multiple instances, which is disposed in the second device. The apparatus supporting multiple instances may be implemented by software, or may be implemented by a combination of software and hardware. The apparatus supporting multiple instances may be a processor in the second device. The second device may be the managing device in the embodiment of fig. 1. For ease of understanding, the execution subject is hereinafter described as an example of the second device.

The first device group may be a cluster of devices supported by a distributed protocol. The first device group may include a plurality of first devices. For example, 5 first devices may be included in the first device group.

The distributed protocol may be a Raft protocol. The consistency problem of the duplicate data on a plurality of devices in the distributed file system can be solved according to the Raft protocol.

The first device group may include a plurality of directory trees, and for any one directory tree, the directory tree may include a root directory and a plurality of sub-directories under the root directory.

Next, the directory tree will be described with reference to fig. 7.

Fig. 7 is a schematic diagram of a directory tree provided in an exemplary embodiment of the present application. Referring to fig. 7, a directory tree 1 may include a root directory and a plurality of subdirectories. The plurality of subdirectories may be subdirectory 1, subdirectory 2 and subdirectory 3, respectively. Subdirectory 1 can include subdirectory 1-1 and subdirectory 1-2; subdirectory 2 can include subdirectory 2-1, subdirectory 2-2 and subdirectory 2-3; under subdirectory 3 may be included subdirectory 3-1.

The first device group may include a plurality of first devices, and the plurality of first devices may include a plurality of directory trees, respectively. The first directory tree to be migrated may be determined in the first device group according to actual conditions, and the number of the first directory trees may be at least one. For example, in an actual operation process, when the directory tree in each first device in the first device group consumes too many resources and the first device cannot provide enough resources, at least one first directory tree may be determined in the first device group, and the at least one first directory tree may be migrated to the second device group.

For example, if the first device group includes 5 first devices, which are respectively the first device 1, the first device 2, the first device 3, the first device 4, and the first device 5, where the 5 first devices respectively include the directory tree-1, the directory tree-2, and the directory tree-3, the second device may determine, in the first device group, that the first directory tree to be migrated is the directory tree-1 and the directory tree-2 in the first device 1.

S602, setting the second equipment in the second equipment group to be in a virtual state, and acquiring a first operation log of the first directory tree from the first equipment group.

The second device group may be another device cluster supported by the distributed protocol. The second device group may include a plurality of second devices. For example, 8 second devices may be included in the second device group.

In the virtual state, the second device may receive an oplog (oplog) without service request processing.

The first oplog includes records of operations on the first directory tree. The first operation log may include a historical first operation log and a latest first operation log corresponding to the first directory tree. The historical first operation log may be an operation log generated before the current time, and the latest operation log may be an operation log generated at the current time. For example, if the current time is 14.

Optionally, the second device may be set to a virtual state to obtain the first operation log of the first directory tree from the first device group.

In an alternative embodiment, the first operation log of the first directory tree may be obtained from the first device group in a feasible manner as follows: receiving a first operation log sent by first main equipment; and storing the first operation log to a first storage space corresponding to the first directory tree.

The first master device refers to a master device in the first device group, and may be determined by election of a plurality of first devices in the first device group. For example, the first device group may include 5 first devices, which are first device 1, first device 2, first device 3, first device 4, and first device 5. For example, the first device 2 may be determined to be the first master device by the 5 first device elections.

For any one device in the distributed system, a plurality of first storage spaces may be included in the device to store a plurality of directory trees. For example, 5 first storage spaces may be included in the first device 1 to store 5 directory trees.

The first storage space may be designated by a Root Server (Root Server) in the distributed file system when creating the directory tree. There may be a correspondence between the directory tree and the first storage space, which may be persisted in the root server.

For example, the correspondence between the directory tree and the first storage space may be as shown in table 3:

TABLE 3

As shown in table 3, the directory tree 1 may include a directory tree 1/subdirectory 1 and a directory tree 1/subdirectory 2, and different subdirectories may correspond to different storage spaces, that is, different subdirectories under one directory tree may be stored in different storage spaces, in other words, one storage space may store a part of one directory tree; directory tree 2 may correspond to first storage space 3, directory tree 4 may correspond to first storage space 4, and a complete directory tree may be stored in one storage space.

In an alternative embodiment, the first operation log sent by the first master device may be received in the following feasible manners: receiving a historical first operation log corresponding to a first directory tree sent by a first main device; storing a historical first operation log and sending a storage success response message to the first main device; and receiving the latest first operation log corresponding to the first directory tree sent by the first main device, and storing the latest first operation log.

For example, if the first master device is a first device 2 in the first device group, the first directory tree is a directory tree 1 in the first device 2, the first operation log includes a historical first operation log and a latest operation log corresponding to the directory tree 1, and the storage space corresponding to the directory tree 1 is a first storage space 1 in the second device 1, the second device 1 may be set in a virtual state, and the second device 1 may receive the historical first operation log sent by the first device 2 and store the historical first operation log in the storage space 1. After successful storage, the second device 1 may transmit a storage success response message to the first device 2. The first device 2 may transmit the latest first operation log to the second device 1, and the second device 1 may store the latest first operation log in the first storage space 1 after receiving the latest first operation log.

S603, when the first operation log corresponding to the first directory tree in the second device is the same as the first operation log corresponding to the first directory tree in the first device, the authority device group of the first directory tree is switched to the second device group.

In an alternative embodiment, the rights device group of the first directory tree may be switched to the second device group by: the method comprises the steps that first indication information is sent to first main equipment when a first operation log corresponding to a first directory tree in second equipment is the same as a first operation log corresponding to the first directory tree in first equipment; receiving a switching log sent by first master equipment; and switching the authority device group of the first directory tree into a second device group according to the switching log.

The first indication information may be used to indicate that the first operation log is completed synchronously, that is, the content of the first operation log corresponding to the first directory tree in the second device is the same as the content of the first operation log corresponding to the first directory tree in the first device.

The switching log is used for indicating that the permission device group of the first directory tree is switched to the second device group. The contents of the handover log may include: a record of the migration of the first directory tree, an address of the second device, and a device identification of the second device.

The address of the second device may be a Transmission Control Protocol (TCP) address.

The device identification may be a Globally Unique Identifier (GUID). The GUID may be a 32-bit hexadecimal value. For example, the GUID may be 6F9619FF-8B86-D011-B42D-00C04FC964FF.

For example, if the first master device is a first device 1, the first directory tree is a directory tree 1 in the first device 1, and the second device is a second device 1, the switching log may be "migrate the directory tree 1 from the first device 1 to the second device 1, the address of the second device 1 is TCP address 1, and the GUID of the second device 1 is 6F9619FF-8B86-D011-B42D-00C04FC964FF".

The rights device group refers to a device group to which a user has access rights.

For example, if the current time is 14. The second device 1 may receive the handover log transmitted by the first device 1. If the switching log is "migrate the directory tree 1 from the first device 1 to the second device 1, the address of the second device 1 is TCP address 1, and the GUID of the second device 1 is 6F9619FF-8B86-D011-B42D-00C04FC964FF", the second device 1 may switch the authorized device group of the directory tree 1 from the first device group to the second device group according to the switching log, and the migration to the directory tree 1 is successful.

In this embodiment of the present application, the second device may determine, in the multiple directory trees of the first device group, a first directory tree to be migrated, set a second device in the second device group to a virtual state, and obtain, from the first device group, a first operation log of the first directory tree. When the first operation log corresponding to the first directory tree in the second device is the same as the first operation log corresponding to the first directory tree in the first device, the permission device group of the first directory tree may be switched to the second device group. Because the first device and the second device can store a plurality of directory trees and can migrate one or more directory trees in any one device, the first device and the second device can dynamically support the plurality of directory trees, and the flexibility of file storage in a file system during distribution is improved.

Next, based on the embodiment shown in fig. 6, the method for migrating the directory tree is further described with reference to fig. 8.

Fig. 8 is a flowchart illustrating another directory tree migration method according to an embodiment of the present application. Referring to fig. 8, the method may include:

s801, determining a first directory tree to be migrated in the plurality of directory trees of the first device group.

It should be noted that, the execution process of step S801 may refer to step S601, and is not described herein again.

S802, setting the second equipment in the second equipment group to be in a virtual state, and acquiring a first operation log of the first directory tree from the first equipment group.

It should be noted that, the execution process of step S802 may refer to step S602, and is not described herein again.

S803, when the first operation log corresponding to the first directory tree in the second device is the same as the first operation log corresponding to the first directory tree in the first device, switching the permission device group of the first directory tree to the second device group.

It should be noted that, the execution process of step S803 may refer to step S603, and details are not described here.

S804, setting the states of the N second directory trees to be null.

The first oplog may include an identification of the first directory tree and identifications of the N second directory trees. For any directory tree, the identity of the directory tree may be a GUID.

After the migration is completed, the first directory tree is included in the second device, and the second directory tree is not included. And the first operation log in the second device also comprises identifications of the N second directory trees. In order to make the directory tree in the second device and the first operation log consistent, the states of the N second directory trees may be set to null to delete the identifier of the second directory tree in the first operation log of the second device.

For example, the first operation log may include identifiers of directory tree 1, directory tree 2, directory tree 3, directory tree 4, and directory tree 5, where directory tree 1 is the first directory tree, and directory tree 2, directory tree 3, directory tree 4, and directory tree 5 are the second directory trees, and the second device may determine identifiers of directory tree 2, directory tree 3, directory tree 4, and directory tree 5, and set the states of the 4 second directory trees to be null according to the identifiers of the 4 second directory trees, so as to delete the identifiers of the 4 second directory trees in the first operation log of the second device.

S805, the state of the second device is switched from the virtual state to the operating state.

After the authorized device group of the first directory tree is switched to the second device group, the device state of the second device may be switched from the virtual state to the running state, so that the second device may be normally served.

S806, obtaining the device identification of the second device, and carrying out main device election in the second device group according to the device identification.

The first operation log may include a device identifier of the second device, and the second device may obtain the device identifier of the second device from the first operation log.

In an alternative embodiment, the primary device election may be performed in the second device group by: determining the equipment identification of each second equipment in the second equipment group; and voting according to the equipment identification, and determining the second equipment with the most votes as second main equipment.

For example, if the second device group includes a second device 1, a second device 2, a second device 3, a second device 4, and a second device 5, the corresponding device identifiers are GUID-1, GUID-2, GUID-3, GUID-4, and GUID-5, respectively. A vote may be made based on the device identification, and if GUID-3 has the most votes, the corresponding second device 3 may be determined to be the second master device.

Optionally, after the second master device is determined, redundant backup may be performed on the second master device and the second slave device in the second device group, so that directory trees stored in the second master device and the second slave device are kept consistent.

In this embodiment of the present application, the second device may determine, in the multiple directory trees of the first device group, a first directory tree to be migrated, set a second device in the second device group to a virtual state, and obtain, from the first device group, a first operation log of the first directory tree. When the first operation log corresponding to the first directory tree in the second device is the same as the first operation log corresponding to the first directory tree in the first device, the permission device group of the first directory tree may be switched to the second device group. The state of the second device can be switched from the virtual state to the running state, the device identifier of the second device is obtained, and the master device election is carried out in the second device group according to the device identifier. Because the first device and the second device can store a plurality of directory trees and can migrate one or more directory trees in any one device, the first device and the second device can dynamically support a plurality of directory trees, and the flexibility of file storage in a file system during distribution is improved.

In actual work, after the second device group normally operates, the following management and maintenance can be performed on the directory tree, the corresponding operation log, the corresponding storage file and the like:

1. optionally, after the second device operates normally, the second device may send the second indication information to the first master device. The second indication information may include an identifier of the first directory tree, which is used to instruct the first master device to delete the first directory tree. The first master device may delete the first directory tree according to the second indication information to release the memory of the first device.

For example, if the first directory tree is directory tree 1, the second indication information includes the identifier of directory tree 1. The second device may send the second indication information to the first master device, so that the first master device deletes the directory tree 1 according to the identifier of the directory tree 1.

2. Alternatively, the number of the first directory tree may be at least one. The second device may generate a second operation log corresponding to each first directory tree and store each second operation log in a corresponding log directory. The log directory may be a Cross Platform Toolkit (CPT) directory. The multiple log directories may be located under the same parent directory. In this way, the second operation log corresponding to each first directory tree can be stored relatively independently, so that each first directory tree can be loaded independently by the second device.

3. Optionally, since the storage file corresponding to the first directory tree is stored in the storage service (ChunkServer), the second device may further determine an identifier of a second storage space occupied by the first directory tree in the storage server, determine the storage file corresponding to the first directory tree in the storage server according to the identifier of the storage space, and update the operation log and/or the storage file according to the operation log and the storage file corresponding to the first directory tree.

The identity of the second storage space may be represented by a storage server identity (ChunkServer ID). The identities of the second storage spaces may be uniformly distributed by a root server in the distributed file system such that the identity of each second storage space is globally unique in the distributed file system.

Optionally, the first directory tree and the identifier of the second storage space may be persistently stored in the root server.

For example, if the first directory tree is directory tree 1, the second device may determine that the identifier of the storage space occupied by directory tree 1 in the storage server may be ChunkServer ID-1, and determine, according to ChunkServer ID-1, the storage file corresponding to directory tree 1 in the storage server.

Since all information about the first directory tree is recorded in the operation log corresponding to the first directory tree, including the name of the storage file corresponding to the first directory tree, the corresponding storage file may be searched in the storage server according to the name of the storage file recorded in the operation file, and it is determined whether the storage file exists. If the corresponding storage file does not exist in the storage server, the name of the storage file in the operation log can be deleted, so that the operation log and the storage file are kept consistent.

4. Optionally, since the second device includes at least one directory tree, the second device may further determine a target directory tree in the first directory tree, and perform detection processing, deletion processing, or copy processing on the target directory tree according to the identifier of the target directory tree.

The identification of the target directory tree may be represented by a GUID. The target directory tree can be processed according to the identification of the target directory tree, so that the persistent operation log can carry the identification of each target directory tree.

The detection processing refers to that whether the storage file corresponding to the target directory tree is consistent with the storage file in the storage server or not is checked through algorithm traversing the target directory tree.

The deletion process means that the target directory tree can be deleted in the first directory tree if the storage file corresponding to the target directory tree does not exist.

The copy processing means that when the target directory tree is subjected to master-slave backup, the target directory tree can be copied to perform backup in other devices in one device group.

For example, if the first directory tree after migration in the second device group includes directory tree 1 and directory tree 2, if directory tree 1 is the target directory tree, probe processing may be performed on directory tree 1 to check whether the storage file corresponding to the target directory tree is consistent with the storage file in the storage server; if the storage file corresponding to the directory tree 1 does not exist, the directory tree 1 may be deleted.

For example, if the second device group includes 5 second devices, which are the second device 1, the second device 2, the second device 3, the second device 4, and the second device 5, and if the directory tree 1 and the directory tree 2 are both in the second device 1 and the directory tree 2 are not present in the other 4 second devices, the directory tree 1 and the directory tree 2 may be copied to backup in the other 4 devices.

Fig. 9 is a first schematic structural diagram of a device supporting multiple instances according to an exemplary embodiment of the present application. Referring to fig. 9, the multi-instance supporting apparatus 100 includes: a first receiving module 101, a first determining module 102 and a creating module 103, wherein,

the first receiving module 101 is configured to receive an instance creation request, where the instance creation request includes an identifier of an instance;

the first determining module 102 is configured to determine, according to the instance creation request, an identifier of a directory tree corresponding to the instance;

the creating module 103 is configured to create the directory tree in a device group, and generate a mapping relationship between the directory tree and the device group.

The apparatus supporting multiple instances provided in the embodiment of the present application may implement the technical solutions shown in the above method embodiments, and the implementation principles and beneficial effects thereof are similar and will not be described herein again.

Fig. 10 is a schematic structural diagram of a device supporting multiple instances according to an exemplary embodiment of the present application. Referring to fig. 8, based on the embodiment shown in fig. 9, the apparatus for supporting multiple instances may further include: the second determination module 104 is configured to determine,

in a possible embodiment, the second determining module 104 is specifically configured to:

In a possible embodiment, the apparatus may further include: a third determination module 105, a setting module 106, an acquisition module 107 and a switching module 108, wherein,

the third determining module 105 is configured to determine a first directory tree to be migrated from a plurality of directory trees of the first device group, where the first device group includes a plurality of first devices, and the plurality of first devices respectively include the plurality of directory trees;

the setting module 106 is configured to set a second device in the second device group to a virtual state;

the obtaining module 107 is configured to obtain a first operation log of the first directory tree from the first device group, where the second device group includes a plurality of second devices;

the switching module 108 is configured to switch the permission device group of the first directory tree to the second device group when the first operation log corresponding to the first directory tree in the second device is the same as the first operation log corresponding to the first directory tree in the first device.

In a possible implementation manner, the obtaining module 107 is specifically configured to:

receiving a historical first operation log corresponding to the first directory tree sent by the first master device;

In a possible implementation, the switching module 108 is specifically configured to:

In a possible implementation manner, the first operation log includes an identifier of the first directory tree and identifiers of N second directory trees, where N is an integer; the setup module 106 is also configured to,

setting the states of the N second directory trees to be null;

in one possible embodiment, the switching module 108 is further configured to,

In a possible embodiment, the apparatus may further include: an election module 109, said election module 109 to,

and the second equipment elects the main equipment in the second equipment group according to the equipment identification so as to determine second main equipment in the second equipment group.

In a possible embodiment, the apparatus may further include: the sending module (110) is used for sending the data,

the sending module 110 is configured to send second indication information to a first master device, where the second indication information includes an identifier of the first directory tree, and the second indication information is used to indicate the first master device to delete the first directory tree.

In a possible embodiment, the number of said first directory trees is greater than 1; the apparatus may further include: a storage module 111, the storage module 111 configured to:

generating a second operation log corresponding to each first directory tree;

In a possible embodiment, the apparatus further comprises: an update module 112, the update module 112 configured to,

In one possible embodiment, the apparatus further comprises: a processing module 113 for determining a target directory tree in the first directory tree;

according to the identification of the target directory tree, at least one of the following treatments is carried out on the target directory tree: a probe process, a delete process, or a copy process.

The exemplary embodiment of the present application provides a schematic structural diagram of an electronic device, please refer to fig. 11, where the electronic device 20 may include a processor 21 and a memory 22. The processor 21, the memory 22, and the various parts are illustratively interconnected by a bus 23.

The memory 22 stores computer-executable instructions;

the processor 21 executes computer-executable instructions stored by the memory 22 to cause the processor 21 to perform a method of supporting multiple instances as shown in the above-described method embodiments.

The electronic device shown in fig. 11 may be the first device or the second device in the above-described embodiments.

Accordingly, embodiments of the present application provide a computer-readable storage medium, in which computer-executable instructions are stored, and when the computer-executable instructions are executed by a processor, the computer-readable storage medium is configured to implement the method supporting multiple instances described in the above method embodiments.

Accordingly, embodiments of the present application may also provide a computer program product, which includes a computer program, and when the computer program is executed by a processor, the method for supporting multiple instances shown in the foregoing method embodiments may be implemented.

As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

The present invention has been described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.

The memory may include forms of volatile memory in a computer readable medium, random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of a computer-readable medium.

Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, a computer readable medium does not include a transitory computer readable medium such as a modulated data signal and a carrier wave.

It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrases "comprising a," "8230," "8230," or "comprising" does not exclude the presence of other like elements in a process, method, article, or apparatus comprising the element.

The above description is only an example of the present application and is not intended to limit the present application. Various modifications and changes may occur to those skilled in the art. Any modification, equivalent replacement, improvement or the like made within the spirit and principle of the present application shall be included in the scope of the claims of the present application.

Claims

1. A method of supporting multiple instances, comprising:

creating the directory tree in a device group and generating a mapping relationship between the directory tree and the device group.

2. The method of claim 1, further comprising:

3. The method according to claim 1 or 2, wherein the method is applied to a distributed file system, and the distributed file system comprises a first device group and a second device group; the method further comprises the following steps:

and when a first operation log corresponding to the first directory tree in the second device is the same as a first operation log corresponding to the first directory tree in the first device, switching the permission device group of the first directory tree to the second device group.

4. The method of claim 3, wherein the second device of the second device group obtaining the first oplog of the first directory tree from the first device group comprises:

5. The method of claim 4, wherein receiving the first operation log sent by the first master device comprises:

6. The method according to any one of claims 3 to 5, wherein switching the authorized device group of the first directory tree to the second device group when the first operation log corresponding to the first directory tree in the second device is the same as the first operation log corresponding to the first directory tree in the first device comprises:

when a first operation log corresponding to the first directory tree in the second device is the same as a first operation log corresponding to the first directory tree in the first device, sending first indication information to a first master device, where the first indication information is used for indicating that the first operation log is completed synchronously;

7. The method of claim 6, wherein the first oplog comprises an identification of the first directory tree and identifications of N second directory trees, wherein N is an integer; the method further comprises the following steps:

setting the states of the N second directory trees to be null;

8. The method according to any one of claims 3-7, further comprising:

9. The method of claim 8, wherein the second device elects a master device among the second device group according to the device identifier to determine a second master device among the second device group, and further comprising:

and sending second indication information to the first main device, wherein the second indication information comprises the identifier of the first directory tree, and the second indication information is used for indicating the first main device to delete the first directory tree.

10. The method of any of claims 3-9, wherein the number of first directory trees is greater than 1; the method further comprises the following steps:

generating a second operation log corresponding to each first directory tree;

11. The method according to any one of claims 3-8, further comprising:

12. The method according to any one of claims 3-11, further comprising:

determining a target directory tree in the first directory tree;

13. A distributed file system comprising a client, a plurality of management devices, and at least one storage server, wherein,

the management device is used for storing the directory tree and executing the method of any one of claims 1 to 12 to realize the migration of the directory tree in different device groups, and the management device is located in one device group;

14. An electronic device, comprising: a memory and a processor;

the memory stores computer-executable instructions;

the processor executing the computer-executable instructions stored by the memory causes the processor to perform the method of supporting multiple instances of any of claims 1 to 12.

15. A computer-readable storage medium having stored thereon computer-executable instructions for implementing the method of any of claims 1 to 12 when executed by a processor.

16. A computer program product comprising a computer program which, when executed by a processor, implements the multi-instance enabled method of any of claims 1 to 12.