CN115955488A - Distributed storage copy cross-computer room placement method and device based on copy redundancy - Google Patents
Distributed storage copy cross-computer room placement method and device based on copy redundancy Download PDFInfo
- Publication number
- CN115955488A CN115955488A CN202310225524.XA CN202310225524A CN115955488A CN 115955488 A CN115955488 A CN 115955488A CN 202310225524 A CN202310225524 A CN 202310225524A CN 115955488 A CN115955488 A CN 115955488A
- Authority
- CN
- China
- Prior art keywords
- copy
- file
- deployed
- copies
- field
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Abstract
The application provides a distributed storage copy cross-computer room placement method and device based on copy redundancy, the method is applied to a distributed storage system, and the method comprises the following steps: deploying a plurality of metadata nodes to a first machine room in the distributed storage system, and determining a main metadata node; setting a copy placing strategy aiming at a file to be deployed; responding to a write-in request initiated by a client, and finishing the cross-machine room placement of the file to be deployed by the main metadata node according to the copy placement strategy of the file to be deployed; the method comprises the steps of obtaining access data of a file to be deployed, and modifying a copy placing strategy of the file to be deployed based on the access data of the file to be deployed. By setting a copy placement strategy, the distributed storage system can sense the copy distribution condition so as to solve the problems of file reading, file writing, copy recovery, copy deletion and cross-machine room flow under the task flow scene of the computing node.
Description
Technical Field
The application relates to the technical field of computers, in particular to a method and a device for placing distributed storage copies across computer rooms based on copy redundancy.
Background
With the advent of the cloud computing era, a multi-cloud architecture has become a mainstream architecture of each large government and enterprise organization, a science and technology company and an internet company. The distributed storage is used as a base of the whole data architecture and can provide stable and reliable data read-write service, so that how to provide the stable and reliable distributed storage read-write service in a cloudy environment is a crucial matter, and the distributed storage read-write service directly determines whether data analysis can fully utilize cloudy resources and whether data decision can be timely transmitted.
At present, the distributed storage based on copy redundancy supports multi-cloud deployment in two situations, the first is that the multi-cloud deployment is not supported, for example, the HDFS itself does not provide a multi-cloud solution, if the distributed storage is forcibly deployed in a multi-cloud mode, the read-write flow of the distributed storage service is far greater than the dedicated bandwidth between the machine rooms, and even normal communication between multiple clouds is caused. The second method is to provide limited support for multi-cloud deployment, and a user can access corresponding distributed storage in different machine rooms by deploying multiple sets of distributed storage, or the user can read corresponding machine room copies in corresponding machine rooms by placing copies across the machine rooms; however, the cost for deploying a plurality of sets of distributed storage is high, the copies are placed across the machine rooms, and only when the copies are placed across the machine rooms and then read, the copies corresponding to the machine rooms are read, and more scenes are not covered. Therefore, it is imperative to solve the problem of cross-machine room traffic of distributed storage to achieve stable and reliable multi-cloud deployment.
Disclosure of Invention
The application provides a method and a device for placing distributed storage copies across machine rooms based on copy redundancy, which are used for solving the problem of flow of distributed storage across machine rooms, so that stable and reliable multi-cloud deployment is realized.
In a first aspect, the present application provides a distributed storage replica cross-room placement method based on replica redundancy, where the method is applied to a distributed storage system, and the method includes:
deploying a plurality of metadata nodes to a first machine room in the distributed storage system, and determining a main metadata node;
setting a copy placing strategy aiming at a file to be deployed, wherein the copy placing strategy comprises the following steps: mainAz, native write, azPolicy, disableRedundant delete, disableBlockRecovery fields; the main server comprises a main server, a native server, an azPolicy field, a disableRedundant delete field and a disableBlockRecovery field, wherein the main server is used for determining a main server room, the native server is used for determining whether a copy is written into a second server room, the azPolicy field is used for determining copy distribution, the disableRedundant delete field is used for temporarily forbidding a redundant copy deletion mechanism under a certain directory, and the disableBlockRecovery field is used for temporarily forbidding a lost copy recovery mechanism under the certain directory;
responding to a write-in request initiated by a client, and finishing the cross-machine room placement of the file to be deployed by the main metadata node according to the copy placement strategy of the file to be deployed;
the method comprises the steps of obtaining access data of a file to be deployed, and modifying a copy placing strategy of the file to be deployed based on the access data of the file to be deployed.
According to the method for placing the distributed storage copies across the machine rooms based on the copy redundancy, the main metadata node completes the placement of the files to be deployed across the machine rooms according to the copy placement strategy of the files to be deployed, and the method comprises the following steps:
acquiring a copy placement strategy and the current deployment copy number of a file to be deployed, and recording the copy placement strategy and the current deployment copy number in the main metadata node;
determining the initial copy number and copy distribution positions of the files to be deployed based on the azPolicy field, wherein the azPolicy field is composed of machine room identifications, the same machine room identifications are adjacent, and the initial copy number is the number of the machine room identifications;
determining a main computer room based on the mainAz field, wherein the main computer room is used for deploying redundant copies of the files to be deployed;
if the number of the currently deployed copies is less than or equal to the initial number of copies, sequentially deploying the copies of the file to be deployed based on the azPolicy field;
if the number of the current deployed copies is more than the number of the initial copies, calculating the number of redundant copies, sequentially deploying the copies of the initial copies of the plurality of files to be deployed based on the azPolicy field, and deploying the copies of the plurality of files to be deployed of the redundant copies in the mainframe room.
According to the method for placing the distributed storage copies across the machine room based on the copy redundancy, the modifying of the copy placing strategy of the file to be deployed comprises adding copies, and the modifying of the copy placing strategy of the file to be deployed based on the access data of the file to be deployed comprises the following steps: based on the access data of the file to be deployed, a client initiates a copy adding request, wherein the copy adding request comprises the number of copies to be added; responding to a copy adding request initiated by a client, and determining the number of copies needing to be added by the main metadata node; and modifying the azPolicy field of the current copy placement strategy according to the sequence from left to right based on the number of the copies needing to be added and the number of the currently deployed copies to obtain the modified copy placement strategy.
According to the method for placing the distributed storage copies across the machine room based on the copy redundancy, the modifying the copy placement strategy of the file to be deployed comprises deleting the copies, and the modifying the copy placement strategy of the file to be deployed based on the access data of the file to be deployed comprises the following steps: based on the access data of the file to be deployed, a client initiates a copy deleting request, wherein the copy deleting request comprises the number of copies to be deleted; responding to a request for deleting copies initiated by a client, and determining the number of copies needing to be deleted by the main metadata node; and modifying the azPolicy field of the current copy placement strategy according to the sequence from right to left based on the number of copies to be deleted and the number of currently deployed copies to obtain the modified copy placement strategy.
According to the distributed storage copy cross-computer room placement method based on copy redundancy, the method further comprises the following steps:
responding to a computing task request initiated by a client, and determining an original file matched with the computing task request by the main metadata node; the computing task request comprises reading an original file, writing an intermediate result, reading the intermediate result and writing a final result;
acquiring a copy placing strategy of the original file;
and modifying the copy placement strategy of the original file, so that an intermediate result generated after the client reads the original file is stored in the second machine room.
According to the method for placing the distributed storage copies across the machine rooms based on the copy redundancy, the copy placing strategy of the original file is modified, so that an intermediate result generated after the original file is read by a client is stored in the second machine room, and the method comprises the following steps: adding a temporary file placing strategy, wherein the temporary file placing strategy comprises the following steps: mainAz, native write, azPolicy, disableReductantDelete, disableBlockRecovery fields, and the native write field is set to true; the temporary file placing strategy is used for storing an intermediate result generated after the client reads the original file.
According to the method for placing the distributed storage copies across the machine rooms based on the copy redundancy, the first machine room is any one of the plurality of machine rooms of the distributed storage system, and the second machine room is the machine room where the client initiating the write-in request is located or the machine room where the client initiating the calculation task request is located.
In a second aspect, the present application further provides a device for placing distributed storage replicas across computer rooms based on replica redundancy, where the device is applied to a distributed storage system, and the device includes:
the node deployment module is used for deploying a plurality of metadata nodes into a first machine room in the distributed storage system and determining a main metadata node;
a copy setting module, configured to set a copy placement policy for a file to be deployed, where the copy placement policy includes: mainAz, native write, azPolicy, disableRedundant delete, disableBlockRecovery fields; the main computer room comprises a main field, a native write field, an azPolicy field, a disableReductantDelete field and a disableBlockRecovery field, wherein the main field is used for determining a main computer room, the native write field is used for determining whether a copy is written into a second computer room, the azPolicy field is used for determining copy distribution, the disableReductantDelete field is used for temporarily forbidding a redundant copy deletion mechanism under a certain directory, and the disableBlockRecovery field is used for temporarily forbidding a lost copy recovery mechanism under a certain directory;
the main metadata node completes the cross-machine room placement of the file to be deployed according to the copy placement strategy of the file to be deployed;
and the copy modification module is used for acquiring the access data of the file to be deployed and modifying the copy placement strategy of the file to be deployed based on the access data of the file to be deployed.
In a third aspect, an embodiment of the present application further provides an electronic device, where the electronic device includes a memory and a processor, where the memory stores a computer program, and when the processor runs the computer program, the processor executes the steps in any implementation manner of the above method for placing distributed storage replicas across computer rooms based on replica redundancy.
In a fourth aspect, an embodiment of the present application further provides a readable storage medium, where a computer program is stored in the readable storage medium, and when the computer program runs on a processor, the computer program executes the steps in any implementation manner of the above method for placing distributed storage replicas across a computer room based on replica redundancy.
To sum up, the distributed storage replica cross-machine-room placement method and device based on the replica redundancy, provided by the embodiment of the application, analyze a scene generating cross-machine-room flow, and set a multi-field replica placement strategy, so that a distributed storage system can sense the distribution situation of the replicas through the replica placement strategy, and meanwhile, the replica placement strategy also solves the cross-machine-room flow under the scenes of file reading, file writing, replica recovery, replica deletion and computing node task flow, thereby greatly reducing the cross-machine-room flow, and providing powerful support for realizing stable and reliable multi-cloud deployment.
Drawings
In order to more clearly illustrate the technical solutions in the present application or the prior art, the drawings needed for the description of the embodiments or the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
Fig. 1 is a schematic flowchart of a method for placing distributed storage replicas across computer rooms based on replica redundancy according to the present application;
fig. 2 is a schematic flowchart of a method for optimizing task traffic of a compute node according to the present application;
FIG. 3 is a schematic structural diagram of a device for placing distributed storage replicas across a computer room based on replica redundancy, provided by the present application;
fig. 4 is a schematic structural diagram of an electronic device provided in an embodiment of the present application.
Icon: 300-a network topology system; 310-node deployment module; 320-copy setting module; 330-copy placement module; 340-a replica modification module; 400-an electronic device; 410-a memory; 420-a processor; 430-bus.
Detailed Description
To make the objects, technical solutions and advantages of the present application clearer, the technical solutions of the present application will be described clearly and completely with reference to the accompanying drawings in the present application, and it is obvious that the described embodiments are some, but not all embodiments of the present application. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
Distributed storage system: massive file storage service is provided, files are organized in a directory tree form, and a linux path-like access mode is provided for users. Common open source implementations are HDFS (Hadoop Distributed File System), alluxio, and the like.
Distributed storage based on replica redundancy: for distributed file storage, a file has multiple copies, each copy is independently available, and as long as one copy is available, the file is considered to be available.
A metadata node: the distributed storage is used for storing nodes of metadata, and receiving metadata requests of users, such as directory viewing, directory creation, directory deletion and the like. The real data read-write service is not provided, a plurality of metadata nodes are generally provided, and the high-availability service is provided in a main/standby mode.
The data node: the distributed storage is used for the nodes for actually storing data, and receiving the requests of reading and writing data of a user, such as reading files, writing files and the like.
Territory (Region): mainly a physical region such as a city, county. A Region will have multiple rooms. The cost and latency of the private line across Region will be high.
Machine room (Zone): a physical machine room, typically several thousands to several tens of thousands of machines, will constitute a machine room.
Available Zone (Available Zone, abbreviated as AZ): refers to an available area consisting of a plurality of zones. The AZ and AZ are connected by a dedicated line, and the distance between the AZ is not too far (< 100 kilometers) generally. Generally speaking, the cross-machine room flow refers to the special line flow between AZs.
The method is used for analyzing and summarizing the cross-machine room flow, and the cross-machine room flow generally occurs in the following five scenes of file reading, file writing, copy recovery, copy deletion and computing node task flow.
(1) Scene one: document reading
A certain client initiates a file reading request, for example, the reading request is to read a file 1, but a copy of the file 1 does not exist in a machine room where the client is located, at this time, the machine room where the client is located needs to read a copy of the file 1 from other machine rooms, that is, the reading request is completed across the machine rooms, and a cross-machine-room flow is generated.
(2) Scene two: file writing
A certain client initiates a file write request, for example, a machine room where the client is located is the machine room 1, and the write request is to write two copies of the file 1 in the machine room 1 and the machine room 2, at this time, when writing the two copies of the file 1 in the machine room 2, cross-machine-room write is required, that is, cross-machine-room traffic is generated.
(3) Scene three: copy recovery
The distributed storage system generally has a lost copy recovery mechanism, and when a copy is damaged or a data node is down, the actual number of copies of the data node will be less than the number of copies recorded by the metadata node, and at this time, the distributed storage system will trigger and start the lost copy recovery mechanism. However, in the process of recovering the duplicate, the data node is generally copied with the copy data to recover the lost duplicate, and if the data nodes copied and copied by each other do not belong to the same computer room, cross-computer-room traffic is generated in the process of recovering the duplicate. Even in extreme cases, such as a large number of data nodes in a certain computer room are down, the lost copy recovery mechanism will generate larger cross-computer-room traffic.
(4) Scene four: deduplication
The distributed storage system generally has a redundant copy deletion mechanism, and when copies are added according to copy adjustment requirements, the actual number of copies of the data nodes is greater than the number of copies recorded by the metadata nodes, and at this time, the distributed storage system triggers and starts the redundant copy deletion mechanism. However, a time difference exists between the two processes of adding the duplicates and updating the number of the duplicates recorded by the metadata node, if the new duplicate is copied across the computer room and the new duplicate is regarded as redundant duplicate deletion, the new duplicate needs to be copied across the computer room again, and cross-computer-room traffic is generated under the condition.
(5) Scene five: computing node task traffic
When a user performs calculation based on distributed storage, since the calculation is generally divided into multiple steps, multiple data nodes in the same computer room may be involved in the calculation process performed by the client, and each data node may need to read related files across the computer room. If reading an original file, writing an intermediate result, reading an intermediate result, writing a final result, and the like in the calculation process involve reading and writing across a machine room, cross-machine-room traffic is generated.
Therefore, in order to solve the problem of cross-machine-room traffic generated in the above five scenarios, the copy placement strategy can be modified, so that the distributed storage system can sense the copy placement strategy to avoid generating the cross-machine-room traffic, and thus stable and reliable multi-cloud deployment is realized.
Fig. 1 is a schematic flowchart of a method for placing a distributed storage replica across a computer room based on replica redundancy, as shown in fig. 1, where the method is applied to a distributed storage system, and the method includes:
The first machine room is any one of a plurality of machine rooms of the distributed storage system.
Specifically, it may be understood that one computer room is randomly selected from a plurality of computer rooms in the distributed storage system as a first computer room, and a plurality of metadata nodes are deployed to the first computer room; one metadata node is selected as a main metadata node, and the rest metadata nodes are selected as slave metadata nodes, so that master-slave synchronization is guaranteed, the machine rooms are not crossed, and consistency and usability of multiple machine rooms are kept. In some examples, when a main metadata node goes down, the remaining metadata nodes may continue to improve service in place of the main metadata node. The setting method can refer to an HDFS (Hadoop distributed File System) double-node setting method, for example, two nodes are adopted, one is in an active state, the other is in a backup state, and when a metadata node of the active goes down, the backup is immediately switched to the active.
102, setting a copy placing strategy aiming at a file to be deployed, wherein the copy placing strategy comprises the following steps: mainAz, native write, azPolicy, disableReductantDelete, disableBlockRecovery fields.
The main server room comprises a main server room field, a native write field, an azPolicy field, a disableRedundant delete field and a disableBlockRecovery field, wherein the main server room field is used for determining a main server room, the native write field is used for determining whether a copy is written into a second server room, the azPolicy field is used for determining copy distribution, the disableRedundant delete field is used for temporarily forbidding a redundant copy deletion mechanism under a certain directory, and the disableBlockRecovery field is used for temporarily forbidding a lost copy recovery mechanism under a certain directory.
Specifically, it can be understood that different copy placement policies are set for different files or directories, a corresponding path key is set for each file or directory in the main metadata node, and the corresponding copy placement policy is stored under the path key, so that the copy placement policy of the file to be deployed is queried subsequently according to the path key. The copy placement policy may be described in the form of a json string, for example,
{
"/":
{
"mainAz": "az1",
"nativeWrite": false,
"azPolicy": "az1,az1,az2",
"disableRedundantDelete": false,
"disableBlockRecovery": false
}
}
wherein, az1 after the main az field indicates that the main computer room is az1; the azPolicy field is followed by 'az 1, az1 and az 2' to indicate that the copies are distributed as az1, az1 and az2 when the file is written, namely, one data node is selected from az1 to place a first copy, a second data node is selected from az1 to place a second copy, and a third data node is selected from az2 to place a third copy; the fields of native write, disableReductantDelete and disableBlockRecovery are all configured as false, which indicates that the functions of the fields are not opened. In addition, if the native write, disableRedundant delete, disableBlockRecovery fields are configured to true, it indicates that the function of the corresponding fields is turned on.
It should be noted that one computer room has multiple data nodes, and when multiple copies of a file to be deployed are deployed to a certain computer room according to a copy placement policy of the file to be deployed, the distributed storage system itself has a corresponding mechanism that can ensure that the multiple copies are placed at different nodes of the computer room.
Specifically, it can be further understood that the copy placement strategy can solve the cross-machine room traffic generated by scenario three or scenario four, and the main principle is as follows: by setting a disableRedundant delete field and configuring the field as true, a redundant duplicate deletion mechanism is disabled in the duplicate recovery process, the recovered duplicate is prevented from being identified as a redundant duplicate, and cross-machine-room flow is prevented from being generated in the duplicate recovery process; even under the extreme condition that a large number of data nodes are down, the replication recovery mechanism can cause a large amount of cross-machine-room flow in the cluster, the disableBlockRecovery field can help to temporarily disable the replication recovery mechanism, so that the distributed storage system has enough time to repair the down data nodes, after the nodes are repaired, the number of actually deployed copies is consistent with the number of copies recorded by the metadata nodes, and then the replication recovery cannot be carried out, and naturally, the cross-machine-room flow cannot be generated.
It should be noted that, each field in the copy placement policy may be understood as a configuration item, and the function of each field is specifically implemented in a code form, and the function of each field may also be modified in an actual application. In some embodiments, the copy placement policy may also be described in the form of a property file, a yaml file, or an xml file.
The write request comprises a file to be deployed and the number of current deployment copies.
Specifically, it may be understood that the main metadata node completes the cross-machine-room placement of the file to be deployed according to the copy placement policy of the file to be deployed, including the following steps:
and step 1031, acquiring a copy placement strategy and the current deployment copy number of the file to be deployed, and recording the strategies and the current deployment copy number in the main metadata node.
Step 1032, determining the initial copy number and copy distribution position of the file to be deployed based on the azPolicy field, wherein the azPolicy field is composed of machine room identifications, the same machine room identifications are adjacent, and the initial copy number is the number of the machine room identifications.
Specifically, the machine room identifiers may be az1, az2, az3, where the adjacent machine room identifiers are the same machine room and are distributed at adjacent positions, for example, an azPolicy field is "az1, az1, az2", when a file is written in a pipeline form, a data node is selected from az1 and written into a first copy; selecting a second data node from az1, and copying data from the first data node, wherein the first node and the second node belong to the same machine room az1, so that cross-private line flow cannot be generated; a third data node is selected from az2 and the data is copied from the second data node. Aiming at the file writing of the scene two, the copy distribution position is adjusted through an azPolicy field so as to realize that the same machine room is in an adjacent position, and the times of crossing the machine rooms are reduced when the file is written, so that the flow of crossing the machine rooms is reduced; and for the copy recovery of the third scenario, copy recovery is performed through the copy distribution sequence of the azPolicy fields, and because the same computer room is in an adjacent position, the data nodes which are down can be copied from the data nodes of the same computer room to recover the copies when the copies are recovered, so that cross-computer-room traffic is avoided.
And 1033, determining a main computer room based on the mainAz field, wherein the main computer room is used for deploying redundant copies of the file to be deployed.
Specifically, the main computer room may be configured in the main engine room, for example, if az1 is located after the main engine room z field, it represents az1, and if az2 is located after the main engine room z field, it represents az2. If the number of currently deployed copies exceeds the initial number of copies configured by azPolicy, the extra copies are placed in the main computer room to ensure that write failure cannot be caused by copy placement policy, thereby ensuring the write success rate of the copy placement policy.
Step 1034, if the number of the currently deployed copies is less than or equal to the initial number of copies, sequentially deploying the copies of the file to be deployed based on the azPolicy field; if the number of the current deployed copies is more than the number of the initial copies, calculating the number of redundant copies, sequentially deploying the copies of the initial copies of the plurality of files to be deployed based on the azPolicy field, and deploying the copies of the plurality of files to be deployed of the redundant copies in the mainframe room.
Specifically, for example, the file to be deployed in the write request is file 1, the mainAz field in the copy placement policy of file 1 is az1, the azPolicy field is "az1, az2, az2", and if the number of currently deployed copies of file 1 is 2, 1 copy of file 1 needs to be deployed in the machine rooms az1 and az2 in sequence; if the number of the current deployed copies of the file 1 is 5, the number of redundant copies needs to be calculated, the number of redundant copies is the difference between the number of deployed copies of the file to be deployed and the initial number of copies, namely the number of redundant copies is 2, then copies of the file 1 are sequentially deployed in machine rooms az1, az2 and az2 based on an azPolicy field, the main machine room is further determined to be az1 through a mainAz field, and the redundant 2 copies are deployed in the machine room az 1.
And 104, acquiring access data of the file to be deployed, and modifying a copy placement strategy of the file to be deployed based on the access data of the file to be deployed.
The access data of the file to be deployed can be monitored in real time or collected periodically. In some embodiments, the access data of the file to be deployed can be read through an audit log of the metadata node, and information such as a file accessed by each user, a request IP/machine room and the like is recorded in the audit log.
Specifically, it may be understood that the modifying the copy placement policy of the file to be deployed includes adding a copy, and the modifying the copy placement policy of the file to be deployed based on the access data of the file to be deployed includes:
step a1, based on the access data of the file to be deployed, a client initiates a copy adding request, wherein the copy adding request comprises the number of copies to be added.
In some embodiments, when monitoring that a certain file access requirement is increased, or when a certain machine room has a file which does not exist when being read, a copy adding request can be initiated to adjust a copy placing strategy and ensure the utilization rate of the file, meanwhile, data which may be read across the machine rooms are configured to be written into the multiple machine rooms, so that the data are ensured to have copies in each machine room, and only the copies of the machine rooms are read during reading, so that the flow of reading the file across the machine rooms is greatly reduced.
Step a2, responding to a copy adding request initiated by a client, and determining the number of copies needing to be added by the primary metadata node.
And a3, modifying the azPolicy field of the current copy placement strategy according to the sequence from left to right based on the number of the copies needing to be added and the number of the currently deployed copies to obtain the modified copy placement strategy.
For example, the azPolicy field of the copy placement policy of a certain file is "az1, az2, az2, az3", but the current number of deployed copies recorded by the primary metadata node is 2 (i.e. the distribution of the actually deployed copies is az1, az 2), and if the number of copies to be added is 1 in the copy adding request, the azPolicy field is modified according to the order from left to right to obtain modified azPolicy fields "az1, az2, az2"; if the copy adding request includes that the number of copies to be added is 2, modifying the azPolicy field according to the sequence from left to right to obtain modified azPolicy fields 'az 1, az2, az2, az 3'.
Specifically, it can be further understood that the modifying the copy placement policy of the file to be deployed includes deleting a copy, and the modifying the copy placement policy of the file to be deployed based on the access data of the file to be deployed includes:
step b1, based on the access data of the file to be deployed, a client side initiates a duplicate deletion request, wherein the duplicate deletion request comprises the number of duplicates needing to be deleted.
In some embodiments, when a decrease in a file access requirement is monitored, a delete copy request may be initiated to adjust the copy placement policy to ensure file utilization.
And b2, responding to a request for deleting the copies initiated by the client, and determining the number of copies needing to be deleted by the main metadata node.
And b3, modifying the azPolicy field of the current copy placement strategy according to the sequence from right to left based on the number of the copies to be deleted and the number of the currently deployed copies to obtain the modified copy placement strategy.
For example, the azPolicy field of the copy placement policy of a certain file is "az1, az2, az2, az3", the number of currently deployed copies recorded by the primary metadata node is 4, and if the copy deletion request includes that the number of copies to be deleted is 1, the azPolicy field is modified according to the sequence from right to left to obtain modified azPolicy fields "az1, az2, az2"; if the copy adding request contains the number of copies to be added of 2, the azPolicy field is modified according to the sequence from left to right to obtain the modified azPolicy fields 'az 1, az 2'.
It is to be noted that, after modifying the copy placement policy of the file to be deployed based on the access data of the file to be deployed, the primary metadata node needs to increase the number of copies based on the need, and update the number of currently deployed copies recorded in the primary metadata node, or update the number of currently deployed copies recorded in the primary metadata node based on the number of copies that needs to be deleted.
According to the distributed storage copy cross-machine-room placement method based on copy redundancy, a scene of cross-machine-room flow is analyzed, a multi-field copy placement strategy is set, and a main machine room for storing redundant copies is determined through a main access z field so as to ensure the writing success rate of file copies; determining a second machine room through the native write field, and storing the temporary file into the second machine room so as to avoid unnecessary cross-machine-room traffic generated in the computing task request; determining copy distribution through the azPolicy field, and strictly increasing or deleting copies according to the copy distribution sequence of the azPolicy field, so that cross-machine room flow in the file writing and copy recovery processes is reduced; the redundant duplicate deletion mechanism is disabled through the disableReductantDelete field, so that the duplicate recovered by error deletion in the duplicate recovery process is avoided, and the generation of cross-machine room flow is avoided; and disabling a lost copy recovery mechanism through a disableBlockRecovery field so as to avoid generating a large amount of cross-machine room traffic in the copy recovery process.
In summary, the distributed storage replica cross-machine-room placement method based on the replica redundancy provided by the embodiment of the application enables a distributed storage system to sense the replica distribution situation through the replica placement strategy by setting the multi-field replica placement strategy, and meanwhile, the replica placement strategy solves the cross-machine-room flow under the situations of file reading, file writing, replica recovery and replica deletion, greatly reduces the cross-machine-room flow, and provides a powerful support for realizing stable and reliable multi-cloud deployment.
Fig. 2 is a schematic flow diagram of a method for optimizing task traffic of a compute node provided in the present application, and as shown in fig. 2, the method further includes optimizing cross-room traffic generated in a scene five, and specifically includes the following steps:
The computing task request comprises reading an original file, writing an intermediate result, reading the intermediate result and writing a final result.
Specifically, it can be understood that the second machine room is a machine room where a client initiating a write request is located, or a machine room where a client initiating a computation task request is located; in some embodiments, the second computer room may also be a computer room where a client initiating the read request is located.
Specifically, it can be further understood that modifying the copy placement policy of the original file includes adding a temporary file placement policy, where the temporary file placement policy includes: mainAz, native write, azPolicy, disableReductantDelete, disableBlockRecovery fields, and the native write field is set to true; the temporary file placing strategy is used for storing an intermediate result generated after the client reads the original file.
The temporary file placement policy includes, for example:
{
"/tmp": {
"mainAz": "az1",
"nativeWrite": true,
"azPolicy": "az1,az1,az1",
"disableRedundantDelete": false,
"disableBlockRecovery": false
}
it is noted that the configuration priority of the native write field is highest, and the azPolicy field and the mainAz field are ignored when the native write field is configured to true. The native write field is a field specially set for a path which does not participate in downstream computing, and after the native write field is started, a client initiates a request in which computer room and writes a copy into which computer room. Therefore, if the client initiating the computing task request opens the native write field based on the temporary file placement strategy, the temporary files are all stored in the machine room (i.e., the second machine room) where the client initiating the computing task request is located, and the temporary files are deleted after the computing task request is completed.
According to the method for optimizing the task flow of the compute node, the copy placement strategy is modified by adding the temporary file placement strategy, the temporary file is stored in the second machine room by using the native write field, cross-machine-room flow is avoided, and therefore cross-machine-room flow under the task flow scene of the compute node is solved.
Fig. 3 is a schematic structural diagram of a device for placing distributed storage replicas across a computer room based on replica redundancy, which can be used to implement the method described in the foregoing embodiment. As shown in fig. 3, the apparatus is applied to a distributed storage system, and the apparatus includes:
a node deployment module 310, configured to deploy a plurality of metadata nodes to a first machine room in the distributed storage system, and determine a main metadata node;
a copy setting module 320, configured to set a copy placement policy for the file to be deployed, where the copy placement policy includes: mainAz, native write, azPolicy, disableRedundant delete, disableBlockRecovery fields; the main computer room comprises a main field, a native write field, an azPolicy field, a disableReductantDelete field and a disableBlockRecovery field, wherein the main field is used for determining a main computer room, the native write field is used for determining whether a copy is written into a second computer room, the azPolicy field is used for determining copy distribution, the disableReductantDelete field is used for temporarily forbidding a redundant copy deletion mechanism under a certain directory, and the disableBlockRecovery field is used for temporarily forbidding a lost copy recovery mechanism under a certain directory;
the replica placement module 330 is configured to, in response to a write request initiated by a client, complete cross-machine-room placement of the file to be deployed by the primary metadata node according to the replica placement policy of the file to be deployed;
the copy modification module 340 is configured to obtain access data of the file to be deployed, and modify a copy placement policy of the file to be deployed based on the access data of the file to be deployed.
For a detailed description of the device for placing the distributed storage copies across the machine room based on the copy redundancy, please refer to the description of the related method steps in the above embodiments, and repeated parts are not described again. The above-described apparatus embodiments are merely illustrative, and "modules" used herein as separate components may be a combination of software and/or hardware that implement predetermined functions, and may or may not be physically separate. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.
Fig. 4 is a schematic structural diagram of an electronic device provided in the present application, and as shown in fig. 4, the electronic device 400 includes: the memory 410 and the processor 420 are connected through the bus 430, the memory 410 stores a computer program, and the processor 420 reads and runs the computer program, so that the electronic device 400 can execute all or part of the flow of the method in the above embodiments to realize the placement of the distributed storage copy across the computer room based on copy redundancy.
The embodiment of the application also provides a readable storage medium, wherein a computer program is stored in the readable storage medium, and when the computer program runs on a processor, the computer program executes the steps in the method for placing the distributed storage copies across the machine room based on the copy redundancy.
It should be understood that the electronic device may be a personal computer, tablet computer, smart phone, or other electronic device with logic computing functionality; the readable storage medium may be a ROM (Read-Only Memory), a RAM (Random Access Memory), a magnetic disk, an optical disk, or the like.
Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not substantially depart from the spirit and scope of the present invention as set forth in the appended claims.
Claims (10)
1. A distributed storage replica cross-computer room placement method based on replica redundancy is applied to a distributed storage system, and is characterized by comprising the following steps:
deploying a plurality of metadata nodes to a first machine room in the distributed storage system, and determining a main metadata node;
setting a copy placing strategy aiming at a file to be deployed, wherein the copy placing strategy comprises the following steps: mainAz, native write, azPolicy, disableRedundant delete, disableBlockRecovery fields; the main server comprises a main server, a native server, an azPolicy field, a disableRedundant delete field and a disableBlockRecovery field, wherein the main server is used for determining a main server room, the native server is used for determining whether a copy is written into a second server room, the azPolicy field is used for determining copy distribution, the disableRedundant delete field is used for temporarily forbidding a redundant copy deletion mechanism under a certain directory, and the disableBlockRecovery field is used for temporarily forbidding a lost copy recovery mechanism under the certain directory;
responding to a write-in request initiated by a client, and finishing the cross-machine room placement of the file to be deployed by the main metadata node according to the copy placement strategy of the file to be deployed;
the method comprises the steps of obtaining access data of a file to be deployed, and modifying a copy placing strategy of the file to be deployed based on the access data of the file to be deployed.
2. The method according to claim 1, wherein the master metadata node completes the cross-machine-room placement of the file to be deployed according to the copy placement policy of the file to be deployed, and the method comprises the following steps:
acquiring a copy placement strategy and the current deployment copy number of a file to be deployed, and recording the copy placement strategy and the current deployment copy number in the main metadata node;
determining the initial copy number and copy distribution positions of the files to be deployed based on the azPolicy field, wherein the azPolicy field is composed of machine room identifications, the same machine room identifications are adjacent, and the initial copy number is the number of the machine room identifications;
determining a main computer room based on the mainAz field, wherein the main computer room is used for deploying redundant copies of the files to be deployed;
if the number of the currently deployed copies is less than or equal to the initial number of copies, sequentially deploying the copies of the file to be deployed based on the azPolicy field;
if the number of the current deployed copies is more than the number of the initial copies, calculating the number of redundant copies, sequentially deploying the copies of the initial copies of the plurality of files to be deployed based on the azPolicy field, and deploying the copies of the plurality of files to be deployed of the redundant copies in the mainframe room.
3. The method according to claim 2, wherein the modifying the copy placement policy of the file to be deployed comprises adding a copy, and the modifying the copy placement policy of the file to be deployed based on the access data of the file to be deployed comprises:
based on the access data of the file to be deployed, a client initiates a copy adding request, wherein the copy adding request comprises the number of copies to be added;
responding to a copy adding request initiated by a client, and determining the number of copies needing to be added by the main metadata node;
and modifying the azPolicy field of the current copy placement strategy according to the sequence from left to right based on the number of the copies needing to be added and the number of the currently deployed copies to obtain the modified copy placement strategy.
4. The method according to claim 2, wherein the modifying the copy placement policy of the file to be deployed comprises deleting a copy, and the modifying the copy placement policy of the file to be deployed based on the access data of the file to be deployed comprises:
based on the access data of the file to be deployed, a client initiates a duplicate deletion request, wherein the duplicate deletion request comprises the number of duplicates needing to be deleted;
responding to a request for deleting copies initiated by a client, and determining the number of copies needing to be deleted by the main metadata node;
and modifying the azPolicy field of the current copy placement strategy according to the sequence from right to left based on the number of copies to be deleted and the number of currently deployed copies to obtain the modified copy placement strategy.
5. The method of claim 1, further comprising:
responding to a computing task request initiated by a client, and determining an original file matched with the computing task request by the main metadata node; the computing task request comprises reading an original file, writing an intermediate result, reading the intermediate result and writing a final result;
acquiring a copy placing strategy of the original file;
and modifying the copy placement strategy of the original file, so that an intermediate result generated after the client reads the original file is stored in the second machine room.
6. The method according to claim 5, wherein the modifying the copy placement policy of the original file to enable an intermediate result generated after the client reads the original file to be stored in the second computer room comprises:
adding a temporary file placing strategy, wherein the temporary file placing strategy comprises the following steps: mainAz, native write, azPolicy, disableReductantDelete, disableBlockRecovery fields, and the native write field is set to true;
the temporary file placing strategy is used for storing an intermediate result generated after the client reads the original file.
7. The method according to claim 6, wherein the first machine room is any one of a plurality of machine rooms of the distributed storage system, and the second machine room is a machine room where a client initiating a write request is located, or a machine room where a client initiating a compute task request is located.
8. A device for placing distributed storage copies across a machine room based on copy redundancy is applied to a distributed storage system, and is characterized by comprising:
the node deployment module is used for deploying a plurality of metadata nodes to a first machine room in the distributed storage system and determining a main metadata node;
a copy setting module, configured to set a copy placement policy for a file to be deployed, where the copy placement policy includes: mainAz, native write, azPolicy, disableRedundant delete, disableBlockRecovery fields; the main server comprises a main server, a native server, an azPolicy field, a disableRedundant delete field and a disableBlockRecovery field, wherein the main server is used for determining a main server room, the native server is used for determining whether a copy is written into a second server room, the azPolicy field is used for determining copy distribution, the disableRedundant delete field is used for temporarily forbidding a redundant copy deletion mechanism under a certain directory, and the disableBlockRecovery field is used for temporarily forbidding a lost copy recovery mechanism under the certain directory;
the main metadata node completes the cross-machine room placement of the file to be deployed according to the copy placement strategy of the file to be deployed;
and the copy modification module is used for acquiring the access data of the file to be deployed and modifying the copy placement strategy of the file to be deployed based on the access data of the file to be deployed.
9. An electronic device, comprising a memory storing a computer program and a processor executing the computer program to perform the copy redundancy based distributed storage copy cross room placement method of any of claims 1 to 7.
10. A readable storage medium having stored therein a computer program which, when run on a processor, performs the replica redundancy based distributed storage replica cross room placement method of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310225524.XA CN115955488B (en) | 2023-03-10 | 2023-03-10 | Distributed storage copy cross-machine room placement method and device based on copy redundancy |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310225524.XA CN115955488B (en) | 2023-03-10 | 2023-03-10 | Distributed storage copy cross-machine room placement method and device based on copy redundancy |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115955488A true CN115955488A (en) | 2023-04-11 |
| CN115955488B CN115955488B (en) | 2023-05-23 |
Family
ID=85891196
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310225524.XA Active CN115955488B (en) | 2023-03-10 | 2023-03-10 | Distributed storage copy cross-machine room placement method and device based on copy redundancy |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115955488B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117240873A (en) * | 2023-11-08 | 2023-12-15 | 阿里云计算有限公司 | Cloud storage system, data reading and writing method, device and storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20110225121A1 (en) * | 2010-03-11 | 2011-09-15 | Yahoo! Inc. | System for maintaining a distributed database using constraints |
| CN103793534A (en) * | 2014-02-28 | 2014-05-14 | 苏州博纳讯动软件有限公司 | Distributed file system and implementation method for balancing storage loads and access loads of metadata |
| CN114003180A (en) * | 2021-11-11 | 2022-02-01 | 中国建设银行股份有限公司 | Data processing method and device based on cross-machine-room Hadoop cluster |
| CN114385561A (en) * | 2022-01-10 | 2022-04-22 | 北京沃东天骏信息技术有限公司 | File management method and device and HDFS system |
| CN115543965A (en) * | 2022-10-21 | 2022-12-30 | 北京火山引擎科技有限公司 | Cross-machine-room data processing method, device, storage medium, and program product |
-
2023
- 2023-03-10 CN CN202310225524.XA patent/CN115955488B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20110225121A1 (en) * | 2010-03-11 | 2011-09-15 | Yahoo! Inc. | System for maintaining a distributed database using constraints |
| CN103793534A (en) * | 2014-02-28 | 2014-05-14 | 苏州博纳讯动软件有限公司 | Distributed file system and implementation method for balancing storage loads and access loads of metadata |
| CN114003180A (en) * | 2021-11-11 | 2022-02-01 | 中国建设银行股份有限公司 | Data processing method and device based on cross-machine-room Hadoop cluster |
| CN114385561A (en) * | 2022-01-10 | 2022-04-22 | 北京沃东天骏信息技术有限公司 | File management method and device and HDFS system |
| CN115543965A (en) * | 2022-10-21 | 2022-12-30 | 北京火山引擎科技有限公司 | Cross-machine-room data processing method, device, storage medium, and program product |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117240873A (en) * | 2023-11-08 | 2023-12-15 | 阿里云计算有限公司 | Cloud storage system, data reading and writing method, device and storage medium |
| CN117240873B (en) * | 2023-11-08 | 2024-03-29 | 阿里云计算有限公司 | Cloud storage system, data reading and writing method, device and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115955488B (en) | 2023-05-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10552064B2 (en) | Enabling data integrity checking and faster application recovery in synchronous replicated datasets | |
| CN109729129B (en) | Configuration modification method of storage cluster system, storage cluster and computer system | |
| US7987158B2 (en) | Method, system and article of manufacture for metadata replication and restoration | |
| US11138156B2 (en) | Continuous data management system and operating method thereof | |
| US11093387B1 (en) | Garbage collection based on transmission object models | |
| US20150213100A1 (en) | Data synchronization method and system | |
| CN107018185B (en) | Synchronization method and device of cloud storage system | |
| EP3745269B1 (en) | Hierarchical fault tolerance in system storage | |
| US7702757B2 (en) | Method, apparatus and program storage device for providing control to a networked storage architecture | |
| US10628298B1 (en) | Resumable garbage collection | |
| CN103597463A (en) | Automatic configuration of a recovery service | |
| CA2893304A1 (en) | Data storage method, data storage apparatus, and storage device | |
| JP5292351B2 (en) | Message queue management system, lock server, message queue management method, and message queue management program | |
| CN115955488B (en) | Distributed storage copy cross-machine room placement method and device based on copy redundancy | |
| JP6070146B2 (en) | Information processing apparatus and backup method | |
| CN112579550B (en) | Metadata information synchronization method and system of distributed file system | |
| CN104951475A (en) | Distributed file system and implementation method | |
| US9037762B2 (en) | Balancing data distribution in a fault-tolerant storage system based on the movements of the replicated copies of data | |
| CN111752892B (en) | Distributed file system and implementation method, management system, equipment and medium thereof | |
| CN116389233A (en) | Container cloud management platform active-standby switching system, method and device and computer equipment | |
| JP6376626B2 (en) | Data storage method, data storage device, and storage device | |
| JP6251965B2 (en) | Information system and database recovery method | |
| CN115470041A (en) | Data disaster recovery management method and device | |
| KR102089710B1 (en) | Continous data mangement system and method | |
| CN111400098A (en) | Copy management method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |