CN113051102B - File backup method, device, system, storage medium and computer equipment - Google Patents
File backup method, device, system, storage medium and computer equipment Download PDFInfo
- Publication number
- CN113051102B CN113051102B CN201911364970.9A CN201911364970A CN113051102B CN 113051102 B CN113051102 B CN 113051102B CN 201911364970 A CN201911364970 A CN 201911364970A CN 113051102 B CN113051102 B CN 113051102B
- Authority
- CN
- China
- Prior art keywords
- backup
- data block
- data
- node
- file
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000003860 storage Methods 0.000 title claims abstract description 187
- 238000000034 method Methods 0.000 title claims abstract description 70
- 230000008569 process Effects 0.000 abstract description 13
- 238000004904 shortening Methods 0.000 abstract description 5
- 238000010586 diagram Methods 0.000 description 6
- 238000009826 distribution Methods 0.000 description 6
- 238000004590 computer program Methods 0.000 description 4
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 230000007547 defect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000009977 dual effect Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000011084 recovery Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1446—Point-in-time backing up or restoration of persistent data
- G06F11/1448—Management of the data involved in backup or backup restore
Abstract
In the technical scheme of the file backup method, the device, the system, the storage medium and the computer equipment provided by the embodiment of the invention, a file backup request is sent to a command node, so that the command node inquires the names of data blocks and node storage addresses in the file according to the backup request, and the names of the data blocks and the node storage addresses are sent to a backup server; generating the workload of the backup proxy server according to the number of the acquired data blocks and the number of the backup proxy servers; and the node storage address of the data block is sent to the backup proxy server matched with the name of the data block, so that the backup proxy server obtains the data block from the data node corresponding to the node storage address according to the node storage address, and backs up the data block to a backup medium, thereby reducing the network resource consumption of the command node in the file backup process, shortening the backup time, improving the bottom input and output of the data node, and further improving the running speed of the whole large data platform.
Description
[ field of technology ]
The present invention relates to the field of the internet, and in particular, to a method, an apparatus, a system, a storage medium, and a computer device for file backup.
[ background Art ]
With the rapid development of information technology, big data are more and more closely related to our lives, and influence on our life is also more and more great. Under the trend of rapid growth of unstructured data, many enterprises build large data platforms of themselves, and the large data platforms are mostly based on distributed system infrastructures (hadoops), such as internet enterprises BAT, hua-si, google, large operators and government systems in various places. However, natural disasters or human factors may cause the damage of a large data platform and cannot be operated for a long time, and if effective data backup and data recovery means and measures are not adopted, data loss is caused, and sometimes the loss is caused and cannot be compensated and estimated.
The Hadoop is provided with a redundant architecture, so that a plurality of files can be placed on different servers, the reliability of the files is greatly guaranteed, but the protection mechanism cannot solve the problems of manual misoperation, historical file tracing and the like, so that the importance of file backup on a big data platform based on the Hadoop is self-evident.
The traditional file backup method of the big data platform mainly comprises the steps of deploying a Hadoop client on a backup server, and acquiring files to a backup medium through a Hadoop client access command node (Namenode). However, the file backup method consumes large network resources of the command node (Namenode), has long backup time, and reduces the bottom input and output of part of data nodes (Datenode), thereby affecting the operation of the whole large data platform.
[ invention ]
In view of the above, the embodiments of the present invention provide a method, an apparatus, a system, a storage medium, and a computer device for file backup, which can solve the problem that the conventional file backup method consumes a large amount of network resources of a command node, has long backup time, and reduces the bottom input and output of a part of data nodes, thereby affecting the operation of the whole large data platform.
In a first aspect, an embodiment of the present invention provides a method for backing up a file, where the method includes:
sending a backup request of a file to a command node, wherein the file comprises at least one data block, the backup request comprises the name of the file, so that the command node inquires the name of the data block and the node storage address of the data block in the file according to the name of the file, and sends the name of the data block and the node storage address of the data block to the backup server;
Generating the workload of the backup proxy server according to the acquired number of the data blocks and the acquired number of the backup proxy servers;
and according to the workload of the backup proxy server, matching the names of the data blocks with different backup proxy servers, and sending the node storage addresses of the data blocks to the backup proxy server matched with the names of the data blocks, so that the backup proxy server obtains the data blocks from the data nodes corresponding to the node storage addresses according to the node storage addresses, and backs up the data blocks to a backup medium.
In another aspect, an embodiment of the present invention provides a method for backing up a file, where the method includes:
the method comprises the steps that a backup server sends a backup request of a file to a command node, wherein the file comprises at least one data block, and the backup request comprises the name of the file;
the command node inquires the name of the data block and the node storage address of the data block in the file according to the name of the file, and sends the name of the data block and the node storage address of the data block to the backup server;
The backup server generates the workload of the backup proxy server according to the acquired number of the data blocks and the acquired number of the backup proxy servers;
the backup server matches the names of the data blocks with different backup proxy servers according to the workload of the backup proxy servers, and sends the node storage addresses of the data blocks to the backup proxy servers matched with the names of the data blocks;
and the backup proxy server acquires the data block from the data node corresponding to the node storage address according to the node storage address, and backs up the data block to a backup medium.
Optionally, the backup server generates the workload of the backup proxy server according to the acquired number of the data blocks and the acquired number of the backup proxy servers, and specifically includes:
acquiring the number m of the data blocks and the number n of the backup proxy servers, wherein the m and the n comprise positive integers;
judging whether the number m of the data blocks is larger than the number n of the backup proxy servers or not;
if the number m of the data blocks is judged to be larger than the number n of the backup proxy servers, setting the workload of the backup proxy servers to be larger than or equal to INT (m/n) data blocks and smaller than or equal to INT (m/n) +1 data blocks;
And if the number m of the data blocks is less than or equal to the number n of the backup proxy servers, setting the workload of the backup proxy servers as 1 or 0 data blocks.
Optionally, the backup proxy server obtains the data block from the data node corresponding to the node storage address according to the node storage address, and backs up the data block to a backup medium, which specifically includes:
the backup proxy server obtains the data block from the data node corresponding to the node storage address according to the node storage address of the data block; judging whether the data block is stored in the backup medium or not;
if the backup proxy server judges that the data block is stored in the backup medium, acquiring a data block pointer of the data block and a backup medium storage address;
and if the backup proxy server judges that the data block is not stored in the backup medium, backing up the data block to the backup medium and acquiring a data block pointer and a backup medium storage address of the data block.
Optionally, the backup proxy server obtains the data block from the data node corresponding to the node storage address according to the node storage address, and after backing up the data block to a backup medium, the backup proxy server further includes:
The backup proxy server sends the backup result of the backup proxy server to the backup server;
and the backup server combines the backup results sent by the backup proxy server into a backup record, and stores the backup record into a database.
Optionally, the backup result includes a name of the data block, the data block pointer, the backup media storage address, and a backup completion condition.
In another aspect, an embodiment of the present invention provides a file backup apparatus, including:
the receiving and transmitting module is used for sending a backup request of a file to a command node, wherein the file comprises at least one data block, the backup request comprises the name of the file, so that the command node inquires the name of the data block and the node storage address of the data block in the file according to the name of the file, and the name of the data block and the node storage address of the data block are sent to the backup server;
the generation module is used for generating the workload of the backup proxy server according to the acquired number of the data blocks and the acquired number of the backup proxy servers;
The matching module is used for matching the names of the data blocks with different backup proxy servers according to the workload of the backup proxy servers;
the receiving and transmitting module is further configured to send the node storage address of the data block to the backup proxy server that matches the name of the data block, so that the backup proxy server obtains the data block from the data node corresponding to the node storage address according to the node storage address, and backs up the data block to a backup medium.
In another aspect, an embodiment of the present invention provides a file backup system, including: the system comprises a backup server, a command node, at least one backup proxy server, at least one data node and a backup medium;
the backup server is used for sending a backup request of a file to the command node, wherein the file comprises at least one data block, and the backup request comprises the name of the file;
the command node is configured to query, according to the name of the file, the name of the data block and a node storage address of the data block in the file, and send the name of the data block and the node storage address of the data block to the backup server;
The backup server is further configured to generate workload of the backup proxy server according to the acquired number of the data blocks and the acquired number of the backup proxy servers; according to the workload of the backup proxy server, the names of the data blocks are matched with different backup proxy servers, and the node storage addresses of the data blocks are sent to the backup proxy server matched with the names of the data blocks;
and the backup proxy server is used for acquiring the data block from the data node corresponding to the node storage address according to the node storage address and backing up the data block to a backup medium.
On the other hand, the embodiment of the invention provides a storage medium, which comprises a stored program, wherein the device where the storage medium is located is controlled to execute the file backup method when the program runs.
In another aspect, an embodiment of the present invention provides a computer device, including a memory and a processor, where the memory is configured to store information including program instructions, and the processor is configured to control execution of the program instructions, where the program instructions, when loaded and executed by the processor, implement the steps of the file backup method described above.
In the technical scheme of the file backup method, the device, the system, the storage medium and the computer equipment provided by the embodiment of the invention, a file backup request is sent to a command node, so that the command node inquires the names of data blocks and node storage addresses in the file according to the backup request, and the names of the data blocks and the node storage addresses are sent to a backup server; generating the workload of the backup proxy server according to the number of the acquired data blocks and the number of the backup proxy servers; and the node storage address of the data block is sent to the backup proxy server matched with the name of the data block, so that the backup proxy server obtains the data block from the data node corresponding to the node storage address according to the node storage address, and backs up the data block to a backup medium, thereby reducing the network resource consumption of the command node in the file backup process, shortening the backup time, improving the bottom input and output of the data node, and further improving the running speed of the whole large data platform.
[ description of the drawings ]
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings that are needed in the embodiments will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flowchart of a method for backing up files according to an embodiment of the present invention;
FIG. 2 is a flowchart of a method for backing up files according to another embodiment of the present invention;
FIG. 3 is a flowchart of a method for backing up files according to another embodiment of the present invention;
FIG. 4 is a flowchart of a method for backing up files according to another embodiment of the present invention;
FIG. 5 is a schematic diagram illustrating a structure of a file backup apparatus according to an embodiment of the present invention;
FIG. 6 is a schematic diagram illustrating a file backup system according to an embodiment of the present invention;
fig. 7 is a schematic diagram of a computer device according to an embodiment of the present invention.
[ detailed description ] of the invention
For a better understanding of the technical solution of the present invention, the following detailed description of the embodiments of the present invention refers to the accompanying drawings.
It should be understood that the described embodiments are merely some, but not all, embodiments of the invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The terminology used in the embodiments of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used in this application and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.
It should be understood that the term "and/or" as used herein is merely one way of describing an association of associated objects, meaning that there may be three relationships, e.g., a and/or b, which may represent: the first and second cases exist separately, and the first and second cases exist separately. In addition, the character "/" herein generally indicates that the front and rear associated objects are an "or" relationship.
Fig. 1 is a flowchart of a method for backing up files according to an embodiment of the present invention, as shown in fig. 1, where the method includes:
step 102, sending a backup request of a file to a command node, wherein the file comprises at least one data block, the backup request comprises a file name, so that the command node queries the name of the data block and a node storage address of the data block in the file according to the file name, and sends the name of the data block and the node storage address of the data block to a backup server.
And 104, generating the workload of the backup proxy server according to the acquired number of the data blocks and the number of the backup proxy servers.
Step 106, according to the workload of the backup proxy server, the names of the data blocks are matched with different backup proxy servers; and sending the node storage address of the data block to a backup proxy server matched with the name of the data block, so that the backup proxy server obtains the data block from the data node corresponding to the node storage address according to the node storage address, and backups the data block to a backup medium.
In the technical scheme of the file backup method provided by the embodiment, a file backup request is sent to a command node, so that the command node queries the names of data blocks and node storage addresses in the file according to the backup request, and sends the names of the data blocks and the node storage addresses to a backup server; generating the workload of the backup proxy server according to the number of the acquired data blocks and the number of the backup proxy servers; and the node storage address of the data block is sent to the backup proxy server matched with the name of the data block, so that the backup proxy server obtains the data block from the data node corresponding to the node storage address according to the node storage address, and backs up the data block to a backup medium, thereby reducing the network resource consumption of the command node in the file backup process, shortening the backup time, improving the bottom input and output of the data node, and further improving the running speed of the whole large data platform.
Fig. 2 is a flowchart of a file backup method according to another embodiment of the present invention, as shown in fig. 2, where the method includes:
step 202, sending a backup request of a file to a command node, wherein the file comprises at least one data block, the backup request comprises a file name, so that the command node queries the name of the data block and a node storage address of the data block in the file according to the file name, and sends the name of the data block and the node storage address of the data block to a backup server.
In this embodiment, the steps are performed by the backup server.
In this embodiment, the backup server sends a backup request of the file to the command node, and the data size is very small, which has little influence on the production network of the command node.
Specifically, the command node needs to form a distributed file by using detailed information of the file, that is, distribution information of all data blocks in the file, and transmit the distributed file to the backup server. The distributed file comprises names of data blocks in the file and node storage addresses of the data blocks. Because the distribution information of the data blocks is recorded when files are created and redundancy is reconstructed in the Hadoop, the data blocks can be obtained through the Hadoop fsck/user/filename-files-blocks-locations-tracks command without too many bottom-layer input and output.
Table 1 distribution information of data blocks in file
| Data block name | Node memory address |
| b1 | 0x001 of data node 1 |
| b2 | 0x002 of data node 1 |
| b3 | 0x003 of data node 1 |
| b4 | 0x004 for data node n |
| b5 | 0x005 of data node 1 |
| b1’ | 0x001 of data node 2 |
| b2’ | 0x002 of data node 2 |
| b3’ | 0x003 of data node 2 |
| b4’ | 0x004 for data node 1 |
| b5’ | 0x005 of data node n |
| b1” | 0x001 of data node 3 |
| b2” | 0x002 of data node 3 |
| b3” | 0x003 of data node 3 |
| b4” | 0x004 for data node 2 |
| b5” | 0x005 of data node 3 |
Examples: the command node inquires that data blocks with the names b1, b2, b3, b4', b5 in the file are distributed on the data node 1 according to the record information of the distributed file system (Hadoop Distributed File System, HDFS); data blocks with the names of b1', b2', b3', b4' are distributed on the data node 2; data blocks with the names of b1', b2', b3', b5' are distributed on the data node 3; the data blocks with the names b4 and b5' are distributed on the data node n, 15 data blocks are distributed on 4 data nodes in total, and the node storage addresses of the data blocks are shown in table 1.
Step 204, obtaining the number m of data blocks and the number n of backup proxy servers, wherein m and n comprise positive integers.
In this embodiment, the backup server obtains the number of data blocks to be backed up according to the names of the data blocks and the node storage addresses of the data blocks sent by the command node. And after receiving the names of the data blocks and the node storage addresses of the data blocks sent by the command nodes, the backup server automatically generates corresponding backup task sequence numbers for the names and the node storage addresses of each data block.
Step 206, judging whether the number m of the data blocks is larger than the number n of the backup proxy servers, if so, executing step 208; if not, go to step 210.
Step 208, setting the workload of the backup proxy server to be greater than or equal to INT (m/n) data blocks and less than or equal to INT (m/n) +1 data blocks; and proceeds to step 212.
Step 210, the workload of the backup proxy server is set to 1 or 0 data blocks.
Step 212, according to the workload of the backup proxy server, the names of the data blocks are matched with different backup proxy servers; and sending the node storage address of the data block to a backup proxy server matched with the name of the data block, so that the backup proxy server obtains the data block from the data node corresponding to the node storage address according to the node storage address, backs up the data block to a backup medium, and sends the backup result of the backup proxy server to the backup server.
Specifically, according to the workload of the backup proxy server, the names of the data blocks are matched with different backup proxy servers through an optimal balancing algorithm. The allocation principle of the optimal balancing algorithm is to match the names of the data blocks to different backup proxy servers as much as possible.
TABLE 2 backup task assignment results
| Backup task sequence number | Name of data block | Data node | Backup proxy server |
| 5-1 | 1 | Data node 1 | Backup proxy server 1 |
| 5-2 | 2 | Data node 1 | Backup proxy server 1 |
| 5-3 | 3 | Data node 2 | Backup proxy server 2 |
| 5-4 | 4 | Data node n | Backup proxy server 3 |
| 5-5 | 5 | Data node 3 | Backup proxy server 2 |
For example: the backup server receives the distributed file sent by the command node, obtains 5 data blocks according to the distributed file and distributes the data blocks on 4 data nodes, and 3 backup proxy servers are obtained. The backup server distributes backup tasks through an optimal balance algorithm, the backup task distribution results are shown in table 2, and the names of the 5 data blocks are respectively 1, 2, 3, 4 and 5. The data blocks 1 and 2 are backed up to a backup medium by a backup proxy server 1 through a data node 1; the data block 3 is backed up to a backup medium by the backup proxy server 2 through the data node 2; the data block 4 is backed up to the backup medium by the backup proxy server 2 through the data node 3; the data blocks 5 are backed up by the backup proxy server 3 via the data nodes n onto the backup medium.
Step 214, combining the backup results sent by the backup proxy server into a backup record, and storing the backup record in the database.
In this embodiment, the backup result includes the name of the data block, the data block pointer, the backup media storage address, and the backup completion condition. The data block pointers are in one-to-one correspondence with the data blocks and the backup media storage addresses and are used for pointing to the backup media storage addresses of the data blocks.
The traditional file backup method mainly comprises the steps of deploying a Hadoop client on a backup server, and acquiring files to a backup medium through a Hadoop client access command node. The conventional file backup method has the following disadvantages:
1) The file is read only through the command node, so that the network resource consumption of the command node is high, and even the operation of the whole large data platform is influenced.
2) The command node transmits data in a single point, and for the environment with larger backup requirement, the whole backup time is too long and the backup performance is poor.
3) Based on file mode transmission, because of single file backup, the whole backup process commands the node to read the strategy unchanged, so that a plurality of files are transmitted to the command node through one or a few data nodes, and the influence on the bottom input and output of part of the data nodes is larger.
In this embodiment, an elastic parallel framework is introduced, files are backed up and stored on a backup medium in a manner that a plurality of proxy backup servers read data blocks from a plurality of data nodes, and finally a complete backup record is logically synthesized on the backup server. In the embodiment, the data blocks in the backup file are transmitted through a plurality of data nodes, so that the network transmission pressure of the command nodes can be reduced, the backup is reduced, and the bottom layer input and output are balanced on the data nodes, thereby improving the backup efficiency of the whole large data platform.
In the technical scheme of the file backup method provided by the embodiment, a file backup request is sent to a command node, so that the command node queries the names of data blocks and node storage addresses in the file according to the backup request, and sends the names of the data blocks and the node storage addresses to a backup server; generating the workload of the backup proxy server according to the number of the acquired data blocks and the number of the backup proxy servers; and the node storage address of the data block is sent to the backup proxy server matched with the name of the data block, so that the backup proxy server obtains the data block from the data node corresponding to the node storage address according to the node storage address, and backs up the data block to a backup medium, thereby reducing the network resource consumption of the command node in the file backup process, shortening the backup time, improving the bottom input and output of the data node, and further improving the running speed of the whole large data platform.
Fig. 3 is a flowchart of a file backup method according to another embodiment of the present invention, as shown in fig. 3, where the method includes:
step 302, the backup server sends a backup request of a file to the command node, wherein the file comprises at least one data block, and the backup request comprises a name of the file.
Step 304, the command node queries the names of the data blocks and the node storage addresses of the data blocks in the file according to the names of the file, and sends the names of the data blocks and the node storage addresses of the data blocks to the backup server.
And 306, the backup server generates the workload of the backup proxy server according to the acquired number of the data blocks and the number of the backup proxy servers.
And 308, the backup server matches the names of the data blocks with different backup proxy servers according to the workload of the backup proxy servers, and sends the node storage addresses of the data blocks to the backup proxy servers matched with the names of the data blocks.
And 310, the backup proxy server acquires the data block from the data node corresponding to the node storage address according to the node storage address, and backs up the data block to the backup medium.
In the technical scheme of the file backup method provided by the embodiment, a backup server sends a file backup request to a command node, wherein the file comprises at least one data block, and the backup request comprises the name of the file; the command node inquires the names of the data blocks in the file and the node storage addresses of the data blocks according to the names of the file, and sends the names of the data blocks and the node storage addresses of the data blocks to the backup server; the backup server generates the workload of the backup proxy server according to the number of the acquired data blocks and the number of the backup proxy servers; the backup server matches the names of the data blocks with different backup proxy servers according to the workload of the backup proxy servers, and sends the node storage addresses of the data blocks to the backup proxy servers matched with the names of the data blocks; the backup proxy server obtains the data blocks from the data nodes corresponding to the node storage addresses according to the node storage addresses, and backs up the data blocks to the backup medium, so that the network resource consumption of the command node in the file backup process can be reduced, the backup time is shortened, the bottom-layer input and output of the data nodes are improved, and the running speed of the whole large data platform is improved.
Fig. 4 is a flowchart of a file backup method according to another embodiment of the present invention, as shown in fig. 4, where the method includes:
step 402, the backup server sends a backup request of a file to the command node, wherein the file includes at least one data block, and the backup request includes a name of the file.
In this embodiment, the backup server sends a backup request of the file to the command node, and the data size is very small, which has little influence on the production network of the command node.
Step 404, the command node queries the names of the data blocks and the node storage addresses of the data blocks in the file according to the names of the file, and sends the names of the data blocks and the node storage addresses of the data blocks to the backup server.
Specifically, the command node needs to form a distributed file by using detailed information of the file, that is, distribution information of all data blocks in the file, and transmit the distributed file to the backup server. The distributed file comprises names of data blocks in the file and node storage addresses of the data blocks. Because the distribution information of the data blocks is recorded when files are created and redundancy is reconstructed in the Hadoop, the data blocks can be obtained through the Hadoop fsck/user/filename-files-blocks-locations-tracks command without too many bottom-layer input and output.
Step 406, the backup server generates workload of the backup proxy server according to the acquired number of data blocks and the number of backup proxy servers.
In this embodiment, the backup server obtains the number of data blocks to be backed up according to the names of the data blocks and the node storage addresses of the data blocks sent by the command node. And after receiving the names of the data blocks and the node storage addresses of the data blocks sent by the command nodes, the backup server automatically generates corresponding backup task sequence numbers for the names and the node storage addresses of each data block.
In this embodiment, step 406 specifically includes:
step 4062, obtaining the number m of data blocks and the number n of backup proxy servers, where m and n comprise positive integers.
Step 4064, judging whether the number m of the data blocks is greater than the number n of the backup proxy servers, if yes, executing step 4066; if not, go to step 4068.
Step 4066, setting the workload of the backup proxy server to be greater than or equal to INT (m/n) data blocks and less than or equal to INT (m/n) +1 data blocks; and proceeds to step 408.
Step 4068, the workload of the backup proxy server is set to 1 or 0 data blocks.
In step 408, the backup server matches the names of the data blocks with different backup proxy servers according to the workload of the backup proxy servers, and sends the node storage addresses of the data blocks to the backup proxy servers matched with the names of the data blocks.
Specifically, according to the workload of the backup proxy server, the names of the data blocks are matched with different backup proxy servers through an optimal balancing algorithm. The allocation principle of the optimal balancing algorithm is to match the names of the data blocks to different backup proxy servers as much as possible.
Step 410, the backup proxy server obtains the data block from the data node corresponding to the node storage address according to the node storage address, and backs up the data block to the backup medium.
In this embodiment, step 410 specifically includes:
step 4102, the backup proxy server obtains the data block from the data node corresponding to the node storage address according to the node storage address of the data block; judging whether the backup medium stores the data block or not, if so, executing step 4104; if not, go to step 4106.
Step 4104, the backup proxy server obtains the data block pointer and the backup media storage address of the data block, and proceeds to step 412.
In step 4106, the backup proxy server backs up the data blocks to the backup medium and obtains the data block pointers and backup medium storage addresses of the data blocks.
For example: the file comprises 5 data blocks, the names of the 5 data blocks are respectively 1, 2, 3, 4 and 5, and the contents of the 5 data blocks are respectively: d139, EF31, 7C4E, 876A, A B. The backup proxy server 1 obtains the contents of the data blocks 1 and 2 through the data node 1; the backup proxy server 2 obtains the content of the data block 3 through the data node 2; the backup proxy server 2 obtains the content of the data block 4 through the data node 3; the backup proxy server 3 obtains the content of the data block 5 via the data node n.
Table 3 backup media existing data block record table
Table 3 is a table of record of existing data blocks of the backup medium, and as can be seen from table 3, data blocks 1, 3, and 4 are already stored in the backup medium, and data blocks 2 and 5 are not stored in the backup medium. Thus, the backup proxy server 1 acquires the data block pointer fs11 of the data block 1 and the backup medium storage address 0xaaa1 from the backup medium; the backup proxy server 2 acquires a data block pointer fs13 of the data block 3 and a backup medium storage address 0xbbb3 from the backup medium; the backup proxy server 2 acquires the data block pointer fs14 of the data block 4 and the backup medium storage address 0xccc4 from the backup medium. Table 4 is a table of data block records in the backup medium after the backup is completed, as shown in table 4, the backup proxy server 1 backs up the content EF31 of the data block 2 to the backup medium, and obtains the data block pointer fs12 of the data block 2 and the backup medium storage address 0xeee2 from the backup medium; the backup proxy server 3 backs up the content a38B of the data block 5 to the backup medium, and acquires the data block pointer fs15 of the data block 5 and the backup medium storage address 0xeee3 from the backup medium.
Table 4 data block record table in backup medium after backup is completed
The data block pointers are in one-to-one correspondence with the data blocks and the backup media storage addresses and are used for pointing to the backup media storage addresses of the data blocks.
In this embodiment, a duplicate data block deleting function is added in the file backup process, that is, duplicate backup is not performed on the data blocks stored in the backup medium, so that the storage space of the backup medium is saved.
Step 412, the backup proxy server sends the backup result of the backup proxy server to the backup server.
In this embodiment, the backup result includes the name of the data block, the data block pointer, the backup media storage address, and the backup completion condition. The data block pointers are in one-to-one correspondence with the data blocks and the backup media storage addresses and are used for pointing to the backup media storage addresses of the data blocks.
In step 414, the backup server combines the backup results sent by the backup proxy server into a backup record, and stores the backup record in the database.
Table 5 backup records
| Backup task sequence number | Name of data block | Data block pointer | Backup media storage address | Backup completion |
| 5-1 | 1 | fs11 | 0xaaa1 | Yes |
| 5-2 | 2 | fs12 | 0xeee2 | Yes |
| 5-3 | 3 | fs13 | 0xbbb3 | Yes |
| 5-4 | 4 | fs14 | 0xccc4 | Yes |
| 5-5 | 5 | fs15 | 0xeee3 | Yes |
For example, the backup proxy server 1 transmits the data block pointer fs11, the backup media storage address 0xaaa1, and the backup completion condition of the data block 1, and the data block pointer fs12, the backup media storage address 0xeee2, and the backup completion condition of the data block 2 to the backup server; the backup proxy server 2 sends the data block pointer fs13 of the data block 3, the backup media storage address 0 xcbb 3 and the backup completion condition, and the data block pointer fs14 of the data block 4, the backup media storage address 0xccc4 and the backup completion condition to the backup server; the backup proxy server 3 sends the data block pointer fs15 of the data block 5, the backup media storage address 0xeee3 and the backup completion to the backup server. The backup server combines the backup results sent by all backup proxy servers into one backup record, as shown in table 5. In the case of the completion of the backup, "Yes" indicates that the backup is successful, and "No" indicates that the backup is failed.
In this embodiment, the backup server combines the backup results sent by the backup proxy server into a backup record, which is only a logical assembly, and the whole process is very short in time, and the occupation of system resources is very small.
On a large data platform, the data blocks of a file in the embodiment are acquired from different data nodes, and are backed up by different backup agents, so that the network pressure of a command node in the traditional file backup method can be relieved, the bandwidth can be fully utilized, the backup efficiency can be increased, and meanwhile, the hot node of the data node in the backup process can be reduced.
The embodiment adopts a dual parallel framework, namely, the data acquisition of the backup file is completed by a plurality of data nodes in parallel, the whole backup task is written into the backup medium by a plurality of backup proxy servers in parallel, and finally, the whole backup record is logically synthesized on the backup server. The technical scheme provided by the embodiment not only inherits the protection of the data in the large data platform by the traditional file backup method, but also makes up the defects of shortage of network resources of command nodes, unbalanced input and output of data nodes and the like in the traditional file backup process, so that the file backup of the whole large data platform becomes more feasible and more efficient.
In the technical scheme of the file backup method provided by the embodiment, a backup server sends a file backup request to a command node, wherein the file comprises at least one data block, and the backup request comprises the name of the file; the command node inquires the names of the data blocks in the file and the node storage addresses of the data blocks according to the names of the file, and sends the names of the data blocks and the node storage addresses of the data blocks to the backup server; the backup server generates the workload of the backup proxy server according to the number of the acquired data blocks and the number of the backup proxy servers; the backup server matches the names of the data blocks with different backup proxy servers according to the workload of the backup proxy servers, and sends the node storage addresses of the data blocks to the backup proxy servers matched with the names of the data blocks; the backup proxy server obtains the data blocks from the data nodes corresponding to the node storage addresses according to the node storage addresses, and backs up the data blocks to the backup medium, so that the network resource consumption of the command node in the file backup process can be reduced, the backup time is shortened, the bottom-layer input and output of the data nodes are improved, and the running speed of the whole large data platform is improved.
Fig. 5 is a schematic structural diagram of a file backup apparatus according to an embodiment of the present invention, as shown in fig. 5, where the apparatus includes: a transceiver module 51, a generation module 52 and a matching module 53.
The transceiver module 51 is configured to send a backup request of a file to the command node, where the file includes at least one data block, and the backup request includes a name of the file, so that the command node queries the name of the data block and a node storage address of the data block in the file according to the name of the file, and sends the name of the data block and the node storage address of the data block to the backup server.
The file backup device provided in this embodiment includes a backup server.
The generating module 52 is configured to generate a workload of the backup proxy server according to the acquired number of data blocks and the number of backup proxy servers.
In this embodiment, the generating module 52 specifically includes:
an acquisition sub-module 521, configured to acquire the number m of data blocks and the number n of backup proxy servers.
A determining sub-module 522 is configured to determine whether the number m of data blocks is greater than the number n of backup proxy servers.
A setting sub-module 523, configured to set the workload of the backup proxy server to be greater than or equal to INT (m/n) data blocks and less than or equal to INT (m/n) +1 data blocks when the judging sub-module 522 judges that the number m of data blocks is greater than the number n of the backup proxy server; and continuing to perform the operation of matching the names of the data blocks with different backup proxy servers according to the workload of the backup proxy servers.
The setting sub-module 523 is further configured to set the workload of the backup proxy server to 1 or 0 data blocks when the judging sub-module 522 judges that the number m of data blocks is less than or equal to the number n of backup proxy servers; and continuing to perform the operation of matching the names of the data blocks with different backup proxy servers according to the workload of the backup proxy servers.
And the matching module 53 is configured to match the names of the data blocks with different backup proxy servers according to the workload of the backup proxy servers.
Specifically, according to the workload of the backup proxy server, the names of the data blocks are matched with different backup proxy servers through an optimal balancing algorithm. The allocation principle of the optimal balancing algorithm is to match the names of the data blocks to different backup proxy servers as much as possible.
The transceiver module 51 is further configured to send the node storage address of the data block to a backup proxy server that matches the name of the data block, so that the backup proxy server obtains the data block from the data node corresponding to the node storage address according to the node storage address, and backs up the data block to the backup medium.
In this embodiment, the backup result includes the name of the data block, the data block pointer, the backup media storage address, and the backup completion condition. The data block pointers are in one-to-one correspondence with the data blocks and the backup media storage addresses and are used for pointing to the backup media storage addresses of the data blocks.
The file backup device provided in this embodiment may be used to implement the file backup method in fig. 1 to 2, and the detailed description may refer to the embodiment of the file backup method, and the description will not be repeated here.
In the technical scheme of the file backup device provided by the embodiment of the invention, a file backup request is sent to a command node, so that the command node inquires the names of data blocks and node storage addresses in the file according to the backup request, and the names of the data blocks and the node storage addresses are sent to a backup server; generating the workload of the backup proxy server according to the number of the acquired data blocks and the number of the backup proxy servers; and the node storage address of the data block is sent to the backup proxy server matched with the name of the data block, so that the backup proxy server obtains the data block from the data node corresponding to the node storage address according to the node storage address, and backs up the data block to a backup medium, thereby reducing the network resource consumption of the command node in the file backup process, shortening the backup time, improving the bottom input and output of the data node, and further improving the running speed of the whole large data platform.
Fig. 6 is a schematic structural diagram of a file backup system according to an embodiment of the present invention, as shown in fig. 6, where the system includes: a backup server 61, a command node 62, at least one backup proxy server 63, at least one data node 64, and a backup medium 65.
The backup server 61 is configured to send a backup request for a file to the command node 62, the file comprising at least one data block, the backup request comprising a name of the file.
A command node 62, configured to query a name of a data block in the file and a node storage address of the data block according to the name of the file, and send the name of the data block and the node storage address of the data block to the backup server 61;
the backup server 61 is further configured to generate a workload of the backup proxy server 63 according to the acquired number of data blocks and the number of backup proxy servers 63; according to the workload of the backup proxy server 63, the names of the data blocks are matched with different backup proxy servers 63, and the node storage addresses of the data blocks are sent to the backup proxy server 63 matched with the names of the data blocks.
In this embodiment, the number of acquired data blocks includes m, and the number of acquired backup proxy servers includes n; when m > n, the workload of the backup proxy server is greater than or equal to INT (m/n) data blocks and less than or equal to INT (m/n) +1 data blocks; when m.ltoreq.n, the workload of the backup proxy server includes 1 or 0 data blocks.
Specifically, the names of the data blocks are matched with different backup proxy servers 63 by an optimal balancing algorithm according to the workload of the backup proxy servers 63. The allocation principle of the optimal balancing algorithm is to match the names of the data blocks to different backup proxy servers as much as possible.
The backup proxy server 63 is configured to obtain, according to the node storage address, a data block from the data node 64 corresponding to the node storage address, and backup the data block to the backup medium 65.
In this embodiment, the backup proxy server 63 is specifically configured to: acquiring a data block from a data node corresponding to the node storage address according to the node storage address of the data block; judging whether the backup medium stores data blocks or not; if the data block is stored in the backup medium, the backup proxy server acquires the data block pointer of the data block and the backup medium storage address, and continues to perform the operation of sending the backup result of the backup proxy server 63 to the backup server 61; if no data block is stored in the backup medium, backing up the data block to the backup medium and acquiring a data block pointer and a backup medium storage address of the data block.
In this embodiment, the backup proxy server 63 is further configured to send the backup result of the backup proxy server 63 to the backup server 61.
In this embodiment, the backup result includes the name of the data block, the data block pointer, the backup media storage address, and the backup completion condition. The data block pointers are in one-to-one correspondence with the data blocks and the backup media storage addresses and are used for pointing to the backup media storage addresses of the data blocks.
In this embodiment, the backup server 61 is further configured to combine the backup results sent by the backup proxy server into a backup record, and store the backup record in the database.
The file backup system provided in this embodiment may be used to implement the file backup method in fig. 3 to 4, and the detailed description may refer to the embodiment of the file backup method, and the description will not be repeated here.
In the technical scheme of the file backup system provided by the embodiment of the invention, a backup server sends a file backup request to a command node, wherein the file comprises at least one data block, and the backup request comprises the name of the file; the command node inquires the names of the data blocks in the file and the node storage addresses of the data blocks according to the names of the file, and sends the names of the data blocks and the node storage addresses of the data blocks to the backup server; the backup server generates the workload of the backup proxy server according to the number of the acquired data blocks and the number of the backup proxy servers; the backup server matches the names of the data blocks with different backup proxy servers according to the workload of the backup proxy servers, and sends the node storage addresses of the data blocks to the backup proxy servers matched with the names of the data blocks; the backup proxy server obtains the data blocks from the data nodes corresponding to the node storage addresses according to the node storage addresses, and backs up the data blocks to the backup medium, so that the network resource consumption of the command node in the file backup process can be reduced, the backup time is shortened, the bottom-layer input and output of the data nodes are improved, and the running speed of the whole large data platform is improved.
Fig. 7 is a schematic diagram of a computer device according to an embodiment of the present invention. As shown in fig. 5, the computer device 20 of this embodiment includes: the processor 21, the memory 22, and the computer program 23 stored in the memory 22 and capable of running on the processor 21, where the computer program 23 is executed by the processor 21 to implement the file backup method in the embodiment, and is not repeated herein. Alternatively, the computer program, when executed by the processor 21, implements the functions of the embodiments applied to each model/unit in the file backup apparatus, and in order to avoid repetition, it is not described in detail herein.
Computer device 20 includes, but is not limited to, a processor 21, a memory 22. It will be appreciated by those skilled in the art that fig. 5 is merely an example of computer device 20 and is not intended to limit computer device 20, and may include more or fewer components than shown, or may combine certain components, or different components, e.g., a computer device may also include an input-output device, a network access device, a bus, etc.
The processor 21 may be a central processing unit (Central Processing Unit, CPU) or other general purpose processor, digital signal processor (Digital Signal Processor, DSP), application specific integrated circuit (Application Specific Integrated Circuit, ASIC), field programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware components, or the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The memory 22 may be an internal storage unit of the computer device 20, such as a hard disk or memory of the computer device 20. The memory 22 may also be an external storage device of the computer device 20, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card) or the like, which are provided on the computer device 20. Further, the memory 22 may also include both internal and external storage units of the computer device 20. The memory 22 is used to store computer programs and other programs and data required by the computer device. The memory 22 may also be used to temporarily store data that has been output or is to be output.
It will be clear to those skilled in the art that, for convenience and brevity of description, specific working procedures of the apparatus and units described above may refer to corresponding procedures in the foregoing method embodiments, which are not described herein again.
In the several embodiments provided by the present invention, it should be understood that the disclosed apparatus and method may be implemented in other manners. For example, the apparatus embodiments described above are merely illustrative, e.g., the division of the elements is merely a logical function division, and there may be additional divisions when actually implemented, e.g., multiple elements or components may be combined or integrated into another system, or some features may be omitted or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or units, which may be in electrical, mechanical or other form.
The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, each functional unit in the embodiments of the present invention may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in hardware plus software functional units.
The integrated units implemented in the form of software functional units described above may be stored in a computer readable storage medium. The software functional unit is stored in a storage medium, and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) or a Processor (Processor) to perform part of the steps of the methods according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
The foregoing description of the preferred embodiments of the invention is not intended to be limiting, but rather to enable any modification, equivalent replacement, improvement or the like to be made within the spirit and principles of the invention.
Claims (10)
1. A method of file backup, the method comprising:
sending a backup request of a file to a command node, wherein the file comprises at least one data block, the backup request comprises the name of the file, so that the command node inquires the name of the data block and the node storage address of the data block in the file according to the name of the file, and sends the name of the data block and the node storage address of the data block to the backup server;
generating the workload of the backup proxy server according to the acquired number of the data blocks and the acquired number of the backup proxy servers;
and according to the workload of the backup proxy server, matching the names of the data blocks with different backup proxy servers, and sending the node storage addresses of the data blocks to the backup proxy server matched with the names of the data blocks, so that the backup proxy server obtains the data blocks from the data nodes corresponding to the node storage addresses according to the node storage addresses, and backs up the data blocks to a backup medium.
2. A method of file backup, the method comprising:
the method comprises the steps that a backup server sends a backup request of a file to a command node, wherein the file comprises at least one data block, and the backup request comprises the name of the file;
the command node inquires the name of the data block and the node storage address of the data block in the file according to the name of the file, and sends the name of the data block and the node storage address of the data block to the backup server;
the backup server generates the workload of the backup proxy server according to the acquired number of the data blocks and the acquired number of the backup proxy servers;
the backup server matches the names of the data blocks with different backup proxy servers according to the workload of the backup proxy servers, and sends the node storage addresses of the data blocks to the backup proxy servers matched with the names of the data blocks;
and the backup proxy server acquires the data block from the data node corresponding to the node storage address according to the node storage address, and backs up the data block to a backup medium.
3. The method of claim 2, wherein the backup server generates the workload of the backup proxy server according to the acquired number of the data blocks and the number of the backup proxy servers, and specifically comprises:
acquiring the number m of the data blocks and the number n of the backup proxy servers, wherein the m and the n comprise positive integers;
judging whether the number m of the data blocks is larger than the number n of the backup proxy servers or not;
if the number m of the data blocks is judged to be larger than the number n of the backup proxy servers, setting the workload of the backup proxy servers to be larger than or equal to INT (m/n) data blocks and smaller than or equal to INT (m/n) +1 data blocks;
and if the number m of the data blocks is less than or equal to the number n of the backup proxy servers, setting the workload of the backup proxy servers as 1 or 0 data blocks.
4. The method according to claim 2, wherein the backup proxy server obtains the data block from the data node corresponding to the node storage address according to the node storage address, and backs up the data block to a backup medium, and specifically includes:
The backup proxy server obtains the data block from the data node corresponding to the node storage address according to the node storage address of the data block; judging whether the data block is stored in the backup medium or not;
if the backup proxy server judges that the data block is stored in the backup medium, acquiring a data block pointer of the data block and a backup medium storage address;
and if the backup proxy server judges that the data block is not stored in the backup medium, backing up the data block to the backup medium and acquiring a data block pointer and a backup medium storage address of the data block.
5. The file backup method according to claim 2 or 4, wherein the backup proxy server obtains the data block from a data node corresponding to the node storage address according to the node storage address, and further comprises, after backing up the data block to a backup medium:
the backup proxy server sends the backup result of the backup proxy server to the backup server;
and the backup server combines the backup results sent by the backup proxy server into a backup record, and stores the backup record into a database.
6. The method of claim 5, wherein the backup result includes a name of the data block, the data block pointer, the backup media storage address, and a backup completion.
7. A file backup apparatus, the apparatus comprising:
the receiving and transmitting module is used for sending a backup request of a file to a command node, wherein the file comprises at least one data block, the backup request comprises the name of the file, so that the command node inquires the name of the data block and the node storage address of the data block in the file according to the name of the file, and the name of the data block and the node storage address of the data block are sent to the backup server;
the generation module is used for generating the workload of the backup proxy server according to the acquired number of the data blocks and the acquired number of the backup proxy servers;
the matching module is used for matching the names of the data blocks with different backup proxy servers according to the workload of the backup proxy servers;
the receiving and transmitting module is further configured to send the node storage address of the data block to the backup proxy server that matches the name of the data block, so that the backup proxy server obtains the data block from the data node corresponding to the node storage address according to the node storage address, and backs up the data block to a backup medium.
8. A file backup system, the system comprising: the system comprises a backup server, a command node, at least one backup proxy server, at least one data node and a backup medium;
the backup server is used for sending a backup request of a file to the command node, wherein the file comprises at least one data block, and the backup request comprises the name of the file;
the command node is configured to query, according to the name of the file, the name of the data block and a node storage address of the data block in the file, and send the name of the data block and the node storage address of the data block to the backup server;
the backup server is further configured to generate workload of the backup proxy server according to the acquired number of the data blocks and the acquired number of the backup proxy servers; according to the workload of the backup proxy server, the names of the data blocks are matched with different backup proxy servers, and the node storage addresses of the data blocks are sent to the backup proxy server matched with the names of the data blocks;
and the backup proxy server is used for acquiring the data block from the data node corresponding to the node storage address according to the node storage address and backing up the data block to a backup medium.
9. A storage medium comprising a stored program, wherein the program, when run, controls a device on which the storage medium resides to perform the file backup method of claim 1.
10. A computer device comprising a memory for storing information including program instructions and a processor for controlling execution of the program instructions, wherein the program instructions, when loaded and executed by the processor, implement the steps of the file backup method of claim 1.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911364970.9A CN113051102B (en) | 2019-12-26 | 2019-12-26 | File backup method, device, system, storage medium and computer equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911364970.9A CN113051102B (en) | 2019-12-26 | 2019-12-26 | File backup method, device, system, storage medium and computer equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113051102A CN113051102A (en) | 2021-06-29 |
| CN113051102B true CN113051102B (en) | 2024-03-19 |
Family
ID=76506013
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911364970.9A Active CN113051102B (en) | 2019-12-26 | 2019-12-26 | File backup method, device, system, storage medium and computer equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113051102B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113535478B (en) * | 2021-07-15 | 2024-01-02 | 天翼云科技有限公司 | Data backup method and device, storage medium and electronic equipment |
| CN113742135B (en) * | 2021-08-24 | 2023-11-10 | 咪咕音乐有限公司 | Data backup method, device and computer readable storage medium |
| US11953995B1 (en) * | 2022-11-04 | 2024-04-09 | Bank Of America Corporation | Centralized data backup platform supporting multiple data environments |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103701661A (en) * | 2013-12-23 | 2014-04-02 | 浪潮(北京)电子信息产业有限公司 | Method and system for realizing node monitoring |
| US9747168B2 (en) * | 2014-10-15 | 2017-08-29 | International Business Machines Corporation | Data block based backup |
| CN110109889A (en) * | 2019-05-09 | 2019-08-09 | 重庆大学 | A kind of distributed memory file management system |
| CN110198346A (en) * | 2019-05-06 | 2019-09-03 | 北京三快在线科技有限公司 | Method for reading data, device, electronic equipment and readable storage medium storing program for executing |
-
2019
- 2019-12-26 CN CN201911364970.9A patent/CN113051102B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103701661A (en) * | 2013-12-23 | 2014-04-02 | 浪潮(北京)电子信息产业有限公司 | Method and system for realizing node monitoring |
| US9747168B2 (en) * | 2014-10-15 | 2017-08-29 | International Business Machines Corporation | Data block based backup |
| CN110198346A (en) * | 2019-05-06 | 2019-09-03 | 北京三快在线科技有限公司 | Method for reading data, device, electronic equipment and readable storage medium storing program for executing |
| CN110109889A (en) * | 2019-05-09 | 2019-08-09 | 重庆大学 | A kind of distributed memory file management system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113051102A (en) | 2021-06-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| AU2016405587B2 (en) | Splitting and moving ranges in a distributed system | |
| US20220188276A1 (en) | Metadata journal in a distributed storage system | |
| CN113051102B (en) | File backup method, device, system, storage medium and computer equipment | |
| CN113672350B (en) | Application processing method and device and related equipment | |
| US20030014522A1 (en) | Activating a volume group without a quorum of disks in the volume group being active | |
| US20170085653A1 (en) | Method, device and system for message distribution | |
| CN111880967A (en) | File backup method, device, medium and electronic equipment in cloud scene | |
| CN102255866A (en) | Method and device for downloading data | |
| CN111930489B (en) | Task scheduling method, device, equipment and storage medium | |
| CN111651424B (en) | Data processing method, device, data node and storage medium | |
| CN111049913B (en) | Data file transmission method and device, storage medium and electronic equipment | |
| CN112433812A (en) | Method, system, equipment and computer medium for virtual machine cross-cluster migration | |
| US20070067488A1 (en) | System and method for transferring data | |
| CN116540938A (en) | Data reading method, device, distributed storage system, equipment and storage medium | |
| US20200348980A1 (en) | Asynchronous and synchronous transmit priority mechanism | |
| CN111104456A (en) | Data persistence storage method and device, computer equipment and storage medium | |
| CN115309491A (en) | Logic algorithm of platform system | |
| US20210240399A1 (en) | Storage system with continuous data verification for synchronous replication of logical storage volumes | |
| CN112131229A (en) | Block chain-based distributed data access method and device and storage node | |
| CN111966650A (en) | Operation and maintenance big data sharing data table processing method and device and storage medium | |
| US20210185123A1 (en) | Distributed information storage system, methods and programs | |
| US11687542B2 (en) | Techniques for in-memory data searching | |
| US11288004B1 (en) | Consensus-based authority selection in replicated network-accessible block storage devices | |
| CN114911421B (en) | Data storage method, system, device and storage medium based on CSI plug-in | |
| CN113703676B (en) | Calling method, device, equipment and medium for double live volumes |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |