CN106484566B - NAS data backup and file fine-grained browsing recovery method based on NDMP protocol - Google Patents
NAS data backup and file fine-grained browsing recovery method based on NDMP protocol Download PDFInfo
- Publication number
- CN106484566B CN106484566B CN201610860017.3A CN201610860017A CN106484566B CN 106484566 B CN106484566 B CN 106484566B CN 201610860017 A CN201610860017 A CN 201610860017A CN 106484566 B CN106484566 B CN 106484566B
- Authority
- CN
- China
- Prior art keywords
- ndmp
- data
- server
- backup
- nas
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1446—Point-in-time backing up or restoration of persistent data
- G06F11/1458—Management of the backup or restore process
- G06F11/1464—Management of the backup or restore process for networked environments
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1446—Point-in-time backing up or restoration of persistent data
- G06F11/1458—Management of the backup or restore process
- G06F11/1469—Backup restoration techniques
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention relates to a NAS data backup and file fine-grained browsing recovery method based on an NDMP protocol, which backs up data in NAS equipment into a backup all-in-one machine through the NDMP protocol, wherein a DMA (direct memory access) module and a first NDMP server are operated in the backup all-in-one machine, a second NDMP server is operated in the NAS equipment, and the backup method comprises the following steps: A1) establishing control connection between the DMA module and the first NDMP server, control connection between the DMA module and the second NDMP server and data connection between the first NDMP server and the second NDMP server; A2) the DMA module sends a backup instruction to the second NDMP server and sends a recovery instruction to the first NDMP server; A3) and the first NDMP server receives the NDMP data stream sent by the second NDMP server, and writes the file object analyzed by the NDMP data stream into the object file system to complete the backup. Compared with the prior art, the method has the advantages of being beneficial to unified storage and management of data and the like.
Description
Technical Field
The invention belongs to the technical field of computer storage technology and data management, and particularly relates to a method for backing up NAS data and browsing and recovering fine file granularity based on an NDMP protocol.
Background
Data backup has been an important research area of computer storage technology and data management technology. It is to copy all or part of the data set from the application host's hard disk or array to other storage media in order to prevent the system from operating erroneously or causing data loss due to system failure.
The traditional data backup mainly adopts a built-in or external tape unit for cold backup. However, this method can only prevent human failures such as misoperation, and the recovery time is long. With the continuous development of the technology, the mass of data is increased, and network backup realized by combining professional data storage management software with corresponding hardware and storage equipment becomes a new favorite. The existing NAS backup mainly adopts the following three technologies:
(1) nfs/cifs technology. The technology utilizes the prior TCP/IP network to transmit data, and can conveniently realize remote protection of data. The NAS data is mounted to the host through nfs/cifs, and NAS file backup is achieved through file backup of the host.
The method has the main defects of occupying host bandwidth and lacking a file consistency time point during backup.
(2) Snapshot backup techniques. Many disk arrays provide SnapShot function, can make full use of SnapShot technology to perform Serverless backup of data, greatly reduces resource occupation of a production system during backup, and can be recovered very quickly. But the backup data is still local to the storage. But if the local storage system is corrupted, the data of the snapshot is also likely to be corrupted. No physical separation of production data and backup data is achieved.
(3) NDMP protocol. NDMP (network data management protocol) is a standard for controlling backup, restore, and data transfer between primary and secondary storage. This technique has the advantage of being inherently thick for a large number of small files.
The NDMP technology is an efficient protection mode aiming at the NAS file server. The data is backed up through the NDMP, and the data does not need to be transmitted through the application program server, so that the load on the application program server is reduced, and the backup speed is improved. NDMP supports mainly four modes of backup: direct connect mode, filler to filler mode, filler to server mode, store to store replication. The biggest difference between the storage-to-storage copy and the other three modes is that the copy is directly used for data transmission between two storages without using a tape. The method has the disadvantages that the backed-up data is stored in a target storage, and is not convenient for unified and autonomous management.
Disclosure of Invention
The invention aims to overcome the defects of the prior art and provides a NAS data backup and file fine-grained browsing recovery method based on an NDMP protocol, wherein during backup, a backup data stream is analyzed into a specific object file; when recovering, the file fine-grained browsing and recovering can be realized, thereby realizing the recovery as required and the transmission as required.
The purpose of the invention can be realized by the following technical scheme:
a NAS data backup method based on an NDMP protocol is characterized in that data in NAS equipment is backed up to a backup all-in-one machine through the NDMP protocol, a DMA module and a first NDMP server are operated in the backup all-in-one machine, a second NDMP server is operated in the NAS equipment, and the backup method comprises the following steps:
A1) establishing control connection between the DMA module and the first NDMP server, control connection between the DMA module and the second NDMP server and data connection between the first NDMP server and the second NDMP server;
A2) the DMA module sends a backup instruction to the second NDMP server and sends a recovery instruction to the first NDMP server;
A3) and the first NDMP server receives the NDMP data stream sent by the second NDMP server, and writes the file object analyzed by the NDMP data stream into the object file system to complete the backup.
The data connection between the first NDMP server and the second NDMP server is realized by the following modes:
A11) the DMA module sends a data interception instruction to the first NDMP server and binds a data interception port;
A12) and the DMA module sends a data connection instruction to the second NDMP server, and establishes a data connection channel through a data interception port of the first NDMP server.
The NDMP data stream is a binary data stream.
The binary data stream comprises a data stream header, a plurality of data content units and a data stream trailer, wherein the data content units comprise file object metadata and file content.
The backup instructions include incremental backup instructions and delta backup instructions.
A NAS data file fine-grained browsing recovery method based on an NDMP protocol is characterized in that data in NAS equipment is backed up to a backup all-in-one machine through the NDMP protocol, a DMA module and a first NDMP server are operated in the backup all-in-one machine, a second NDMP server is operated in the NAS equipment, and the recovery method comprises the following steps:
B1) the DMA module obtains a path of a file needing to be restored;
B2) establishing control connection between the DMA module and the first NDMP server, control connection between the DMA module and the second NDMP server and data connection between the first NDMP server and the second NDMP server;
B3) the DMA module sends a backup instruction to the first NDMP server and sends a recovery instruction to the second NDMP server;
B4) the first NDMP server reads a file to be restored from the object file system according to the path to generate an NDMP data stream;
B5) and the second NDMP server receives the NDMP data stream sent by the first NDMP server, restores the NDMP data stream into a file stored in the NAS, and finishes restoration.
The data connection between the first NDMP server and the second NDMP server is realized by the following modes:
B11) the DMA module sends a data interception instruction to the first NDMP server and binds a data interception port;
B12) and the DMA module sends a data connection instruction to the second NDMP server, and establishes a data connection channel through a data interception port of the first NDMP server.
The NDMP data stream is a binary data stream.
The binary data stream comprises a data stream header, a plurality of data content units and a data stream trailer, wherein the data content units comprise file object metadata and file content.
Compared with the prior art, the invention provides a unified and autonomous management data backup and recovery method for the NAS device compatible with the NDMP technology by using the storage-to-storage copy in the NDMP technology and combining a backup all-in-one machine, and has the following beneficial effects:
1. the backup all-in-one machine is used as a storage medium, and the data on the NAS storage equipment is backed up to the all-in-one machine, so that the unified storage and management of the data are facilitated.
2. The backup all-in-one machine and the NAS storage device adopt a TCP/IP network connection mode, and the backup device of the backup all-in-one machine is not directly bound with the NAS, so that the backup device can be ensured to be simultaneously used as the backup device of a plurality of NAS devices instead of being directly bound on a certain NAS device, and meanwhile, the backup all-in-one machine can also be used as the backup storage device of other backup modes.
3. In the prior art, the data backup of the NAS generally adopts a direct connection mode, the NAS and the backup equipment cannot be far away, and the invention adopts TCP/IP to realize remote backup, namely the backup equipment and the NAS equipment cannot be in one place.
4. Based on the NDMP protocol, the DMA module is realized, the unified operation of backup and recovery of various storages is realized, and a set of backup and recovery programs do not need to be developed independently for each storage.
5. The data stream generated by the NAS storage is analyzed and decomposed into specific file objects, the specific file objects are stored in an object file system of the backup all-in-one machine, and a user can browse and restore file-level fine granularity, so that real on-demand restoration and on-demand transmission are realized.
Drawings
FIG. 1 is a schematic diagram of the connection between the backup all-in-one machine and the NAS device according to the present invention;
FIG. 2 is a flow chart illustrating a backup method according to the present invention;
FIG. 3 is a schematic diagram of the format of a binary data stream used in the present invention;
FIG. 4 is a flow chart of the recovery method of the present invention.
Detailed Description
The invention is described in detail below with reference to the figures and specific embodiments. The present embodiment is implemented on the premise of the technical solution of the present invention, and a detailed implementation manner and a specific operation process are given, but the scope of the present invention is not limited to the following embodiments.
The invention designs and realizes a DMA module in an NDMP backup model and NDMP SERVER service for receiving and sending NDMP data stream in a backup all-in-one machine based on an NDMP protocol. A DMA module and a first NDMP server run in the backup all-in-one machine, and a second NDMP server run in the NAS device.
1. The DMA module, i.e., the network attached data management application, is the control center for the NDMP backup recovery program.
2. NDMP SERVER (NDMP server) in the backup kiosk, NDMPSERVER running in the backup kiosk, respond to the DMA commands for backup and restore. Backup and restore of data with stored NDMPSERVER. Fig. 1 is a deployment diagram of DMA modules, NDMP SERVER (i.e., a first NDMP server) and NDMP SERVER of a NAS storage device (i.e., a second NDMP server).
During backup, real-time analysis of the NDMP binary backup data stream is realized in NDMP SERVER, and NAS files are analyzed and written into an object file system of the all-in-one machine. When browsing is recovered, the files in the object file system are directly displayed, and browsing and recovery of file-level fine granularity can be realized. And recovering the file, namely only reading and transmitting the object file to be recovered, and for the object file not to be recovered, reading and transmitting are not required.
The present invention will be described with reference to a backup and restore operation as an example, which is used to backup and restore data on NAS storage.
As shown in fig. 2, the steps in backup are as follows:
in step S101, a DMA Control center (DMA module) and a Control channel (NDMP Control Connection) of NDMP SERVER in the all-in-one machine are established.
In step S102, a DMA Control center (DMA module) and a stored Control channel (NDMP Control Connection) of NDMP SERVER are established.
In step S103, the DMA sends a data interception command to NDMP SERVER of the all-in-one machine through the control channel, and binds a port to perform data interception; the DMA sends a Data Connection command to the stored NDMP SERVER through the control channel, and establishes a Data Connection channel (NDMP Data Connection) through a Data listening port of the all-in-one machine NDMP SERVER.
In step S104, the DMA sends a backup command to NDMP SERVER stored in the NAS, stores the generated binary backup data, and streams the binary backup data to NDMP SERVER service of the backup all-in-one machine through the previously established data channel.
In step S105, the DMA sends a restore command to NDMP SERVER of the backup all-in-one machine through the control channel, and after receiving the restore command, NDMP SERVER of the backup all-in-one machine receives binary data from the data channel.
In step S106, the NDMP SERVER of the backup all-in-one machine parses the binary data, parses the entire binary data stream into corresponding directory or file objects, and saves the directory or file objects in the object file system. The binary stream data format is shown in fig. 3.
The data stream is divided into three parts of a data stream header, data content and a data stream tail, the data content is composed of a plurality of units, one unit represents a file or a directory, and each unit is divided into metadata and file content. The metadata includes file attributes such as a path, a name, a file system type, and a size in the NAS corresponding to the file. Based on these metadata, the entire binary stream can be partitioned into individual specific file objects.
In step S107, the backup all-in-one machine completes parsing all the binary data streams sent by the NAS storage. And the backup is completed.
Description of the drawings: for NAS incremental backup and delta backup, the NDMP protocol provides instructions for incremental backup and delta backup, sends corresponding instructions to NAS storage, and generates corresponding data streams.
As shown in fig. 4, the steps in recovery are as follows:
in step S201, the user browses the NAS file successfully backed up through the object file system on the backup all-in-one machine, and selects a file path to be restored and transfers the file path to the DMA.
In step S202, the DMA and stored NDMP SERVER and NDMP control channels are established.
In step S203, an NDMP control channel of NDMP SERVER in the DMA and backup all-in-one machine is established.
In step S204, the DMA sends a data interception command to NDMP SERVER of the all-in-one machine through the control channel, and binds a port to perform data interception; the DMA sends a data connection command to the stored NDMP SERVER through the control channel and establishes a data connection through the data listening port of the all-in-one machine NDMP SERVER.
In step S205, the DMA sends a backup command to NDMP SERVER of the backup kiosk.
In step S206, the DMA sends a resume instruction to the NAS storage NDMP SERVER.
In step S207, the NDMP SERVER of the backup all-in-one machine reads the file object to be restored from the object file system according to the file path to be restored, generates a binary data stream, and writes the binary data stream into the previously established data channel.
In step S208, the DMA sends a restore command to the NAS storage NDMP SERVER, NAS to read the binary data stream from the data channel, and restore the binary data stream to the file stored in the NAS storage.
In step S209, the backup all-in-one machine NDMP SERVER finishes sending the binary data stream, and the NAS storage NDMPSERVER finishes receiving the corresponding NDMP binary stream, and the recovery is finished.
Claims (5)
1. A NAS data backup method based on an NDMP protocol is characterized in that data in NAS equipment is backed up to a backup all-in-one machine through the NDMP protocol, the backup all-in-one machine and NAS storage equipment adopt a TCP/IP network connection mode, a DMA module and a first NDMP server are operated in the backup all-in-one machine, a second NDMP server is operated in the NAS equipment, and the backup method comprises the following steps:
A1) establishing control connection between the DMA module and the first NDMP server, control connection between the DMA module and the second NDMP server and data connection between the first NDMP server and the second NDMP server;
A2) the DMA module sends a backup instruction to the second NDMP server and sends a recovery instruction to the first NDMP server;
A3) the first NDMP server receives the NDMP data stream sent by the second NDMP server, and writes the file object analyzed by the NDMP data stream into the object file system to complete backup;
the NDMP data stream is a binary data stream, the binary data stream comprises a data stream header, a plurality of data content units and a data stream tail, and the data content units comprise file object metadata and file content.
2. The NDMP protocol-based NAS data backup method according to claim 1, wherein the data connection between the first NDMP server and the second NDMP server is implemented by:
A11) the DMA module sends a data interception instruction to the first NDMP server and binds a data interception port;
A12) and the DMA module sends a data connection instruction to the second NDMP server, and establishes a data connection channel through a data interception port of the first NDMP server.
3. The NDMP protocol-based NAS data backup method according to claim 1, wherein the backup instruction comprises an incremental backup instruction and a delta backup instruction.
4. A NAS data file fine-grained browsing recovery method based on an NDMP protocol is characterized in that data in NAS equipment is backed up to a backup all-in-one machine through the NDMP protocol, the backup all-in-one machine and NAS storage equipment adopt a TCP/IP network connection mode, a DMA module and a first NDMP server are operated in the backup all-in-one machine, a second NDMP server is operated in the NAS equipment, and the recovery method comprises the following steps:
B1) the DMA module obtains a path of a file needing to be restored;
B2) establishing control connection between the DMA module and the first NDMP server, control connection between the DMA module and the second NDMP server and data connection between the first NDMP server and the second NDMP server;
B3) the DMA module sends a backup instruction to the first NDMP server and sends a recovery instruction to the second NDMP server;
B4) the first NDMP server reads a file to be restored from the object file system according to the path to generate an NDMP data stream;
B5) the second NDMP server receives the NDMP data stream sent by the first NDMP server, and restores the NDMP data stream into a file stored in the NAS to complete restoration;
the NDMP data stream is a binary data stream, the binary data stream comprises a data stream header, a plurality of data content units and a data stream tail, and the data content units comprise file object metadata and file content.
5. The NDMP protocol-based NAS data file fine-grained browsing recovery method according to claim 4, wherein the data connection between the first NDMP server and the second NDMP server is realized by:
B11) the DMA module sends a data interception instruction to the first NDMP server and binds a data interception port;
B12) and the DMA module sends a data connection instruction to the second NDMP server, and establishes a data connection channel through a data interception port of the first NDMP server.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610860017.3A CN106484566B (en) | 2016-09-28 | 2016-09-28 | NAS data backup and file fine-grained browsing recovery method based on NDMP protocol |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610860017.3A CN106484566B (en) | 2016-09-28 | 2016-09-28 | NAS data backup and file fine-grained browsing recovery method based on NDMP protocol |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106484566A CN106484566A (en) | 2017-03-08 |
| CN106484566B true CN106484566B (en) | 2020-06-26 |
Family
ID=58268206
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610860017.3A Active CN106484566B (en) | 2016-09-28 | 2016-09-28 | NAS data backup and file fine-grained browsing recovery method based on NDMP protocol |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106484566B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110535946A (en) * | 2019-08-30 | 2019-12-03 | 北京浪潮数据技术有限公司 | A kind of distributed memory system and data transmission method, device, equipment, medium |
| CN114546980B (en) * | 2022-04-25 | 2022-07-08 | 成都云祺科技有限公司 | Backup method, system and storage medium of NAS file system |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102024016A (en) * | 2010-11-04 | 2011-04-20 | 天津曙光计算机产业有限公司 | Rapid data restoration method for distributed file system (DFS) |
| CN103858109A (en) * | 2011-10-28 | 2014-06-11 | 株式会社日立制作所 | Information processing system and file restoration method using same |
| CN104461776A (en) * | 2014-11-26 | 2015-03-25 | 上海爱数软件有限公司 | Application disaster tolerance method based on CDP and iSCSI virtual disk technology |
| CN104883265A (en) * | 2014-02-27 | 2015-09-02 | 中兴通讯股份有限公司 | Backup file data retransmission method, apparatus and system |
| CN105446830A (en) * | 2015-11-09 | 2016-03-30 | 上海爱数信息技术股份有限公司 | Data backup method based on NDMP (Network Data Management Protocol) disk-to-disk replication technology |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| AU2003282361A1 (en) * | 2002-11-20 | 2004-06-15 | Filesx Ltd. | Fast backup storage and fast recovery of data (fbsrd) |
| WO2014139069A1 (en) * | 2013-03-11 | 2014-09-18 | 华为技术有限公司 | Method and apparatus for repairing video file |
-
2016
- 2016-09-28 CN CN201610860017.3A patent/CN106484566B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102024016A (en) * | 2010-11-04 | 2011-04-20 | 天津曙光计算机产业有限公司 | Rapid data restoration method for distributed file system (DFS) |
| CN103858109A (en) * | 2011-10-28 | 2014-06-11 | 株式会社日立制作所 | Information processing system and file restoration method using same |
| CN104883265A (en) * | 2014-02-27 | 2015-09-02 | 中兴通讯股份有限公司 | Backup file data retransmission method, apparatus and system |
| CN104461776A (en) * | 2014-11-26 | 2015-03-25 | 上海爱数软件有限公司 | Application disaster tolerance method based on CDP and iSCSI virtual disk technology |
| CN105446830A (en) * | 2015-11-09 | 2016-03-30 | 上海爱数信息技术股份有限公司 | Data backup method based on NDMP (Network Data Management Protocol) disk-to-disk replication technology |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106484566A (en) | 2017-03-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11429305B2 (en) | Performing backup operations using replicas | |
| US9389800B1 (en) | Synthesizing virtual machine disk backups | |
| US9501367B2 (en) | Systems and methods for minimizing network bandwidth for replication/back up | |
| US8370302B2 (en) | Method and apparatus for block based volume backup | |
| US9483359B2 (en) | Systems and methods for on-line backup and disaster recovery with local copy | |
| US9679039B1 (en) | Continuous protection of data and storage management configuration | |
| US10496487B1 (en) | Storing snapshot changes with snapshots | |
| CN101221521B (en) | Backup management and file access service system and method, exchange device | |
| US9152642B2 (en) | Systems and methods for on-demand data storage | |
| US7958372B1 (en) | Method and apparatus to convert a logical unit from a first encryption state to a second encryption state using a journal in a continuous data protection environment | |
| US9448893B1 (en) | Asynchronous replication correctness validation | |
| US20160042046A1 (en) | Distributed data store | |
| US9558075B2 (en) | Synthetic full backup generation | |
| US20140181016A1 (en) | Asynchronous replication correctness validation | |
| US20140181021A1 (en) | Back up using locally distributed change detection | |
| CN101808127B (en) | Data backup method, system and server | |
| US20090113145A1 (en) | Data transfer | |
| CN110109778A (en) | A kind of a large amount of small data file backup methods and restoration methods | |
| WO2020063600A1 (en) | Data disaster recovery method and site | |
| US20070055835A1 (en) | Incremental replication using snapshots | |
| US20070022319A1 (en) | Maintaining and using information on updates to a data group after a logical copy is made of the data group | |
| JP5868986B2 (en) | Recovery by item | |
| US20220398240A1 (en) | Granular replication of volume subsets | |
| CN103593264A (en) | System and method for remote wide area network disaster recovery backup | |
| CN105589887A (en) | Data processing method for distributed file system and distributed file system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |