New! View global litigation for patent families

CN102722536B - Method and device for realizing snapshot based on distributed file system - Google Patents

Method and device for realizing snapshot based on distributed file system Download PDF

Info

Publication number
CN102722536B
CN102722536B CN 201210159290 CN201210159290A CN102722536B CN 102722536 B CN102722536 B CN 102722536B CN 201210159290 CN201210159290 CN 201210159290 CN 201210159290 A CN201210159290 A CN 201210159290A CN 102722536 B CN102722536 B CN 102722536B
Authority
CN
Grant status
Grant
Patent type
Prior art keywords
method
device
realizing
snapshot
system
Prior art date
Application number
CN 201210159290
Other languages
Chinese (zh)
Other versions
CN102722536A (en )
Inventor
欧阳伟
Original Assignee
中兴通讯股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Grant date

Links

Abstract

本发明公开一种基于分布式文件系统的快照实现方法及装置,该方法包括复制数据块的元数据创建快照文件,并保存;将所述快照文件和所述数据块的元数据与所述数据块进行关联。 Snapshot method and apparatus of the present invention is implemented based on distributed file system is disclosed, the method comprising the replication metadata file data blocks to create a snapshot and save; metadata file and the snapshot of the data block with the data block association. 本发明通过以上技术方案,解决现有的快照技术无法满足分布式文件系统的需求这一技术问题。 Through the above technical solution of the present invention to solve the existing snapshot technology can not meet the needs of distributed file system of the technical problems.

Description

基于分布式文件系统的快照实现方法及装置 Method and apparatus for realization of snapshot-based Distributed File System

技术领域 FIELD

[0001] 本发明涉及存储领域,尤其涉及一种基于分布式文件系统的快照实现方法及装置。 [0001] The present invention relates to the field of memory, particularly to a distributed file system snapshot-based method and apparatus to achieve.

背景技术 Background technique

[0002] 随着存储应用需求的提高,用户需要在线方式进行数据保护,快照就是在线存储设备防范数据丢失的有效方法之一。 [0002] With the increased storage requirements of the application, users need online data protection, snapshots is an effective way to prevent online storage of data loss. 快照技术最常见的用途是快速、简易恢复意外擦除或损坏的数据,快照技术可以实现两项功能:整合资源和迀移数据。 The most common use snapshot technology is fast, easy to recover data accidentally erased or damaged, snapshot technology can achieve two functions: integration of resources and Gan shift data. 有了快照技术,存储系统中数据的可靠性得到极高提升,并且为软件测试、回归测试等提供有力的数据保证。 With snapshot technology, the reliability of the data storage system has been highly improved, and as software testing, regression testing and other data provide strong assurance. 在现有的存储系统中,快照技术可以在存储系统中的多个层面实现:有基于文件系统的快照、有基于逻辑卷设备的快照、有基于设备驱动的快照以及基于虚拟化的快照等。 In a conventional storage systems, snapshots can be stored in multiple levels of system implementation: there is a snapshot-based file system, there are based on a snapshot of a logical volume device with a device driver snapshots and virtualization-based snapshot and so on.

[0003] 然而,现有成熟的快照技术,不管是全拷贝的快照,还是增量备份快照,最终都是对硬件存储设备如磁盘、磁阵等上存储的实际数据进行快照备份。 [0003] However, existing mature snapshots, the snapshot copy Whether full or incremental backup snapshots are ultimately the actual data stored on hardware devices such as disk storage, and other magnetic array snapshot backup. 在分布式文件系统中,除了存储设备上的实际数据外,还有大量的元数据存储在单独的元数据服务器中,不仅要对实际数据进行快照备份,同时还要对元数据的更新修改进行备份快照,因此现有的快照技术无法满足分布式文件系统的需求。 In a distributed file system, in addition to the actual data on the storage device, there are a lot of metadata stored in a separate metadata server, not only snapshot backups of actual data, but also to update the metadata modifications made backup snapshot, so existing snapshot technology can not meet the needs of distributed file system.

发明内容 SUMMARY

[0004] 本发明提供一种基于分布式文件系统的快照实现方法及装置,解决现有的快照技术无法满足分布式文件系统的需求这一技术问题。 [0004] The present invention provides a distributed file system snapshot implemented method and apparatus to solve the conventional snapshot technique is based on the distributed file system can not meet the needs of the technical problem.

[0005] 为解决上述技术问题,本发明采取以下技术方案。 [0005] To solve the above problems, the present invention adopts the following technical solutions.

[0006] 一种基于分布式文件系统的快照实现方法,包括: [0006] an implementation method based on a distributed file system snapshots, including:

[0007] 复制数据块的元数据创建快照文件,并保存; [0007] metadata replication of data blocks to create a snapshot file and save it;

[0008] 将所述快照文件和所述数据块的元数据与所述数据块进行关联。 [0008] The metadata file and the snapshot of the data block is associated with the data block.

[0009] 在本发明一实施例中,该方法还包括:在需要对数据块进行更新操作时,判断所述数据块是否同时与所述快照文件、所述数据块的元数据关联,若是,对所述数据块进行复制,得到新建数据块;对所述数据块和所述新建数据块中的任一者进行更新操作,另一者与所述快照文件关联后保存。 [0009] In an embodiment of the present invention, the method further comprising: when a block of data needs to update, determining whether the data block while the snapshot file, the metadata associated with the data block, and if so, copying the data block to obtain new data block; the data block and the new data block in any of the update operation, the other one is associated with the snapshot file saved.

[0010] 在本发明一实施例中,该方法还包括:若否,则直接对所述数据块进行更新操作。 [0010] In an embodiment of the present invention, the method further comprising: if not, the data block directly update operation. [0011] 在本发明一实施例中,该方法还包括:建立与数据块对应的引用计数器;对所述数据块当前被所述快照文件、所述数据块的元数据关联的次数进行计数。 [0011] In an embodiment of the present invention, the method further comprising: establishing a reference corresponding to the data block counter; the number of times the data block of the current element is the snapshot file, the data block associated with the data is counted.

[0012] 在本发明一实施例中,判断所述数据块是否同时与所述快照文件、所述数据块的元数据关联的方法包括:判断所述引用计数器的计数是否大于1,若是,则判断为所述数据块同时与所述快照文件、所述数据块的元数据关联。 [0012] In an embodiment of the present invention, it is determined whether the data block while the snapshot file, the metadata associated with the data block comprising: determining whether the count of the reference counter is greater than 1, and if yes, the data block is determined simultaneously with the snapshot file, data blocks associated with the data element.

[0013] 一种基于分布式文件系统的快照实现装置,包括快照文件创建模块和关联关系建立模块,其中, [0013] based on a snapshot of distributed file system implementation apparatus including a snapshot file creation module and relationships established module,

[0014] 所述快照文件创建模块用于复制数据块的元数据创建快照文件,并保存; [0014] The metadata snapshot file creation means for creating a snapshot copy of the file data blocks, and save;

[0015] 所述关联关系建立模块用于将所述快照文件和所述数据块的元数据与所述数据块进彳T关联。 [0015] The means for establishing the association relationship metadata file and the snapshot of the data block and the data block into the associated T left foot.

[0016] 在本发明一实施例中,该装置还包括判断模块、数据块更新模块和数据块新建模块,其中,所述判断模块在所述数据块更新模块需要对数据块进行更新操作时,判断所述数据块是否同时与所述快照文件、所述数据块的元数据关联;所述数据块新建模块用于在所述判断模块的判断结果为是时,对所述数据块进行复制,得到新建数据块;所述数据块更新模块用于对所述数据块和所述新建数据块中的任一者进行更新操作,或者在所述判断模块判断结果为否时,直接对所述数据块进行更新操作;所述关联关系建立模块用于将所述数据块和所述新建数据块中的另一者与所述快照文件关联后保存。 [0016] In an embodiment of the present invention, the apparatus further comprises a determining module, a data block and a data updating module block new module, wherein, the determining module in the data block needs updating module updates the data block operation, determining whether the data block while the snapshot file, the data element associated with the data block; the data block for the new module is the determination result of the determining module is yes, copy the data block, to obtain new data block; means for updating the data block of the data block and the new data block in any of the update operation, or when the determining module determines that the result is NO, the data is directly update blocks; means for establishing the association will be saved after the other of the new data block and the data block associated with the snapshot file.

[0017] 在本发明一实施例中,该装置还包括与数据块对应的引用计数模块,所述引用计数模块用于对所述数据块当前被所述快照文件、所述数据块的元数据关联的次数进行计数。 [0017] In an embodiment of the present invention, the apparatus further includes reference data corresponding to the block counting module, a reference to the metadata means for counting the data block currently the snapshot file, the data block counting the number of times associated.

[0018] 在本发明一实施例中,判断模块用于判断所述引用计数器的计数是否大于1,若是,则判断为所述数据块同时与所述快照文件、所述数据块的元数据关联。 [0018] In an embodiment of the present invention, determination means for determining whether the count of the reference counter is greater than 1, and if yes, determines that the data block, metadata while the data block associated with the snapshot file .

[0019] 本发明提供一种基于分布式文件系统的快照实现方法及装置,本发明提供的快照实现方法相对于现有成熟的快照实现方法,新建的快照文件只是复制数据块的元数据,将得到的快照文件和数据块的元数据与该相同数据块进行关联,能够满足分布式文件系统的需求,还减少了对数据块的重复复制的过程,降低了系统资源和IO资源的消耗,改善了存储系统在引入快照机制后系统性能普遍变得较低的问题。 [0019] The present invention provides a distributed file system snapshot implemented method and apparatus, the present invention provides a method to achieve a snapshot of the prior snapshot mature implementation method, simply copying the new snapshot file metadata data blocks, the metadata snapshot file and data blocks obtained with the same data block is associated, to meet the needs of distributed file system, the process also reduces duplicate copy of the data block, reducing consumption of system resources and IO resources, improve after the introduction of the storage system snapshot mechanism system performance generally become less of a problem.

附图说明 BRIEF DESCRIPTION

[0020] 图1为本发明一实施例提供的基于分布式文件系统的快照实现方法的流程图; [0020] FIG. 1 is a flowchart of a method implemented based on distributed file system snapshot according to an embodiment of the present invention;

[0021] 图2为本发明一实施例提供的分布式文件系统的示意图。 [0021] FIG. 2 is a schematic view of a distributed file system according to an embodiment of the present invention.

具体实施方式 detailed description

[0022] 本发明的总体构思是,在对源文件做快照时,只是复制数据块的元数据,得到快照文件,源文件的各个数据块(chunk)的内容暂不复制,而是等到需要对数据块进行更新操作(如修改、删除、增加内容等)时,再复制数据块的内容。 [0022] The general concept of the present invention is that, in making the snapshot of the source file, only data blocks replication metadata, to give a snapshot of the file, the contents of each data block of a source file (the chunk) is not yet copied, but wait until the need for when the update data blocks (e.g., edit, delete, add content, etc.), and then copy the contents of the data block. 本发明的快照文件由复制数据块的元数据得来。 Snapshot file of the present invention obtained by the metadata to copy the data blocks.

[0023] 下面通过具体实施方式结合附图对本发明作进一步详细说明。 [0023] The following figures present invention will be further described in detail by specific embodiments in combination.

[0024] 源文件往往由多个数据块存储,假设源文件由4个数据块存储,分别计为数据块A、B、C和D。 [0024] The source file is often stored in a plurality of data blocks, assume that the source file data from four memory blocks, respectively, in terms of data blocks A, B, C, and D. 快照前,该源文件的元信息包括pA、pB、pC、pD,如表1所示,快照前,快照文件为空。 Before the snapshot, the meta information comprises a source file pA, pB, pC, pD, as shown in Table 1, before the snapshot, the snapshot file is empty.

[0025] 表1 [0025] TABLE 1

[0026] [0026]

Figure CN102722536BD00051

[0027] 由于,快照后,快照文件和数据块的元数据可能引用同一数据块,因此,该实施例引入引用计数模块,可以在分布式文件系统中的元数据服务器上,针对一源文件的各数据块,分别设置对应的引用计数模块,各引用计数模块为相应数据块当前被快照文件、数据块的元数据关联的次数进行计数;或者也可以针对一源文件共设置一个引用计数模块,该引用计数模块分别为该源文件的各个数据块当前被快照文件、数据块的元数据关联的次数进行计数。 [0027] Since, after the snapshot, the snapshot file metadata and data blocks may reference the same data block, therefore, this embodiment introduces a reference count module may be a metadata server in a distributed file system for a source file each data block, respectively provided reference count corresponding module, each of the reference count module corresponding data block currently snapshot files, the number of data associated metadata block counting; or also were set a reference count module for a source file, the reference count module, respectively, the number of metadata associated with the data block for counting each data block of the source file is a snapshot of the current file. 未进行快照前,因各数据块仅被其元数据引用,所以各数据块的引用计数均为1,如表2所示。 No prior snapshot, because each data block is its metadata reference only, the reference to the block count of each data are 1, as shown in Table 2.

[0028] 表2 [0028] TABLE 2

[0029] [0029]

Figure CN102722536BD00061

[0030] 接收到用户的快照请求后,可以针对一源文件中的各个数据块,分别制作一个快照文件,也可以针对一个源文件的所有数据块共制作一个快照文件,该快照文件中包括各个数据块的快照文件,以针对一个源文件的所有数据块共制作一个快照文件为例。 [0030] After receiving the snapshot request from a user may be directed to each data block a source file, each making a snapshot file, may be co-create a snapshot file for all data blocks of a source file, the snapshot file comprises various snapshot file data blocks to all data blocks for a total of source file to make a snapshot file as an example. 如图1 所示,按照如下步骤对该源文件进行快照: 1, the following steps take snapshots of the source file:

[0031] S101、复制各数据块的元数据,得到快照文件1,并将快照文件1保存在本地存储媒介。 [0031] S101, the replication metadata of each data block, to give a snapshot of the file, and the snapshot files are stored in a local storage medium. 快照文件1具有与源文件的元数据相同的信息,由表1可知,当前源文件的元数据包括pA、pB、pC、pD,因此,快照文件1同样包括pA、pB、pC、pD,如表3所示。 Snapshot file 1 has a metadata source file of the same information, can be seen from Table 1, the current metadata source file comprising pA, pB, pC, pD, therefore, the snapshot file 1 also comprises pA, pB, pC, pD, such as As shown in table 3.

[0032] 表3 [0032] TABLE 3

[0033] [0033]

Figure CN102722536BD00062

[0034] S102、将快照文件1和源文件的元数据与相同数据块进行关联,各数据块同时被其源文件的元数据和快照文件1引用,因此,各数据块的引用计数由1变为2。 [0034] S102, the metadata snapshot file 1 and source file are associated with the same data blocks, each data block is simultaneously a source file metadata and snapshot file a reference, and therefore, each data block reference count is changed from 1 2. 如表4所示。 As shown in Table 4.

Figure CN102722536BD00063

[0035] 表4 [0035] TABLE 4

[0036] [0036]

[0037] 假设用户在快照操作后需要对数据块A进行更新操作,如需要对数据块A进行写入,则还包括如下主要步骤: [0037] Assume that the user needs to operate the block A is updated after snapshot operation, such as the need to write the data block A, the main steps further comprising:

[0038] S103、通过分布式文件系统中的文件访问客户端接收用户的更新操作请求,判断数据块A对应的引用计数是否大于1,若是,则说明数据块A同时与快照文件、源文件的元数据关联,进入步骤S104,否则,说明数据块A没有同时与快照文件、源文件的元数据关联,正常情况下,要么数据块A仅与源文件的元数据关联,说明数据块A还未进行过快照或者进行过快照后,快照文件1已删除,要么数据块A仅与快照文件1关联,说明数据块A是仅用于备份保存的数据块,进入步骤108。 [0038] S103, the client receives a user access to a distributed file system file update operation request, data block A is determined whether the count is greater than the corresponding reference 1, if, while the data of the block A and the snapshot file, the source file metadata associated proceeds step S104, otherwise, there is no data block a while the snapshot file metadata associated with the source file, under normal circumstances, only the block a or the metadata associated with the source file, the block a has not been described after been subjected to a snapshot or snapshots, the snapshot file 1 is deleted, or is associated with only a snapshot file data block a, the block a is the only explanation for backing up saved data blocks, proceeds to step 108.

[0039] S104、对数据块A进行复制,得到新建数据块Al。 [0039] S104, the data block A is copied to obtain new data block Al.

[0040] 具体的,可以选择一个新的数据块句柄A1,要求分布式文件系统中每个拥有数据块A的文件访问服务器都创建一个叫Al的新数据块,新建数据块Al的数据内容从文件访问服务器本地的数据块A复制过来。 [0040] In particular, you can select a new data block handle A1, it requires a distributed file system, each file has a data block A to access the server creates a new block called Al, the data content of the new data block from Al local file access server data blocks a copied.

[0041] S105、对数据块A和新建数据块Al中的任一者进行更新操作,另一者与快照文件关联后备份保存。 [0041] S105, and the new data block A data block Al of any one update operation, the snapshot file associated with the other backup storage.

[0042] 假如选择对新建数据块Al进行更新操作,那么保持数据块A与快照文件1的关联关系后保存在本地存储媒介,删除源文件的元数据中数据块A的元数据,同时加入数据块Al的元数据,那么当前源文件的元数据更新为:pAl、pB、pC、pD,由于数据块Al并未进行快照,因此,快照文件1依然是pA、pB、pC、pD,如表5所示。 [0042] If selection of a new data block Al update operation, the holding block A snapshot file relationships 1 stored in meta data in the local storage medium, deleting the source file's metadata of the data block A, while adding data Al metadata block, then the current source file metadata update: pAl, pB, pC, pD, since Al is not a snapshot of the data block, therefore, still a snapshot file pA, pB, pC, pD, as shown in table 5 FIG.

[0043] 表5 [0043] TABLE 5

[0044] [0044]

Figure CN102722536BD00071

[0045] 删除源文件的元数据中数据块A的元数据的同时,相应地删除数据块A的元数据与数据块A的关联关系,因此,数据块A的引用计数减1。 Association [0045] The metadata file while deleting the source data block A metadata, deleting the corresponding metadata block A data block A is, therefore, reference data block A count by 1. 加入数据块Al的元数据的同时, 相应地加入源文件的元数据对数据块Al的引用计数,由于其仅被更新后的源文件的元数据引用,因此其引用计数为1。 Al is added simultaneously block of metadata, the metadata added corresponding source file reference count data blocks Al, which are cited merely because a metadata source file after the update, so the reference count of 1. 如表6所示。 As shown in Table 6.

Figure CN102722536BD00072

[0046] 表6 [0046] TABLE 6

[0047] [0047]

[0048] S106、元数据服务器把数据块Al的元信息回应给文件访问客户端; [0048] S106, the metadata server data blocks Al meta information file access response to the client;

[0049] S107、文件访问客户端根据用户的写入指令向数据块Al正常写入数据。 [0049] S107, the file access client writing data to a data block in accordance with the user Al normal write command.

[0050] S108、根据用户的写入指令直接对数据块A进行写入,不需要对数据块A进行复制。 [0050] S108, the data block A is written according to a user's direct write command, the data need not be copied block A.

[0051] 假设用户需要对源文件进行追加,如追加数据块E,则还包括如下主要步骤: [0051] Assume that the user needs to be added to source files, such as additional data block E, the main steps further comprising:

[0052] S109、元数据服务器把数据块E的元信息加入当前源文件的元数据,参照表5,当前源文件的元数据包括:pAl、pB、pC、pD,因此,加入数据块E的元信息之后,源文件的元数据更新为:pAl、pB、pC、pD、pE,但是由于数据块pAl、pE并未进行快照,因此,快照文件l依然是pA、pB、pC、pD,如表7所示。 [0052] S109, the metadata server data block E of meta information added to the current metadata source file, referring to Table 5, the current metadata source file includes: pAl, pB, pC, pD, therefore, added to the data block E after the meta information, meta data source file is updated to: pAl, pB, pC, pD, pE, but because the data block pAl, pE not a snapshot, therefore, the snapshot file l still pA, pB, pC, pD, such as As shown in table 7.

[0053] 表7 [0053] TABLE 7

[0054] [0054]

Figure CN102722536BD00081

[0055] S110、加入数据块E的元信息的同时,相应地加入源文件的元数据对数据块E的引用计数,由于其仅被更新后的源文件的元数据引用,因此其引用计数为1。 [0055] S110, while adding the meta-information data block E, and correspondingly join the source file metadata reference count of the data block E, since the metadata of its source files only after being updated references, thus its reference count is 1. 如表8所示。 As shown in Table 8.

[0056] 表8 [0056] TABLE 8

[0057] [0057]

Figure CN102722536BD00082

[0058] 假设用户需要对源文件中的某个数据块进行删除,如删除数据块D,则还包括如下主要步骤: [0058] Assume that the user needs a data block in the source file for deletion, such as deleting the data block D, the main steps further comprising:

[0059] Slll、元数据服务器从当前源文件的元数据中删除数据块D的元信息,参照表7, 当前源文件的元数据包括:pAl、pB、pC、pD、pE,因此,删除数据块D的元信息之后,源文件的元数据更新为:pAl、pB、pC、pE,但是数据块D的快照文件未被删除,因此,快照文件1依然是pA、pB、pC、pD,如表9所示。 [0059] Slll, the metadata server from metadata of the current source file meta-information data block D, reference to Table 7, the current metadata source file includes: pAl, pB, pC, pD, pE, and therefore, deletion of data after the meta-information block D, metadata source file is updated to: pAl, pB, pC, pE, but the data block D snapshot files are not deleted, therefore, remains a snapshot file 1 pA, pB, pC, pD, such as As shown in table 9.

[0060] 表9 [0060] Table 9

[0061] [0061]

Figure CN102722536BD00083

[0062] S112、删除数据块D的元信息的同时,相应地删除源文件的元数据对数据块D的引用计数,因此数据块D的引用计数减1。 [0062] S112, the meta-information deleting data block D at the same time, deleting the corresponding source file metadata reference count of the data block D, and therefore reference data block D count by 1. 如表10所示。 As shown in Table 10.

[0063] 表10 [0063] TABLE 10

[0064] [0064]

Figure CN102722536BD00091

[0065] 假设用户需要对当前的源文件再作快照,接收到用户的快照请求后,还包括如下主要步骤: [0065] Assume that the user needs to then make the current snapshot of the source file, the user receives the snapshot request, further comprising the main steps:

[0066] S113、复制当前源文件中各数据块的元数据,得到快照文件2,并将该快照文件2 保存在本地存储媒介。 [0066] S113, the current source file replication metadata of each data block, to give a snapshot of the file 2, 2 and the snapshot files in the local storage medium. 快照文件2具有与当前源文件的元数据相同的信息,参考表9可知, 当前源文件的元数据包括pAl、pB、pC、pE,因此,快照文件2同样包括pAl、pB、pC、pE,而快照文件I依然包括pA、pB、pC、pD。 Snapshot file 2 having metadata current source file of the same information, with reference to Table 9, the current metadata source file includes pAl, pB, pC, pE, therefore, the snapshot file 2 also comprises pAl, pB, pC, pE, The snapshot file I still include pA, pB, pC, pD. 如表11所示。 As shown in Table 11.

[0067] 表11 [0067] Table 11

[0068] [0068]

Figure CN102722536BD00092

[0069] S114、将快照文件2和源文件的元数据与相同数据块进行关联,因此,对于数据块A1、B、C、E而言,其引用计数均加1,其他数据块引用计数不变。 [0069] S114, the metadata snapshot file 2 and source file are associated with the same data block, and therefore, the data blocks A1, B, C, E, its reference count is incremented by one each, other reference data block count is not change. 表10更新为表12,如表12 所示。 Table 10 Table 12 updates, as shown in Table 12.

Figure CN102722536BD00093

[0070] 表12 [0070] Table 12

[0071] [0071]

[0072] 假设用户需要删除快照文件1,接收到用户的删除请求后,还包括如下主要步骤: [0072] Assume that the user needs to delete a snapshot file, after receiving the deletion request of the user, further comprising the following main steps:

[0073] S115、如同删除普通文件一样,删除快照文件1,对于快照文件1所涉及的数据块A、B、C、D而言,失去了一个快照镜像,表11更新为表13,如表13所示。 [0073] S115, as ordinary file delete, delete a snapshot file, the data file snapshot block A according to 1, B, C, D, the loss of a snapshot image, the table 11 is updated to Table 13, Table 13 FIG.

[0074] 表13 [0074] TABLE 13

[0075] [0075]

Figure CN102722536BD00101

[0076] S116、删除快照文件1时,相应地更新数据块的引用计数,对于快照文件1所涉及的数据块A、B、C、D而言,其计数均减1,其他数据块引用计数不变。 [0076] S116, 1, the corresponding block update snapshot file delete reference count for the block A snapshot file related to 1, B, C, D, its average count minus 1, the reference count of other data blocks constant.

[0077] 表12更新为表14,如表14所示。 [0077] Table 12 updates table 14, as shown in Table 14.

[0078] 表14 [0078] TABLE 14

[0079] [0079]

Figure CN102722536BD00102

[0080] 图2为本发明一实施例提供的分布式文件系统,包括元数据服务器11、文件访问客户端12、文件访问服务器13和存储媒介14。 [0080] FIG. 2 is a distributed file system according to an embodiment of the invention, including the metadata server 11, the file access client 12, the file access server 13 and the storage medium 14.

[0081] 元数据服务器11负责管理分布式文件系统内所有文件的文件名、数据块等元数据信息;向文件访问客户端12提供元数据写入和查询等操作;在接收到来自用户的快照请求后,复制数据块的元数据创建快照文件,并将快照文件和数据块的元数据与该相同数据块进行关联;还用于为数据块被快照文件、数据块的元数据关联的次数进行计数。 Metadata within [0081] The metadata server 11 manages the distributed file system file names of all files, and other information data blocks; file access client terminal 12 to provide metadata query and write operations; snapshot receiving from a user after the request, the duplicated data block metadata file to create a snapshot, and the snapshot file metadata and data blocks is associated with the same data block; is further configured to perform snapshot files, the number of metadata associated with the data block is a data block count.

[0082] 文件访问客户端12负责为分布式文件系统面向的应用程序提供类似于标准文件系统的接口调用服务,同时充当发起快照流程的客户端,接收来自用户的快照请求,然后与元数据服务器11、文件访问服务器13交互,完成快照流程。 [0082] 12 file access client is responsible for providing interface call service similar to the standard file system for distributed file system-oriented applications, while serving as a snapshot of the process initiated by the client to receive a snapshot request from the user, then the metadata server 11, 13 interactive file access server, complete snapshot process.

[0083] 文件访问服务器13负责与存储媒介14进行交互,进行数据块的读写操作;响应文件访问客户端12的数据读写请求,从存储媒介14上读取数据并返回给文件访问客户端12 ; 从文件访问客户端12读取数据并写入存储媒介14 ;在接收到来自用户对数据块的更新操作请求后,先判断该数据块是否同时与快照文件、数据块的元数据关联,若是,则不能直接对该数据块进行更新操作,而是需要对该数据块进行复制,得到新建数据块,对原数据块和新建数据块中的任一者进行更新操作,另一者与快照文件关联后保存,这样便能够将源文件的元数据和快照文件所指向的数据块区分开来,与快照文件关联的数据块被备份保存。 [0083] The file access server 13 is responsible for interacting with the storage medium 14, reads and writes data block; response file 12 to read and write data access client request, read data from the storage medium 14 and returns to the file access client 12; accessing a file from the client terminal 12 reads and writes the data storage medium 14; after receiving the update operation request blocks of data from a user, to judge whether the metadata block associated with the snapshot file while the data block, if yes, the data block can not directly update operation, but requires replication of the data block to obtain new data block, the original data block and the new data block in any one of update operation, and the other snapshots save the file association, so that it is possible to source files and metadata snapshot file pointed to distinguish the data blocks, the data blocks associated with the snapshot file is backed up. 如果判断结果为该数据块没有同时与快照文件、数据块的元数据关联,正常情况下,要么该数据块仅与数据块的元数据关联,则说明该数据块还未进行过快照或者进行过快照后,快照文件已删除,要么该数据块仅与快照文件关联,则说明该数据块是被备份保存的数据块, 那么,可以根据用户的更新操作请求直接对该数据块进行更新操作,不需要对该数据块进行复制。 If the judgment result at the same time that the data block is not the snapshot file, meta data associated with the data block, under normal circumstances, only the data block or data blocks associated with the metadata, it indicates that the data block has not been carried out or snapshots after the snapshot, the snapshot file has been deleted, or the data block is associated with only a snapshot file, it indicates that the data block is stored in the backup data block, then the operation may be updated according to the update operation request directly to the user's data block, not copying the data block is required.

[0084] 存储媒介14可以为普通的IDE磁盘或SATA磁盘,用于存放数据块。 [0084] The storage medium 14 may be a conventional IDE disks or SATA disks for storing data blocks.

[0085] 具体的,为了实现快照功能,元数据服务器11上可设置快照文件创建模块111、关联关系建立模块112,并针对源文件的各数据块,分别设置对应的引用计数模块113,各引用计数模块113分别为相应数据块被快照文件、数据块的元数据关联的次数进行计数。 [0085] Specifically, in order to achieve the snapshot function, a metadata server may be provided on 11 the snapshot file creation module 111, an association relationship creation module 112, and for each data block of a source file, are provided reference count module 113 corresponding to each reference counting module 113 respectively corresponding data block is the snapshot file, the number of metadata associated with the data block is counted. 当然,在另一实施例中,也可以针对源文件的各数据块,共设置一个对应的引用计数模块,该引用计数模块分别为该源文件的各个数据块被快照文件、数据块的元数据关联的次数进行计数。 Of course, in another embodiment, for each data block may be the source file corresponding to a set of common reference count module, the reference count module respectively for each data block of the source file is a snapshot file, metadata of the data block counting the number of times associated. 文件访问服务器13上可设置判断模块131、数据块更新模块132和数据块新建模块133,各t吴块的功能如下: File access server determination module 13 may be provided 131, a module 132 to update the data blocks and new data blocks module 133, each of t Wu functional blocks as follows:

[0086] 在接收到来自用户的快照请求后,快照文件创建模块11可以针对源文件中的每个数据块,分别制作一个快照文件,也可以针对一个源文件的所有数据块共制作一个快照文件,该快照文件中包括各个数据块的快照文件。 [0086] Upon receiving the request from the user of the snapshot, the snapshot module 11 may create a file for each data block in the source file, create a snapshot file, respectively, may be co-production of a snapshot file for all data blocks of a source file the snapshot file, each data block includes a snapshot file. 快照文件创建模块11具体用于复制数据块的元数据,以作为快照文件,源文件的各个数据块的内容暂不复制,将快照文件保存在存储媒介14上。 Snapshot file is created metadata module 11 particularly for copying data blocks, as a snapshot file, the content of each source file data block is not yet copied to the snapshot files saved on the storage medium 14. 此外,快照文件创建模块11创建快照文件的过程中,为了保证快照文件的准确性,可以禁止或延迟一切对该源文件的更新操作。 Furthermore, the process snapshot file creation module 11 to create a snapshot file, in order to ensure the accuracy of the snapshot file can be disabled or delayed update all the source files.

[0087] 快照文件创建模块11创建快照文件之后,关联关系建立模块112用于将其创建的快照文件和数据块的元数据与该相同数据块进行关联。 After [0087] the snapshot file creation module 11 creates a snapshot file, association relationship creation module 112 of the metadata file and the snapshot data is created which blocks associated with the same data block.

[0088] 进行快照前,各数据块仅被其源文件的元数据引用,因此,此时对应引用计数模块113的引用计数均为1 ;在快照文件创建模块11每为该数据块创建一个快照文件,且关联关系建立模块112将该快照文件与该数据块关联之后,其计数加1,在每一快照文件被删除并解除了与该数据块的关联关系,或者,该数据块被删除,其元数据与该数据块的关联关系被解除之后,其计数相应减1。 Before [0088] snapshot, the data blocks are only the source file metadata reference, therefore, this case corresponds to the reference count module reference count 113 are both 1; creating module 11 for each data block in the snapshot file to create a snapshot after the document, and the association relationship creation module 112 associated with the snapshot file data block, which count is incremented, at each snapshot file to be deleted and releasing the association with the data block, or the block is deleted, after its relation to the metadata associated data block is released, its count is decremented correspondingly.

[0089] 判断模块131用于在接收到来自用户对数据块的更新操作请求后,先判断与该数据块对应的引用计数模块113的计数是否大于1,若是,则判断为该数据块同时与快照文件、数据块的元数据关联,则数据块更新模块132不能直接对该数据块进行更新操作,而是需要数据块新建模块133对该数据块进行复制,得到新建数据块,并保存在存储媒介14中。 [0089] The determining module 131 for the update operation after receiving a request from a user to a data block, first determine the data block corresponding to the reference count module 113 whether the count is greater than 1, and if yes, determines that the data block and simultaneously snapshot file, meta data associated with the data block, the data block update module 132 not directly update the data block, a new block of data but requires the module data block copy 133, to obtain new data blocks, and stored in the memory media 14. 数据块更新模块132用于对原数据块和新建数据块中的任一者进行更新操作,若选择对原数据块进行更新操作,关联关系建立模块112用于将新建数据块与快照文件关联后备份保存,若选择对新建数据块进行更新操作,关联关系建立模块112用于保持原数据块与快照文件关联后备份保存,并删除源文件的元数据中的原数据块的元数据,加入新建数据块的元数据,建立新建数据块的元数据与新建数据块的关联关系。 Module 132 for updating the data blocks to the original data block and the new data block in any one of the update is performed if the selection of the original data block update operation, association relationship creation module 112 for associating a new file data block snapshot backup storage, if the selection of the new data block update operation, association relationship creation module 112 after storage for holding the backup data block associated with the original snapshot file, and delete the original data block of the metadata the metadata in the source file, add a new metadata of the data block, the new relationship metadata associated with the new data block a data block. 同时,由于原数据块的元数据被删除,其与原数据块的关联关系被解除,因此,与原数据块对应的引用计数模块113的引用计数减1,还可以针对新建数据块,建立对应的另一引用计数模块,由于新建数据块由于被其元数据引用,与其对应的引用计数模块计数为1。 Meanwhile, since the meta data of the original data block is deleted, its association with the original data block is released, so that the original data block corresponding references counting modules referenced 113. count by 1, also for a new data block, in association another reference count module, because due to the new data block reference in its metadata, the corresponding reference count for block count 1. 若判断模块131的判断结果为该数据块没有同时与快照文件、数据块的元数据关联,那么,数据块更新模块132用于根据用户的更新操作请求直接对原数据块进行更新操作,不需要对原数据块进行复制。 If the result of determination module 131 determines that the data blocks are not simultaneously snapshot file, data blocks associated with the metadata, then the module 132 updating blocks for updating the original data block update operation directly according to the user's operation request does not need copying the original data block.

[0090] 本发明提供的快照实现方法相对于现有成熟的快照实现方法,新建的快照文件是由复制数据块的元数据而得来的,得到的快照文件和该数据块的元数据与该相同数据块进行关联,能够满足分布式文件系统的需求,还减少了对数据块的重复复制的过程,降低了系统资源和IO资源的消耗,改善了存储系统在引入快照机制后系统性能普遍变得较低的问题。 [0090] The method of the present invention to provide snapshots for the snapshot with respect to existing mature implementation, new metadata snapshot file is copied to the obtained data block, metadata snapshot file and the data block obtained with the associating the same data block, to meet the needs of distributed file system, the process also reduces duplicate copy of the data block, reducing consumption of system resources and IO resources, the storage system is improved after the introduction of snapshot mechanism generally varying system performance the problem was low.

[0091] 以上内容是结合具体的实施方式对本发明所作的进一步详细说明,不能认定本发明的具体实施只局限于这些说明。 [0091] The above is described in further detail with reference to specific embodiments of the present invention made by the present invention should not be considered limited to these specific embodiments described. 对于本发明所属技术领域的普通技术人员来说,在不脱离本发明构思的前提下,还可以做出若干简单推演或替换,都应当视为属于本发明的保护范围。 Those of ordinary skill in the art for the present invention, without departing from the spirit of the present invention, can make various simple deduction or replacement, should be deemed to belong to the scope of the present invention.

Claims (8)

1. 一种基于分布式文件系统的快照实现方法,其特征在于,包括: 复制数据块的元数据创建快照文件,并保存; 将所述快照文件和所述数据块的元数据与所述数据块进行关联,关联后,所述数据块同时被所述快照文件、所述数据块的元数据引用; 还包括:在需要对数据块进行更新操作时,判断所述数据块是否同时与所述快照文件、 所述数据块的元数据关联,若是,对所述数据块进行复制,得到新建数据块;对所述数据块和所述新建数据块中的任一者进行更新操作,另一者与所述快照文件关联后保存; 判断所述数据块是否同时与所述快照文件、所述数据块的元数据关联,指的是判断所述数据块是否同时被所述快照文件、所述数据块的元数据引用。 A distributed file system based on a snapshot implemented method, comprising: a metadata creating block copy data snapshot file, and save; metadata file and the snapshot of the data block with the data after the block association, association, the data block is simultaneously the snapshot file metadata reference to the data block; further comprising: when the need to update the data blocks, the data block is determined simultaneously with the snapshot files, the metadata associated with the data block, and if yes, copying the data block to obtain new data block; any of said new data block and the data block of one update operation, and the other after storage associated with the snapshot file; determining whether the data block while the snapshot file, the data element associated with the data block, refers to whether the data block is determined simultaneously by the snapshot file, the data metadata block references.
2. 如权利要求1所述的基于分布式文件系统的快照实现方法,其特征在于,若否,则直接对所述数据块进行更新操作。 2. snapshots distributed file system based on the method of claim 1, wherein, if not, the data block directly update operation.
3. 如权利要求1或2所述的基于分布式文件系统的快照实现方法,其特征在于,还包括:建立与数据块对应的引用计数器;对所述数据块当前被所述快照文件、所述数据块的元数据关联的次数进行计数。 3. The snapshot may be implemented based on the distributed file system of claim 1 or claim 2, characterized in that, further comprising: establishing a reference corresponding to the data block counter; the data block of the current by the snapshot file, the metadata associated number of said data blocks is counted.
4. 如权利要求3所述的基于分布式文件系统的快照实现方法,其特征在于,判断所述数据块是否同时与所述快照文件、所述数据块的元数据关联的方法包括:判断所述引用计数器的计数是否大于1,若是,则判断为所述数据块同时与所述快照文件、所述数据块的元数据关联。 The determination: implemented method as claimed snapshot based on the distributed file system according to claim 3, wherein determining whether the data block while the snapshot file, the metadata associated with the data block comprising if said count is greater than a reference counter, and if yes, determining that the data block associated with the element while the snapshot file, the data block.
5. -种基于分布式文件系统的快照实现装置,其特征在于,包括快照文件创建模块和关联关系建立模块,其中, 所述快照文件创建模块用于复制数据块的元数据创建快照文件,并保存; 所述关联关系建立模块用于将所述快照文件和所述数据块的元数据与所述数据块进行关联,关联后,所述数据块同时被所述快照文件、所述数据块的元数据引用; 还包括判断模块、数据块更新模块和数据块新建模块,其中, 所述判断模块在所述数据块更新模块需要对数据块进行更新操作时,判断所述数据块是否同时与所述快照文件、所述数据块的元数据关联,所述判断所述数据块是否同时与所述快照文件、所述数据块的元数据关联,指的是判断所述数据块是否同时被所述快照文件、 所述数据块的元数据引用; 所述数据块新建模块用于在所述判断模块的判断结果为是 5. - kind of distributed file system snapshot based implementation of apparatus comprising snapshot file creation module and the association relationship creation module, wherein the snapshot file is created for copying the metadata module to create a snapshot file of data blocks, and save; means for establishing the association relationship metadata file and the snapshot of the data block associated with the data block, after the association, the data block is simultaneously the snapshot file, the data block metadata reference; further comprising a time determining module, a data block and a data updating module block new module, wherein, the determining module updating module in the data block needs to update the data block, the data block is determined simultaneously with the said snapshot associated metadata file, said data blocks, said determining whether the data block while the snapshot file, the metadata associated with the data block, refers to whether the data block is determined by the simultaneously snapshot file, metadata for the data block reference; the data block for the new module is the determination result of the determining module is yes 时,对所述数据块进行复制,得到新建数据块; 所述数据块更新模块用于对所述数据块和所述新建数据块中的任一者进行更新操作; 所述关联关系建立模块用于将所述数据块和所述新建数据块中的另一者与所述快照文件关联后保存。 When the data block to be copied, to obtain new data block; means for updating the data block of the data block and the new data block in any one of the update operation; the association relationship creation module in the other of the data block and the new data block with the saved snapshot associated with the file.
6. 如权利要求5所述的基于分布式文件系统的快照实现装置,其特征在于,所述数据块更新模块还用于在所述判断模块判断结果为否时,直接对所述数据块进行更新操作。 Snapshot distributed file system based implementation of the apparatus 5 when the determining module determines the result is negative, the data block directly as claimed in claim 6, wherein the data updating module is further configured to block update operation.
7. 如权利要求5或6所述的基于分布式文件系统的快照实现装置,其特征在于,还包括与数据块对应的引用计数模块,所述引用计数模块用于对所述数据块当前被所述快照文件、所述数据块的元数据关联的次数进行计数。 7. The distributed file system based snapshots realized 5 or claim 6, characterized in that the module further comprising a reference count corresponding to the data block, the reference count module for the data block currently the snapshot file, the number of metadata associated with the data block is counted.
8.如权利要求7所述的基于分布式文件系统的快照实现装置,其特征在于,所述判断模块用于判断所述引用计数器的计数是否大于1,若是,则判断为所述数据块同时与所述快照文件、所述数据块的元数据关联。 8. A system as claimed in distributed file snapshot based implementation if the device of claim 7, it is determined that the data block simultaneously, characterized in that said means for determining determines whether the count of the reference counter is greater than 1, metadata associated with the snapshot file, the data block.
CN 201210159290 2012-05-22 2012-05-22 Method and device for realizing snapshot based on distributed file system CN102722536B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN 201210159290 CN102722536B (en) 2012-05-22 2012-05-22 Method and device for realizing snapshot based on distributed file system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN 201210159290 CN102722536B (en) 2012-05-22 2012-05-22 Method and device for realizing snapshot based on distributed file system

Publications (2)

Publication Number Publication Date
CN102722536A true CN102722536A (en) 2012-10-10
CN102722536B true CN102722536B (en) 2015-06-03

Family

ID=46948297

Family Applications (1)

Application Number Title Priority Date Filing Date
CN 201210159290 CN102722536B (en) 2012-05-22 2012-05-22 Method and device for realizing snapshot based on distributed file system

Country Status (1)

Country Link
CN (1) CN102722536B (en)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103559139B (en) * 2013-10-23 2016-08-17 华为技术有限公司 A data storage method and apparatus
CN103842954B (en) * 2013-12-05 2017-02-15 华为技术有限公司 A data processing method in a memory system, devices, systems, and servers
CN103678715B (en) * 2013-12-31 2017-06-23 无锡城市云计算中心有限公司 A distributed file system metadata information management support snapshots

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1567262A (en) * 2003-06-10 2005-01-19 联想(北京)有限公司 On-line data backup method based on data volume snapshot
CN1655146A (en) * 2004-02-12 2005-08-17 国际商业机器公司 Method and apparatus for file system snapshot persistence

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1567262A (en) * 2003-06-10 2005-01-19 联想(北京)有限公司 On-line data backup method based on data volume snapshot
CN1655146A (en) * 2004-02-12 2005-08-17 国际商业机器公司 Method and apparatus for file system snapshot persistence

Also Published As

Publication number Publication date Type
CN102722536A (en) 2012-10-10 application

Similar Documents

Publication Publication Date Title
US8204859B2 (en) Systems and methods for managing replicated database data
US7613750B2 (en) Creating frequent application-consistent backups efficiently
US7899788B2 (en) Using a data protection server to backup and restore data on virtual servers
US7257690B1 (en) Log-structured temporal shadow store
US7320059B1 (en) Methods and apparatus for deleting content from a storage system
US7596713B2 (en) Fast backup storage and fast recovery of data (FBSRD)
US20100199042A1 (en) System and method for secure and reliable multi-cloud data replication
US8352422B2 (en) Data restore systems and methods in a replication environment
US20090300079A1 (en) Integrated remote replication in hierarchical storage systems
US20050216535A1 (en) Backup method, storage system, and program for backup
US20090249005A1 (en) System and method for providing a backup/restore interface for third party hsm clients
US20070271428A1 (en) Method and apparatus of continuous data backup and access using virtual machines
US8898112B1 (en) Write signature command
US20050149683A1 (en) Methods and systems for data backups
US20050015415A1 (en) Method, system, and program for performing an input/output operation with respect to a logical storage device
US20110246416A1 (en) Stubbing systems and methods in a data replication environment
US20080104146A1 (en) System for automatically shadowing encrypted data and file directory structures for a plurality of network-connected computers using a network-attached memory with single instance storage
US20070050415A1 (en) Methods and apparatus for scheduling an action on a computer
US8924668B1 (en) Method and apparatus for an application- and object-level I/O splitter
US20110246429A1 (en) Stub file prioritization in a data replication system
US20100070466A1 (en) Data transfer techniques within data storage devices, such as network attached storage performing data migration
US20110161299A1 (en) Systems and methods for performing data management operations using snapshots
US7877357B1 (en) Providing a simulated dynamic image of a file system
US8954663B1 (en) System, method and computer program product for synchronizing data written to tape including writing an index into a data partition so that data can be recovered in case of failure
US20110258461A1 (en) System and method for resource sharing across multi-cloud arrays

Legal Events

Date Code Title Description
C06 Publication
C10 Request of examination as to substance
C14 Granted