CN104158898B - Method for updating a distributed file system file layout - Google Patents

Method for updating a distributed file system file layout Download PDF

Info

Publication number
CN104158898B
CN104158898B CN 201410421947 CN201410421947A CN104158898B CN 104158898 B CN104158898 B CN 104158898B CN 201410421947 CN201410421947 CN 201410421947 CN 201410421947 A CN201410421947 A CN 201410421947A CN 104158898 B CN104158898 B CN 104158898B
Authority
CN
Grant status
Grant
Patent type
Prior art keywords
file
metadata
client
step
layout
Prior art date
Application number
CN 201410421947
Other languages
Chinese (zh)
Other versions
CN104158898A (en )
Inventor
郭照斌
季旻
姜国梁
马振杰
杨鹏
Original Assignee
曙光信息产业股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Grant date

Links

Abstract

本发明提供了一种分布式文件系统中文件布局的更新方法,在没有分布式锁的分布式文件系统中,当客户端向元数据服务器打开文件时,元数据服务器记录下该文件被该客户端打开的打开状态,当客户端完成数据访问向元数据服务器关闭文件时,元数据服务器删除该客户端打开该文件的记录。 The present invention provides a method for updating a distributed file system layout file, the file system is not distributed in a distributed lock, when the client opens the file to the server metadata, the metadata server of the file record is the client open end of the open state, when the client server data access is completed close the file, delete the metadata server to the client-side metadata to open the file's record. 当文件的元数据布局有改动,需要更新时,元数据服务器根据其记录的文件打开记录,先向记录的客户端发送更新文件布局的请求,等所有客户端完成更新后,将更新后的元数据保存。 After the element layout file when the metadata changes are required to update the metadata records according to the file server to open the recording, the recording Xianxiang client sends a request to update a file layout, etc. after completion of updating all clients will be updated data retention. 本发明在不需要分布式锁的情况下,完成了元数据布局的一致性更新,保证了元数据在多客户端间的一致性访问,并且减少了申请分布式锁的开销。 The present invention without the need for distributed lock to complete the layout update metadata consistency, to ensure consistency between multiple access to the metadata client application and reduces the overhead of distributed lock.

Description

一种分布式文件系统中文件布局的更新方法 Method for updating a distributed file system file layout

技术领域 FIELD

[0001] 本发明涉及一种更新方法,具体涉及一种分布式文件系统中文件布局的更新方法。 [0001] The present invention relates to a method for updating, particularly relates to a method for updating a file in the distributed file system layout.

背景技术 Background technique

[0002] 分布式文件系统一般包含客户端,元数据服务器和数据服务器,客户端负责文件数据的访问接口制定,元数据服务器处理文件的布局及属性,数据服务器存储文件的数据内容。 [0002] distributed file system generally includes a client, the data content metadata server and data server, the client is responsible for the development of access interface file data, layout and metadata attributes of a file server processing, data storage file server.

[0003] 对于分布式文件系统,多个客户端同时访问一个文件时,为了保证其访问数据的一致性,该文件的元数据更新时必须有一种方法使得访问该元数据的客户端都感知到,传统的做法是通过元数据的分布式锁来实现,大致过程为,客户端访问文件时必须先申请该文件的分布式锁,更新元数据时也需要在锁的保护内进行,而该方法的缺点有二,一是分布式锁实现难度很大,且维护起来很不方便,部分分布式文件系统没有分布式锁,二是每次访问元数据时都需要加锁,降低了元数据访问的效率,增加了访问元数据的时延和开销。 [0003] For a distributed file system, when multiple clients access a file, in order to ensure the consistency of its data access, it must have the file when the metadata is updated such that a process accessing the metadata clients are perceived the traditional approach is achieved through a distributed lock metadata, the general process is, you must first apply for a client to access files distributed lock the file, update metadata also needs to be in the lock to protect data, and the method there are two shortcomings, one is distributed when the lock is very difficult to achieve, and very easy to maintain, no part of the distributed file system distributed lock, the second is every need to lock access to metadata, reducing the metadata access the efficiency and increasing access to the metadata latency and overhead.

发明内容 SUMMARY

[0004] 为了克服上述现有技术的不足,本发明提供一种分布式文件系统中文件布局的更新方法,只需要在访问文件的第一次时,由元数据服务器将文件的打开状态记录下来,而更新文件布局时只需要通知指定的记录中的客户端即可,该发明减少了元数据访问频繁加锁的开销,并且可以保证文件布局的一致性更新。 [0004] In order to overcome the above deficiencies of the prior art, the present invention provides a method for updating a file in the distributed file system layout, only when accessing a file for the first time, by the metadata server opens a file recorded state , but only you need to be notified when the update file layout specified record client can, the invention reduces the metadata access frequently locked overhead, and can ensure the consistency of the file layout updates.

[0005] 为了实现上述发明目的,本发明采取如下技术方案: [0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006]本发明提供一种分布式文件系统中文件布局的更新方法,所述方法包括以下步骤: [0006] The present invention provides a method for updating files in a distributed file system layout, said method comprising the steps of:

[0007]步骤1:元数据服务器记录文件的打开状态; [0007] Step 1: records the metadata server file open;

[0008]步骤2:元数据服务器删除客户端打开文件的打开状态; [0008] Step 2: Remove the metadata server client opens a file open state;

[0009]步骤3:元数据服务器保存更新后的文件布局。 After the layout file to save the updated metadata server: [0009] Step 3.

[0010] 所述步骤1包括以下步骤: [0010] Step 1 comprises the steps of:

[0011]步骤1-1:客户端向元数据服务器发起打开文件的请求; [0011] Step 1-1: The client initiates a request to a metadata server open the file;

[0012]步骤1-2:元数据服务器判断是否已记录该文件的打开状态,若未记录则记录该文件被该客户端打开的打开状态。 [0012] Step 1-2: the metadata server determines whether the records of the file open, if not the file is recorded is recorded to open the open end of the customer.

[0013]元数据服务器记录文件被该客户端打开的打开状态有以下方式: [0013] metadata server log file is opened in the opened state of the client the following ways:

[0014]方式1:记录到文件的扩展属性上; [0014] Mode 1: the extended attribute record to the file;

[0015]方式2:统一记录到指定的元数据文件夹中。 [0015] Embodiment 2: Unified specified metadata record to the folder.

[0016]所述步骤2包括以下步骤: [0016] Step 2 comprises the steps of:

[0017] 步骤2-1:客户端操作完文件后,向元数据服务器发送关闭文件的请求; [0017] Step 2-1: After operating the client file, close the file transmission request to the metadata server;

[0018] 步骤2-2:元数据服务器删除该客户端的打开文件的打开状态。 [0018] Step 2-2: Metadata server deletes the client opens the file open.

[0019]所述步骤3包括以下步骤: [0019] Step 3 comprises the following steps:

[0020]步骤3-1:文件布局有改动需要更新时,元数据服务器根基文件打开状态的记录, 向记录的客户端发送更新文件布局的请求; [0020] Step 3-1: When there are changes to update the layout file, the metadata server records the basic file open state, sends a request to update the file to the client the layout of the record;

[0021]步骤3-2:所有客户端完成文件布局更新后,将更新后的文件布局保存。 [0021] Step 3-2: After the completion of all client file layout update, save the file layout updated.

[0022]与现有技术相比,本发明的有益效果在于: [0022] Compared with the prior art, the beneficial effects of the present invention:

[0023]本发明提供了一种分布式文件系统中文件布局的更新方法,在没有分布式锁的分布式文件系统中,当客户端向元数据服务器打开文件时,元数据服务器记录下该文件被该客户端打开的打开状态,当客户端完成数据访问向元数据服务器关闭文件时,元数据服务器删除该客户端打开该文件的记录。 [0023] The present invention provides a method for updating a distributed file system layout file, the file system is not distributed in a distributed lock, when the client opens the file to the server metadata, the metadata server record file open end of the open state of the client, when the client server data access is completed close the file, metadata server deletes the client record to open the metadata of the file. 当文件的元数据布局有改动,需要更新时,元数据服务器根据其记录的文件打开记录,先向记录的客户端发送更新文件布局的请求,等所有客户成更新后,将更新后的兀数据保存。 Wu data layout file when the metadata changes are required to update the metadata records according to the file server to open the recording, the recording Xianxiang client sends a request to update a file layout, like all the clients to update, updated save. 本发明在不需要分布式锁的情况下,完成了兀数据布局的一致性更新,保证了元数据在多客户端间的一致性访问,并且减少了申请分布式锁的开销。 The present invention without the need for distributed lock to complete the layout of the data consistency update Wu, to ensure consistency between multiple access to the metadata client application and reduces the overhead of distributed lock.

附图说明 BRIEF DESCRIPTION

[0024]图1是分布式文件系统中文件布局的更新方法流程图; [0024] FIG. 1 is a method for updating files in a distributed file system layout flowchart;

[0025]图2是文件打开记录在内存中保存形式示意图。 [0025] FIG. 2 is a schematic view of a file stored in the form of open records in memory.

具体实施方式 detailed description

[0026]下面结合附图对本发明作进一步详细说明。 [0026] The following figures of the present invention will be further described in detail in conjunction.

[0027]如图1,本发明提供一种分布式文件系统中文件布局的更新方法,所述方法包括以下步骤: [0027] As shown in FIG 1, the present invention provides a method for updating files in a distributed file system layout, said method comprising the steps of:

[0028]步骤1:元数据服务器记录文件的打开状态; [0028] Step 1: records the metadata server file open;

[0029]步骤2:元数据服务器删除客户端打开文件的打开状态; [0029] Step 2: Remove the metadata server client opens a file open state;

[0030]步骤3:元数据服务器保存更新后的文件布局。 After the layout file to save the updated metadata server: [0030] Step 3.

[0031]所述步骤1包括以下步骤: [0031] Step 1 comprises the steps of:

[0032]步骤1-1:客户端向元数据服务器发起打开文件的请求; [0032] Step 1-1: The client initiates a request to a metadata server open the file;

[0033]步骤1-2:元数据服务器判断是否已记录该文件的打开状态,若未记录则记录该文件被该客户端打开的打开状态。 [0033] Step 1-2: the metadata server determines whether the records of the file open, if not the file is recorded is recorded to open the open end of the customer.

[0034]元数据服务器记录文件被该客户端打开的打开状态有以下方式: [0034] metadata server log file is opened in the opened state of the client the following ways:

[0035]方式1:记录到文件的扩展属性上; [0035] Mode 1: the extended attribute record to the file;

[0036]方式2:统一记录到指定的元数据文件夹中。 [0036] Embodiment 2: Unified specified metadata record to the folder.

[0037]所述步骤2包括以下步骤: [0037] Step 2 comprises the steps of:

[0038]步骤2-1:客户端操作完文件后,向元数据服务器发送关闭文件的请求; [0038] Step 2-1: After operating the client file, close the file transmission request to the metadata server;

[0039]步骤2-2:元数据服务器删除该客户端的打开文件的打开状态。 [0039] Step 2-2: Metadata server deletes the client opens the file open.

[0040]所述步骤3包括以下步骤: [0040] Step 3 comprises the following steps:

[0041]步骤3-1:文件布局有改动需要更新时,元数据服务器根基文件打开状态的记录, 向记录的客户端发送更新文件布局的请求; [0041] Step 3-1: When there are changes to update the layout file, the metadata server records the basic file open state, sends a request to update the file to the client the layout of the record;

[0042]步骤3-2:所有客户端完成文件布局更新后,将更新后的文件布局保存。 [0042] Step 3-2: After the completion of all client file layout update, save the file layout updated.

[0043]实施例 [0043] Example

[0044] A.文件的打开状态在内存中的记录方法: The recording method in the open state memory [0044] A. file:

[0045]在内存中,采用客户端id为链表头的双向链表和以文件inode为链表头的双向链表交叉的解构存储,方便其记录和查找,具体见附图2。 [0045] In the memory using a client id is doubly linked list head and a file inode doubly linked list head is a cross deconstruction storage, easy to find and record, see Figure 2 particularly.

[0046] B.在磁盘上采用对应的b+树存储,以文件inode为关键字,其内被打开的客户端为key保存。 [0046] B. on the disk using the corresponding b + tree is stored, a file inode for the keyword which the client is opened to save key.

[0047]本发明提供了一种分布式文件系统中文件布局的更新方法,在没有分布式锁的分布式文件系统中,当客户端向元数据服务器打开文件时,元数据服务器记录下该文件被该客户端打开的打开状态,当客户端完成数据访问向元数据服务器关闭文件时,元数据服务器删除该客户端打开该文件的记录。 [0047] The present invention provides a method for updating a distributed file system layout file, the file system is not distributed in a distributed lock, when the client opens the file to the server metadata, the metadata server record file open end of the open state of the client, when the client server data access is completed close the file, metadata server deletes the client record to open the metadata of the file. 当文件的元数据布局有改动,需要更新时,元数据服务器根据其记录的文件打开记录,先向记录的客户端发送更新文件布局的请求,等所有客户端完成更新后,将更新后的元数据保存。 After the element layout file when the metadata changes are required to update the metadata records according to the file server to open the recording, the recording Xianxiang client sends a request to update a file layout, etc. after completion of updating all clients will be updated data retention. 本发明在不需要分布式锁的情况下,完成了元数据布局的一致性更新,保证了元数据在多客户端间的一致性访问,并且减少了申请分布式锁的开销。 The present invention without the need for distributed lock to complete the layout update metadata consistency, to ensure consistency between multiple access to the metadata client application and reduces the overhead of distributed lock.

[0048]最后应当说明的是:以上实施例仅用以说明本发明的技术方案而非对其限制,所属领域的普通技术人员参照上述实施例依然可以对本发明的具体实施方式进行修改或者等同替换,这些未脱离本发明精神和范围的任何修改或者等同替换,均在申请待批的本发明的权利要求保护范围之内。 [0048] Finally, it should be noted that: the above embodiments are intended to illustrate the present invention rather than limiting, those of ordinary skill in the art modifications may be made to the specific embodiments of the present invention with reference to the above-described embodiments or equivalent replacements , any of these modifications are not departing from the spirit and scope of the present invention, or equivalents, it is claimed in the copending application of the present invention within the scope of the claims.

Claims (1)

  1. 1.一种分布式文件系统中文件布局的更新方法,其特征在于:所述方法包括以下步骤: 步骤1:元数据服务器记录文件的打开状态; 步骤2:元数据服务器删除客户端打开文件的打开状态; 步骤3:元数据服务器保存更新后的文件布局; 所述步骤1包括以下步骤: 步骤1-1:客户端向元数据服务器发起打开文件的请求; 步骤1-2:元数据服务器判断是否己记录该文件的打开状态,若未记录则记录该文件被该客户端打开的打开状态; 元数据服务器记录文件被该客户端打开的打开状态有以下方式: 方式1:记录到文件的扩展属性上; 方式2:统一记录到指定的元数据文件夹中; 所述步骤2包括以下步骤: 步骤2-1:客户端操作完文件后,向元数据服务器发送关闭文件的请求; 步骤2-2:元数据服务器删除该客户端的打开文件的打开状态; 所述步骤3包括以下步骤: 步骤3-1:文件 The method of updating the file layout A distributed file system, which is characterized in that: said method comprises the following steps: Step 1: the metadata server log files opened state; Step 2: Remove the client metadata server to open the file opened state; step 3: metadata file server stores the updated layout; step 1 comprises the following steps: step 1: the client initiates a request to a metadata server open the file; step 1-2: Analyzing the metadata server whether the file has records open state, if no record is the record file is opened the open state of the client; metadata server log file is opened in the opened state of the client has the following modes: mode 1: to record extension files the attribute; mode 2: unified specified metadata record to the folder; step 2 comprises the following steps: step 2-1: after the operation is complete the client file, close the file transmission request to the metadata server; step 2 2: remove the client metadata server file open end of the open state; step 3 comprises the following steps: step 3-1: file 局有改动需要更新时,元数据服务器根据文件打开状态的记录,向记录的客户端发送更新文件布局的请求; 步骤3-2:所有客户端完成文件布局更新后,将更新后的文件布局保存。 Needs to be updated when there are changes Board, according to the file server open the metadata recording state, sends a request to update the layout file to the client recorded; Step 3-2: After the completion of all client layout update file, the file layout of the updated storage .
CN 201410421947 2014-08-25 2014-08-25 Method for updating a distributed file system file layout CN104158898B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN 201410421947 CN104158898B (en) 2014-08-25 2014-08-25 Method for updating a distributed file system file layout

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN 201410421947 CN104158898B (en) 2014-08-25 2014-08-25 Method for updating a distributed file system file layout

Publications (2)

Publication Number Publication Date
CN104158898A true CN104158898A (en) 2014-11-19
CN104158898B true CN104158898B (en) 2018-01-19

Family

ID=51884303

Family Applications (1)

Application Number Title Priority Date Filing Date
CN 201410421947 CN104158898B (en) 2014-08-25 2014-08-25 Method for updating a distributed file system file layout

Country Status (1)

Country Link
CN (1) CN104158898B (en)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101697168A (en) * 2009-10-22 2010-04-21 中国科学技术大学; Method and system for dynamically managing metadata of distributed file system
CN101751415A (en) * 2008-12-09 2010-06-23 中国移动通信集团公司 Metadata service system metadata synchronized method and writing server updating method
CN102447742A (en) * 2011-11-24 2012-05-09 中兴通讯股份有限公司 Dynamic data active and standby synchronization method and system as well as metadata server
CN103152398A (en) * 2013-02-07 2013-06-12 中国科学院计算技术研究所 Method and system for file synchronization among multiple terminal files

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8214329B2 (en) * 2008-08-26 2012-07-03 Zeewise, Inc. Remote data collection systems and methods

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101751415A (en) * 2008-12-09 2010-06-23 中国移动通信集团公司 Metadata service system metadata synchronized method and writing server updating method
CN101697168A (en) * 2009-10-22 2010-04-21 中国科学技术大学; Method and system for dynamically managing metadata of distributed file system
CN102447742A (en) * 2011-11-24 2012-05-09 中兴通讯股份有限公司 Dynamic data active and standby synchronization method and system as well as metadata server
CN103152398A (en) * 2013-02-07 2013-06-12 中国科学院计算技术研究所 Method and system for file synchronization among multiple terminal files

Also Published As

Publication number Publication date Type
CN104158898A (en) 2014-11-19 application

Similar Documents

Publication Publication Date Title
US6789094B2 (en) Method and apparatus for providing extended file attributes in an extended attribute namespace
US20080005145A1 (en) Data processing
US20090313260A1 (en) Methods and systems for assisting information processing by using storage system
US20080162482A1 (en) Providing Enterprise Management of Amorphous Communities
US20130218934A1 (en) Method for directory entries split and merge in distributed file system
US20120330904A1 (en) Efficient file system object-based deduplication
US20120317155A1 (en) Hybrid Data Management System and Method for Managing Large, Varying Datasets
CN102411637A (en) Metadata management method of distributed file system
US20080033920A1 (en) Method and apparatus for searching metadata
US20080183662A1 (en) Resolving at least one file-path for a change-record of a computer file-system object in a computer file-system
CN103856567A (en) Small file storage method based on Hadoop distributed file system
US20110184907A1 (en) Method and system for guaranteed traversal during shadow migration
CN102291450A (en) Online hierarchical data storage method for internal cluster storage system
US7610304B2 (en) Techniques for performing file operations involving a link at a database management system
CN103530327A (en) Method for migrating data from non-relational database to relational database
US20120221523A1 (en) Database Backup and Restore with Integrated Index Reorganization
US20070299814A1 (en) Autonomic lock escalation in an sql environment
CN101739310A (en) Method and device for cycling backup
US20130054540A1 (en) File system object-based deduplication
US20070143286A1 (en) File management method in file system and metadata server therefor
US20080016107A1 (en) Data processing
US20100036872A1 (en) Data management method
CN102541985A (en) Organization method of client directory cache in distributed file system
US8170985B2 (en) Primary stub file retention and secondary retention coordination in a hierarchical storage system
US8516022B1 (en) Automatically committing files to be write-once-read-many in a file system

Legal Events

Date Code Title Description
C06 Publication
C10 Entry into substantive examination
GR01