CN106919713B - Cluster file system and distributed file system multi-client file control method - Google Patents
Cluster file system and distributed file system multi-client file control method Download PDFInfo
- Publication number
- CN106919713B CN106919713B CN201710146257.1A CN201710146257A CN106919713B CN 106919713 B CN106919713 B CN 106919713B CN 201710146257 A CN201710146257 A CN 201710146257A CN 106919713 B CN106919713 B CN 106919713B
- Authority
- CN
- China
- Prior art keywords
- callback
- client
- function
- file system
- file
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 28
- 230000006870 function Effects 0.000 claims description 117
- 238000012545 processing Methods 0.000 claims description 10
- 238000012217 deletion Methods 0.000 claims description 6
- 230000037430 deletion Effects 0.000 claims description 6
- 101000878595 Arabidopsis thaliana Squalene synthase 1 Proteins 0.000 claims description 3
- 239000011800 void material Substances 0.000 description 33
- 101150026173 ARG2 gene Proteins 0.000 description 11
- 101100005166 Hypocrea virens cpa1 gene Proteins 0.000 description 11
- 101100379634 Xenopus laevis arg2-b gene Proteins 0.000 description 11
- 101100379633 Xenopus laevis arg2-a gene Proteins 0.000 description 9
- 101150088826 arg1 gene Proteins 0.000 description 9
- 101100323865 Xenopus laevis arg1 gene Proteins 0.000 description 2
- 238000004590 computer program Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- WGZDBVOTUVNQFP-UHFFFAOYSA-N N-(1-phthalazinylamino)carbamic acid ethyl ester Chemical compound C1=CC=C2C(NNC(=O)OCC)=NN=CC2=C1 WGZDBVOTUVNQFP-UHFFFAOYSA-N 0.000 description 1
- VQLYBLABXAHUDN-UHFFFAOYSA-N bis(4-fluorophenyl)-methyl-(1,2,4-triazol-1-ylmethyl)silane;methyl n-(1h-benzimidazol-2-yl)carbamate Chemical compound C1=CC=C2NC(NC(=O)OC)=NC2=C1.C=1C=C(F)C=CC=1[Si](C=1C=CC(F)=CC=1)(C)CN1C=NC=N1 VQLYBLABXAHUDN-UHFFFAOYSA-N 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000004224 protection Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/182—Distributed file systems
- G06F16/1824—Distributed file systems implemented using Network-attached Storage [NAS] architecture
- G06F16/183—Provision of network file services by network file servers, e.g. by using NFS, CIFS
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention provides a novel cluster file system and a multi-client file control method of a distributed file system. According to the invention, through the mode of registering the callback function, when one client operates the file of the distributed file system, all clients which have registered with the file system receive the operation of the file, and then execute the callback function and execute the corresponding operation, thereby achieving the control of multiple clients. The method and the device are simple to implement and can well control the operation of the file.
Description
Technical Field
The invention relates to the field of computers, in particular to a cluster file system and a distributed file system multi-client file control method.
Background
After entering the information age, with the continuous development of technologies, the storage requirements in various industries are increased in a geometric trend, which results in that the conventional local file system cannot meet the requirements of services. Thus, various different types of distributed file systems have come into play, such as CEPH, Lustre, GPFS, Google FS, etc., in response to application requirements.
For a distributed file system, different clients are not visible to each other for the operation of file system metadata and data, since their clients are distributed on different physical machines. Common client protections are NFS, FUSE, CIFS, etc. For the client with the cache, the client will keep the reference count of the file after creating the file, at this time, if other clients delete the file, the file deletion is unsuccessful and the space is not released because the reference count of the file is held by other clients.
Therefore, a novel cluster file system and a distributed file system multi-client file control method are necessary.
Disclosure of Invention
In order to solve the technical problems, the technical scheme of the invention is as follows:
the invention provides a cluster file system, which comprises a distributed file system and a plurality of clients mounted to the cluster file system, wherein callback registration functions are added in the distributed file system and respectively provide processing for the callback functions, the plurality of clients respectively register the callback functions defined by the clients to the file system through interfaces, after the clients execute file operation, the distributed file system receives a request, after the request is processed, all the callback functions in a callback deletion function queue are executed, the corresponding clients execute the registered functions, and attribute information of files is updated to own caches.
Furthermore, the callback registration function added in the distributed file system is realized by declaring a function pointer, declaring a structural body containing the function pointer and defining a registration interface of the callback function.
Furthermore, the plurality of clients register the self-defined callback function to the file system through the interface respectively by declaring a function pointer, defining the callback function to be actually executed, initializing the function pointer and registering the self-defined callback function to the file system through the interface function.
Further, the function registered by the corresponding client for executing is file _ op _ update.
Further, the client is one of an NFS client, a CIFS client, a FUSE client, and an NFS-Ganesha client.
In addition, the invention also provides a method for controlling the files of the distributed file system on the multiple clients, which comprises the following steps:
SS 1: adding callback registration functions in the distributed file system, and respectively providing processing for the callback functions;
SS 2: the client registers a callback function defined by the client to the file system through the interface respectively;
SS 3: the client executes file operation, the distributed file system receives the request, executes all callback functions in the callback deletion function queue after processing, the corresponding client executes the registered functions, and updates the attribute information of the file to the own cache.
Further, step SS1 specifically includes the following steps:
1) declaring function pointer client _ ino _ callback _ t
2) Declaring the function pointer client _ entry _ callback _ t
3) Declaration function pointer client _ remote _ callback _ t
4) Declaration function pointer client _ getgroups _ callback _ t
5) Declaring a structure containing 1) -4) four function pointers
6) And defining a registration interface of the callback function.
Further, step SS2 specifically includes the following steps:
1) declaring a function pointer file _ op _ callback
2) Defining the actual execution of the callback function file _ op _ update
3) Initializing function pointers args to file _ op _ update
4) The self-defined callback function is then registered with the file system via the interface function nfs _ reg _ cb.
Further, the corresponding client executes the registered function file _ op _ update in step SS 3.
Further, the client may be one of an NFS client, a CIFS client, a FUSE client, and an NFS-Ganesha client.
By the cluster file system and the file control method of the multi-client of the distributed file system, the distributed file system provides a callback registration function, the client wants to register the callback function of the distributed file system, and when other clients execute operation, all client functions for registering the callback can be called back to execute. Therefore, the distributed file system can accurately control files under multiple clients.
Drawings
FIG. 1 is a block diagram of a cluster file system architecture according to the present invention.
FIG. 2 is a flow chart of a method for implementing multi-client file control in a distributed file system according to the present invention.
Detailed Description
The technical scheme of the invention is further explained by combining the drawings and the specific embodiments in the specification. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
The invention provides a novel cluster file system and a multi-client file control method of the distributed file system, and aims to realize that the distributed file system accurately controls files under multiple clients.
The method is characterized in that a callback registration function is provided in the distributed file system, the client registers the callback function of the client to the distributed file system, and when other clients execute operation, all client functions which are registered and callback are called back to execute.
The specific implementation process is as follows:
FIG. 1 is a block diagram of a cluster file system architecture according to the present invention.
As shown in fig. 1, the cluster file system according to the embodiment of the present invention includes: the distributed file system is added with callback registration functions which are used for processing inode, retry, mount and groups callback functions respectively, the multiple clients register the callback functions defined by the multiple clients to the file system through interfaces respectively, after the clients execute file operation, the distributed file system receives a request, executes the callback to delete all the callback functions in a function queue after processing, the corresponding clients execute the registered functions, and update attribute information of files to own caches.
In one embodiment of the present invention, the client may be an NFS client, a CIFS client, a FUSE client, an NFS-Ganesha client, or the like.
In one embodiment of the invention, the callback registration function added in the distributed file system is realized by declaring a function pointer, declaring a structural body containing the function pointer and defining a registration interface of the callback function.
In one embodiment of the invention, the plurality of clients register the self-defined callback function to the file system through the interface respectively by declaring a function pointer, defining the callback function to be actually executed, initializing the function pointer and registering the self-defined callback function to the file system through the interface function.
In one embodiment of the present invention, the function registered by the corresponding client to execute is file _ op _ update.
Fig. 2 is a flowchart of a method for implementing multi-client file control of the distributed file system in the cluster file system according to an embodiment of the present invention.
As shown in fig. 2, the method for implementing multi-client file control in a distributed file system according to an embodiment of the present invention includes:
SS 1: adding callback registration functions in the distributed file system, and respectively providing processing for the callback functions;
SS 2: the client registers a callback function defined by the client to the file system through the interface respectively;
SS 3: the client executes file operation, the distributed file system receives the request, executes all callback functions in the callback deletion function queue after processing, the corresponding client executes the registered functions, and updates the attribute information of the file to the own cache.
In one embodiment of the present invention, the client may be an NFS client, a CIFS client, a FUSE client, an NFS-Ganesha client, or the like.
In an embodiment of the present invention, step SS1 specifically includes the following steps:
1) declaring function pointer client _ ino _ callback _ t
typedef void(*client_ino_callback_t)(void*arg1，void*arg2，void*arg3，void*arg4)；
// a pointer client _ ino _ callback _ t is asserted, which points to the parameters argl, arg2, arg3, arg4, returning a function with a value of void.
2) Declaring the function pointer client _ entry _ callback _ t
typedef void(*client_dentry_callback_t)(void*arg1，void*arg2，void*arg3，void*arg4)；
// a pointer client _ dense _ callback _ t is asserted, which points to the parameters arg1, arg2, arg3, arg4, returning a function with a value of void.
3) Declaration function pointer client _ remote _ callback _ t
typedef void(*client_remount_callback_t)(void*arg1，void*arg2，void*arg3，void*arg4)；
// a pointer client _ remote _ callback _ t is asserted, which points to the parameters arg1, arg2, arg3, arg4, returning a function with a value of void.
4) Declaration function pointer client _ getgroups _ callback _ t
typedef void(*client_getgroups_callback_t)(void*arg1，void*arg2，void*arg3，void*arg4)；
// a pointer client _ getgroups _ callback _ t is asserted, which points to the parameters arg1, arg2, arg3, arg4, returning a function with a value of void.
5) Declaring a structure containing 1) -4) four function pointers
6) Register interface for defining callback function
In an embodiment of the present invention, step SS2 specifically includes the following steps:
1) declaring a function pointer file _ op _ callback
typedefvoid(*file_op_callback)(void*argl，void*arg2，void*arg3，void*arg4)；
2) Defining the actual execution of the callback function file _ op _ update
void file_op_update(void*arg1，void*arg2，void*arg3，void*arg4){
update_file_attrs(arg1，arg2，arg3，arg4)；
}
// operations actually performed, i.e. operations to update files
3) Initializing function pointers args to file _ op _ update
file_op_callback args＝file_op_update；//
4) Then register the callback function defined by itself to the file system through the interface function nfs _ reg _ cb, and execute nfs _ reg _ cb (cmount, args);
in an embodiment of the present invention, the function registered by the corresponding client in step SS3 is file _ op _ update.
According to the method, the callback function is registered, when one client operates the file of the distributed file system, all clients which have registered in the file system receive the operation of the file, and then the callback function is executed, and the corresponding operation is executed. Thereby achieving multi-client control.
This patent realizes simply, can be fine control the operation of file.
Although some aspects have been described in the context of an apparatus, it is clear that these aspects also represent a description of the corresponding method, where a block or device corresponds to a method step or a feature of a method step. Similarly, aspects described in the context of a method step also represent a description of a corresponding block or item or a feature of a corresponding apparatus. Some or all of the method steps may be performed by (or using) hardware means, such as a microprocessor, programmable computer, or electronic circuitry. One or more of the most important method steps may be performed by such means.
The implementation can be performed in hardware or in software or using a digital storage medium, e.g. a floppy disk, a DVD, a blu-ray, a CD, a ROM, a PROM, an EPROM, an EEPROM, or a flash memory, having electronically readable control signals stored thereon, which cooperate (or are capable of cooperating) with a programmable computer system such that the respective method is performed. A data carrier may be provided with electronically readable control signals capable of cooperating with a programmable computer system such that the method described herein is performed.
The described implementations may also take the form of a computer program product with program code means for performing the method when the computer program product runs on a computer. The program code may be stored on a machine readable carrier.
The foregoing description is illustrative only, and it is to be understood that modifications and variations in the arrangements and details described herein will be apparent to those skilled in the art. It is therefore intended that the scope of the appended claims be limited only by the specific details presented by way of the foregoing description and explanation.
Claims (6)
1. A cluster file system comprises a distributed file system and a plurality of clients mounted to the cluster file system, and is characterized in that callback registration functions are added in the distributed file system and are used for processing the callback functions respectively, the plurality of clients register the callback functions defined by themselves to the file system through interfaces respectively, after the clients execute file operations, the distributed file system receives requests, after the requests are processed, all the callback functions in a callback deletion function queue are executed, the corresponding clients execute the registered functions, and attribute information of files is updated to caches of the corresponding clients; the callback registration function added in the distributed file system is realized by declaring a function pointer, declaring a structural body containing the function pointer and defining a registration interface of the callback function;
the method is characterized in that the plurality of clients register the self-defined callback function to the file system through the interface respectively by declaring a function pointer, defining the callback function actually executed, initializing the function pointer and registering the self-defined callback function to the file system through the interface function.
2. The clustered file system of claim 1, wherein the function registered for execution by the corresponding client is file _ op _ update.
3. The clustered file system as in claim 1, wherein the client is one of an NFS client, a CIFS client, a FUSE client, and an NFS-Ganesha client.
4. A method for controlling files of multiple clients of a distributed file system is characterized by comprising the following steps:
SS 1: adding callback registration functions in the distributed file system, and respectively providing processing for the callback functions;
SS 2: the client registers a callback function defined by the client to the file system through the interface respectively;
SS 3: the client executes file operation, the distributed file system receives the request, executes all callback functions in the callback deletion function queue after processing, the corresponding client executes the registered functions, and updates the attribute information of the file to the own cache;
step SS1 specifically includes the following steps:
1) declaring function pointer client _ ino _ callback _ t
2) Declaring the function pointer client _ entry _ callback _ t
3) Declaration function pointer client _ remote _ callback _ t
4) Declaration function pointer client _ getgroups _ callback _ t
5) Declaring a structure containing 1) -4) four function pointers
6) Defining a registration interface of a callback function;
step SS2 specifically includes the following steps:
1) declaring a function pointer file _ op _ callback
2) Defining the actual execution of the callback function file _ op _ update
3) Initializing function pointers args to file _ op _ update
4) The self-defined callback function is then registered with the file system via the interface function nfs _ reg _ cb.
5. The method according to claim 4, wherein the function registered by the corresponding client in step SS3 is file _ op _ update.
6. The method of claim 4, wherein the client is an NFS client,
One of a CIFS client, a FUSE client and an NFS-Ganesha client.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710146257.1A CN106919713B (en) | 2017-03-13 | 2017-03-13 | Cluster file system and distributed file system multi-client file control method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710146257.1A CN106919713B (en) | 2017-03-13 | 2017-03-13 | Cluster file system and distributed file system multi-client file control method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106919713A CN106919713A (en) | 2017-07-04 |
| CN106919713B true CN106919713B (en) | 2021-03-16 |
Family
ID=59462157
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710146257.1A Active CN106919713B (en) | 2017-03-13 | 2017-03-13 | Cluster file system and distributed file system multi-client file control method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106919713B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108900607B (en) * | 2018-06-28 | 2021-06-29 | 郑州云海信息技术有限公司 | SMB protocol request processing method and device and server |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103995752A (en) * | 2014-06-16 | 2014-08-20 | 上海斐讯数据通信技术有限公司 | Intermodule notification callback method and module interaction structure |
| CN104113587A (en) * | 2014-06-23 | 2014-10-22 | 华中科技大学 | Client metadata buffer optimization method of distributed file system |
| CN105700939A (en) * | 2016-04-21 | 2016-06-22 | 北京京东尚科信息技术有限公司 | Method and system for multi-thread synchronization in distributed system |
| CN106130868A (en) * | 2016-06-02 | 2016-11-16 | 腾讯科技（深圳）有限公司 | A kind of processing method adjusting back message and queue management system |
| CN106357723A (en) * | 2016-08-15 | 2017-01-25 | 杭州古北电子科技有限公司 | Synchronous system and method for multi-cluster information caching based on cloud host |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8504597B2 (en) * | 2005-09-09 | 2013-08-06 | William M. Pitts | Distributed file system consistency mechanism extension for enabling internet video broadcasting |
| CN102387204B (en) * | 2011-10-21 | 2014-12-10 | 中国科学院计算技术研究所 | Method and system for maintaining consistency of cluster caching |
| CN102541984B (en) * | 2011-10-25 | 2013-08-28 | 曙光信息产业(北京)有限公司 | File system of distributed type file system client side |
| CN103853613A (en) * | 2012-12-04 | 2014-06-11 | 中山大学深圳研究院 | Method for reading data based on digital family content under distributed storage |
| CN103873523B (en) * | 2012-12-14 | 2018-02-27 | 北京东方通科技股份有限公司 | client cluster access method and device |
| CN104601702B (en) * | 2015-01-19 | 2018-06-05 | 迈普通信技术股份有限公司 | Cluster remote procedure calling (PRC) method and system |
-
2017
- 2017-03-13 CN CN201710146257.1A patent/CN106919713B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103995752A (en) * | 2014-06-16 | 2014-08-20 | 上海斐讯数据通信技术有限公司 | Intermodule notification callback method and module interaction structure |
| CN104113587A (en) * | 2014-06-23 | 2014-10-22 | 华中科技大学 | Client metadata buffer optimization method of distributed file system |
| CN105700939A (en) * | 2016-04-21 | 2016-06-22 | 北京京东尚科信息技术有限公司 | Method and system for multi-thread synchronization in distributed system |
| CN106130868A (en) * | 2016-06-02 | 2016-11-16 | 腾讯科技（深圳）有限公司 | A kind of processing method adjusting back message and queue management system |
| CN106357723A (en) * | 2016-08-15 | 2017-01-25 | 杭州古北电子科技有限公司 | Synchronous system and method for multi-cluster information caching based on cloud host |
Non-Patent Citations (1)
| Title |
|---|
| 面向PB级存储系统的元数据集群管理容错方法研究与实现;王涌;《中国优秀硕士学位论文全文数据库 信息科技辑》;20090715;13 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106919713A (en) | 2017-07-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11288267B2 (en) | Pluggable storage system for distributed file systems | |
| US10896157B2 (en) | Clone file backup and restore | |
| US10025528B2 (en) | Managing transformations of snapshots in a storage system | |
| US8903773B2 (en) | Computer file storage, backup, restore and retrieval | |
| US9811329B2 (en) | Cloud based file system surpassing device storage limits | |
| US9870151B1 (en) | Backup time deduplication of common virtual disks from virtual machine backup images | |
| CN107577420B (en) | File processing method and device and server | |
| US8949829B1 (en) | Virtual machine disaster recovery | |
| US20170116188A1 (en) | Method and system for backup and recovery | |
| US20070130228A1 (en) | Filesystem snapshot enhancement to improve system performance | |
| US9396074B2 (en) | Virtual database rewind | |
| US9268778B2 (en) | Efficient backup of virtual data | |
| US20230018235A1 (en) | Utilizing a tablespace to export to a native database recovery environment | |
| US11640461B2 (en) | Secure runtime for virtual machines | |
| CN109522332A (en) | Customer profile data merging method, device, equipment and readable storage medium storing program for executing | |
| CN111651424B (en) | Data processing method, device, data node and storage medium | |
| US11500738B2 (en) | Tagging application resources for snapshot capability-aware discovery | |
| CN106919713B (en) | Cluster file system and distributed file system multi-client file control method | |
| US8671075B1 (en) | Change tracking indices in virtual machines | |
| US10127270B1 (en) | Transaction processing using a key-value store | |
| US8700676B2 (en) | System and method for appending metadata to objects | |
| JP2017167811A (en) | Backup controller, backup controlling method, and program | |
| US20210279087A1 (en) | Secure runtime for virtual machines | |
| CN114297196B (en) | Metadata storage method and device, electronic equipment and storage medium | |
| CN113243008A (en) | Distributed VFS with shared page cache |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20210207 Address after: Building 9, No.1, guanpu Road, Guoxiang street, Wuzhong Economic Development Zone, Wuzhong District, Suzhou City, Jiangsu Province Applicant after: SUZHOU LANGCHAO INTELLIGENT TECHNOLOGY Co.,Ltd. Address before: Room 1601, 16 / F, 278 Xinyi Road, Zhengdong New District, Zhengzhou City, Henan Province, 450000 Applicant before: ZHENGZHOU YUNHAI INFORMATION TECHNOLOGY Co.,Ltd. |
|
| TA01 | Transfer of patent application right | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |