CN1293492C - Computer group file system uniformity dynamic inspection method - Google Patents
Computer group file system uniformity dynamic inspection method Download PDFInfo
- Publication number
- CN1293492C CN1293492C CNB2003101135525A CN200310113552A CN1293492C CN 1293492 C CN1293492 C CN 1293492C CN B2003101135525 A CNB2003101135525 A CN B2003101135525A CN 200310113552 A CN200310113552 A CN 200310113552A CN 1293492 C CN1293492 C CN 1293492C
- Authority
- CN
- China
- Prior art keywords
- file
- metadata
- meta data
- file system
- cluster
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Abstract
The present invention provides a method for dynamically checking the consistency of a cluster file system. The present invention comprises the steps that at the client end, a user application program invokes a file to cause consistency check; a metadata server checks metadata stored in the invoked file; if the metadata stored in the file is correct, the subsequent operation is continuously executed; if the metadata stored in the file is wrong, the metadata file is deleted, and error information is returned. By using dynamic check technology inside the cluster file system, the consistency of the file metadata and data is checked in real time; in this way, the continuity of a system application and the correctness and the integrity of the system are ensured, and CFS is made to have high availability; besides, the design of an external fsck tool can be simplified.
Description
Technical field
The present invention relates to a kind of management method of file system of computer group, the consistance dynamic chek method of the especially a kind of file system metadata server of a group of planes and the information of storage server same file; Belong to the computer group technical field.
Background technology
Computer group file system (Cluster File System is called for short CFS) is used for the distributed document service is provided under cluster environment, is the important component part of high-performance server.
In order to solve the handling problem of large-scale parallel IO, the distributed design that CFS adopts control and storage to disperse, metadata operation is by the meta data server centralized control, data manipulation can utilize the parallel IO that carries out of a plurality of storage servers in the group of planes to handle by a plurality of storage server parallel control like this.
File storage realizes single system mapping by meta data server in different servers.So-called single system mapping is meant that the user can sign in to the arbitrary client node in the group of planes, by using Network of Workstation software, as using a machine conveniently to operate, for file system, the document directory structure that the user saw should be identical.The fileinfo of meta data server storage All Files is used for providing unified file directory tree to the user.Meta data server is set up a metadata corresponding file with it for each file, stores the fileinfo of its corresponding storage server file.
In distributed file processing, if when the file operation of client does not have normal termination, storage server or meta data server are because unexpected and the machine of delaying just might cause the inconsistent of certain file metadata and actual storage data.Therefore, the consistance that how to guarantee file metadata in the distributed cluster file system and data is a key technical problem in the distributed cluster file system, and is anxious to be solved.
Summary of the invention
Technical matters to be solved by this invention is: propose a kind of cluster file system consistance dynamic chek method, solve the file metadata in the distributed cluster file system and the consistency problem of data.
Method of the present invention comprises the steps:
Step 1, in client, user application calls file, causes consistency check;
Step 2, the metadata of the file that called is checked meta data server judges whether the meta data file length that needs to check is zero; If be zero, this document metadata error then, if non-vanishing, then metadata is correct; Described metadata adopts the mode of once writing the generation metadata to generate, to guarantee the carrying out metadata inspection fast;
If the metadata length of step 3 file is correct, continue to carry out follow-up operation; If this meta data file is then deleted in the metadata size error, return error message.
The present invention is by coming the consistance of real-time inspection file metadata and data in the inner use of cluster file system dynamic chek technology, come system is carried out consistency check completely in conjunction with outside file tree traversal instrument simultaneously, not only guarantee the continuity of system applies but also guaranteed the correctness and the integrality of system, and made CFS have better high availability.Owing to can cause consistency check by catalogue browsing, outside fsck instrument be as long as directory tree of traversal just can be realized the consistency check to this directory tree, thereby can simplify the design of fsck instrument.
Description of drawings
Fig. 1 is the synoptic diagram of file system consistency checking method of the present invention.
Embodiment
According to the principle of CFS decentralised control, the dynamic conformance checking process of CFS need realize at the meta data server end of CFS.
Method of the present invention comprises the steps:
Step 1, in client, user application calls file, causes consistency check;
Step 2, the metadata of the file that called is checked meta data server judges whether the meta data file length that needs to check is zero; If be zero, this document metadata error then, if non-vanishing, then metadata is correct; Described metadata adopts the mode of once writing the generation meta data file to generate, to guarantee the carrying out metadata inspection fast;
If the metadata length of step 3 file is correct, continue to carry out follow-up operation; If this meta data file is then deleted in the metadata size error, return error message.Do not carry out consistency check for not invoked file.That is to say and just do consistency check when file is accessed.If adopt regular test mode,, can when checking, take too much system resource if file is many; Because for large server, user's applicating frequency and intensity are uncertain, so adopt the mode of making regular check on obviously not to be suitable for large-scale Network of Workstation.The mode that the present invention has checked when having adopted a kind of the use because the process of checking is very fast, can not used the user to impact; Guaranteed the consistance of file metadata and data again.
The user application that calls file described in the step 1 of the present invention comprises all user programs, browses a catalogue (as the ls operation) such as the user, will carry out consistency check to the All Files under this catalogue.
For the simplest mode of consistency check of file is exactly that metadata with file compares with the file data that is stored in storage server, has judged whether inconsistent information; But such method speed is too slow, if the file of once checking is too much, the overlong time that can cause the user to wait for influences the efficient of integrity service device.
Therefore, can adopt a kind of mode that generates metadata of once writing, generate metadata, to guarantee the to carry out metadata inspection fast.The metadata of file or catalogue adopts once writing of local file system to call, and makes meta data file or write success like this, or meta data file length is zero.When checking, only need judge on meta data server whether the metadata length of the file that needs inspection is zero; If be zero, then this document metadata error is deleted this metadata, has also deleted this document accordingly.
The fileinfo that metadata comprises is once write successfully the back usually and just can not changed, as the genus people of file, at the file attribute informations such as distribution of storage server; But file size is the information that possible change, when beginning to store, and the metadata of file and data consistent, but storage server uses in order to reduce the space, may compress file automatically and causes file size to change.Therefore, adopt the metadata of no length file in the meta data server, promptly the length of log file is not avoided the inconsistent of metadata length and physical length.And when file status (statfs), the visit storage server obtains file size, i.e. meta data server visit storage server obtains the length of file.
Carry out consistency check in all application programs in order to be implemented in, the present invention can adopt and realize calling of consistency check in the lookup of Virtual File System VFS, is the synoptic diagram of realizing consistency check in the lookup operation as shown in Figure 1.Like this, even the 1s operation also can cause consistency check, made things convenient for writing of outside fsck instrument.
Application program will realize file operation, must call the lookup operation of the Virtual File System VFS of core layer, CFS sends the lookup request by the CFS client to meta data server after visiting through kernel, and the meta data server file that request relates to according to lookup begins consistency check; Judge whether the meta data file length that needs to check is zero, if be zero, then deletes this meta data file, returns the error message of lookup request,, then continue to carry out normal lookup operation if meta data file length is non-vanishing.
It should be noted last that: above embodiment is the unrestricted technical scheme of the present invention in order to explanation only, although the present invention is had been described in detail with reference to the foregoing description, those of ordinary skill in the art is to be understood that: still can make amendment or be equal to replacement the present invention, and not breaking away from any modification or partial replacement of the spirit and scope of the present invention, it all should be encompassed in the middle of the claim scope of the present invention.
Claims (5)
1, a kind of cluster file system consistance dynamic chek method is characterized in that: comprise the steps:
Step 1, in client, user application calls file, causes consistency check;
Step 2, meta data server check that to the metadata of the file that called meta data server judges whether the meta data file length that needs to check is zero; If be zero, this document metadata error then, if non-vanishing, then metadata is correct; Described metadata adopts the mode of once writing the generation metadata to generate, to guarantee the carrying out metadata inspection fast;
If it is correct that step 3 is called the metadata of file, continue to carry out follow-up operation; If metadata error is then deleted this meta data file, return error message.
2, cluster file system consistance dynamic chek method according to claim 1 is characterized in that: the user application that calls file described in the described step 1 comprises that all relate to the user program of file operation.
3, cluster file system consistance dynamic chek method according to claim 1, it is characterized in that: the meta data file of storing metadata in the described meta data server, if write success, then meta data file length is non-vanishing, if write failure, then meta data file length is zero.
4, cluster file system consistance dynamic chek method according to claim 1 is characterized in that: the length information that does not comprise the described file that calls in the metadata in the described step 2.
5, according to the arbitrary described cluster file system consistance dynamic chek method of claim 1-4, it is characterized in that: can adopt the initiation that in the lookup of VFS, realizes consistency check.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CNB2003101135525A CN1293492C (en) | 2003-11-17 | 2003-11-17 | Computer group file system uniformity dynamic inspection method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CNB2003101135525A CN1293492C (en) | 2003-11-17 | 2003-11-17 | Computer group file system uniformity dynamic inspection method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN1619521A CN1619521A (en) | 2005-05-25 |
| CN1293492C true CN1293492C (en) | 2007-01-03 |
Family
ID=34759971
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CNB2003101135525A Expired - Fee Related CN1293492C (en) | 2003-11-17 | 2003-11-17 | Computer group file system uniformity dynamic inspection method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN1293492C (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN100559763C (en) * | 2007-08-28 | 2009-11-11 | 中国科学院软件研究所 | A kind of integrity check method of telecommunication network service |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104104731B (en) * | 2014-07-28 | 2018-02-02 | 浪潮(北京)电子信息产业有限公司 | A kind of method and device for safeguarding data consistency |
| CN107239470B (en) * | 2016-03-28 | 2021-05-28 | 咪咕音乐有限公司 | Data file detection method and device |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5946690A (en) * | 1996-12-17 | 1999-08-31 | Inca Technology, Inc. | NDC consistency reconnect mechanism |
| US5999976A (en) * | 1997-07-11 | 1999-12-07 | International Business Machines Corporation | Parallel file system and method with byte range API locking |
| US6385625B1 (en) * | 1998-04-28 | 2002-05-07 | Sun Microsystems, Inc. | Highly available cluster coherent filesystem |
-
2003
- 2003-11-17 CN CNB2003101135525A patent/CN1293492C/en not_active Expired - Fee Related
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5946690A (en) * | 1996-12-17 | 1999-08-31 | Inca Technology, Inc. | NDC consistency reconnect mechanism |
| US5999976A (en) * | 1997-07-11 | 1999-12-07 | International Business Machines Corporation | Parallel file system and method with byte range API locking |
| US6385625B1 (en) * | 1998-04-28 | 2002-05-07 | Sun Microsystems, Inc. | Highly available cluster coherent filesystem |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN100559763C (en) * | 2007-08-28 | 2009-11-11 | 中国科学院软件研究所 | A kind of integrity check method of telecommunication network service |
Also Published As
| Publication number | Publication date |
|---|---|
| CN1619521A (en) | 2005-05-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20220043830A1 (en) | Versioned hierarchical data structures in a distributed data store | |
| US6895413B2 (en) | System and method for performing an on-line check of a file system | |
| US6754799B2 (en) | System and method for indexing and retrieving cached objects | |
| KR20210040850A (en) | Method, apparatus, device, and storage medium for parsing document | |
| CN108491163B (en) | Big data processing method and device and storage medium | |
| US11076020B2 (en) | Dynamically transitioning the file system role of compute nodes for provisioning a storlet | |
| KR101591116B1 (en) | Systems and methods for platform-independent data file transfers | |
| CN110975293A (en) | Method, device, server and medium for establishing resource reference relation table | |
| WO2004077274A2 (en) | Multi-tier business layer architecture for information systems | |
| CN103150225B (en) | Disk full abnormity fault tolerance method of object parallel storage system based on application level agent | |
| CN111177159A (en) | Data processing system and method and data updating equipment | |
| CN114003562B (en) | Directory traversal method, device and equipment and readable storage medium | |
| Fan et al. | Gear: Enable efficient container storage and deployment with a new image format | |
| CN1293492C (en) | Computer group file system uniformity dynamic inspection method | |
| CN113032356A (en) | Cabin distributed file storage system and implementation method | |
| US20040015522A1 (en) | Apparatus, system and method of providing a stackable private write file system | |
| CN112506432A (en) | Dynamic and static separated real-time data storage and management method and device for electric power automation system | |
| Tchaye-Kondi et al. | Hadoop perfect file: a fast access container for small files with direct in disc metadata access | |
| CN109117426A (en) | Distributed networks database query method, apparatus, equipment and storage medium | |
| CN113535227A (en) | Deployment method and device of digital technology architecture | |
| CN110866165A (en) | Network video acquisition method and system | |
| Cavage et al. | Bringing arbitrary compute to authoritative data | |
| CN116991562B (en) | Data processing method and device, electronic equipment and storage medium | |
| US11757703B1 (en) | Access requests processing and failover handling across multiple fault tolerance zones | |
| CN115599790B (en) | Data storage system, data processing method, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20070103 Termination date: 20201117 |
|
| CF01 | Termination of patent right due to non-payment of annual fee |