CN111309682B - Recycle bin file cleaning method and device based on distributed file system - Google Patents
Recycle bin file cleaning method and device based on distributed file system Download PDFInfo
- Publication number
- CN111309682B CN111309682B CN202010081957.9A CN202010081957A CN111309682B CN 111309682 B CN111309682 B CN 111309682B CN 202010081957 A CN202010081957 A CN 202010081957A CN 111309682 B CN111309682 B CN 111309682B
- Authority
- CN
- China
- Prior art keywords
- file
- copy
- recycle bin
- switching
- processing module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 27
- 238000004140 cleaning Methods 0.000 title claims abstract description 21
- 238000012545 processing Methods 0.000 claims description 48
- 238000012217 deletion Methods 0.000 description 4
- 230000037430 deletion Effects 0.000 description 4
- 238000010586 diagram Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/16—File or folder operations, e.g. details of user interfaces specifically adapted to file systems
- G06F16/162—Delete operations
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/17—Details of further file system functions
- G06F16/1737—Details of further file system functions for reducing power consumption or coping with limited storage space, e.g. in mobile devices
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/182—Distributed file systems
Abstract
The invention provides a recycle bin file cleaning method based on a distributed file system, which comprises the following steps: s1: storing the file to be deleted in the recycle bin; s2: judging whether the file is on the main MDS, if so, switching to S7, and if not, switching to S3; s3: migrating the file metadata master to a master MDS, and generating a metadata copy at an original storage position; s4: judging whether the copy in the S3 is quoted, if so, switching to S5, and if not, switching to S6; s5: waiting for the referenced copy to be released; s6: deleting the original data copy; s7: and deleting the file. The method realizes automatic cleaning of the file of the recycle bin and solves the problem of occupied space of the recycle bin. Meanwhile, the application also provides a device for realizing the method.
Description
Technical Field
The invention relates to the technical field of computers, in particular to a recycle bin file cleaning method and device based on a distributed file system.
Background
In the distributed file system, the recycle bin is a very important function, and is similar to the Windows operating system, and is mainly used for storing document materials temporarily deleted by a user, and files stored in the recycle bin can be recovered.
In some application scenarios, the distributed file system needs to store massive data, so that the requirement on space utilization is very high, and if a recycle bin function is used, the fast release of disk space is an important subject.
In the prior art, when file data is really deleted, the existing mechanism completely depends on a timing task to clean metadata in a cache, so that space release is not timely.
Disclosure of Invention
The embodiment of the invention provides a recycle bin file cleaning method and device based on a distributed file system, which are used for automatically cleaning recycle bin files and solving the problem of space occupation of a recycle bin.
In order to solve the technical problem, the embodiment of the invention discloses the following technical scheme:
the embodiment of the invention provides a recycle bin file cleaning method based on a distributed file system, which comprises the following steps:
s1: storing the file to be deleted in the recycle bin;
s2: judging whether the file is on the main MDS, if so, switching to S7, and if not, switching to S3;
s3: migrating the file metadata master to a master MDS, and generating a metadata copy at an original storage position;
s4: judging whether the copy in the S3 is quoted, if so, switching to S5, and if not, switching to S6;
s5: waiting for the referenced copy to be released;
s6: deleting the original data copy;
s7: and deleting the file.
Further, the method also comprises the following steps:
s8: judging whether expired files exist in the recycle bin, if yes, switching to S9, and if not, returning to S1;
s9: judging whether the metadata of the expired file has a copy, if so, switching to S10, and if not, switching to S13;
s10: judging whether the copy in the S9 is quoted, if so, switching to S11, and if not, switching to S12;
s11: waiting for the referenced copy to be released;
s12: deleting the metadata copy of the expired file;
s13: and deleting the expired file.
Further, the expired files in step S8 include files actively deleted by performing steps S1-S7 and expired files regularly cleared by the recycle bin.
Further, in step S8, the specific method for determining whether the expired file exists in the recycle bin includes: and detecting whether the uncleaned files exist under the free directory, and if so, judging that the expired files exist in the recycle bin.
In order to implement the foregoing method, an embodiment of the present application further provides a recycle bin file cleaning apparatus based on a distributed file system, where the apparatus includes:
the first judgment module is used for judging whether the file to be deleted is on the main MDS, if so, switching to the first processing module, and if not, switching to the second judgment module;
the first processing module is used for migrating the metadata master of the file to be deleted to the master MDS and generating a metadata copy in an original storage position;
the second judgment module is used for judging whether the metadata copy in the first processing module is quoted, if so, the copy is released to be transferred to the second processing module, and if not, the copy is directly transferred to the second processing module;
the second processing module is used for deleting the original data copy;
and the third processing module is used for deleting the file to be deleted.
Further, the device further comprises:
the third judging module is used for judging whether the expired files exist in the recycle bin or not, if yes, the fourth judging module is switched to, and if not, the fourth judging module returns to the first judging module;
the fourth judging module is used for judging whether the metadata of the expired file has a copy, if so, switching to the fifth judging module, and if not, switching to the fifth processing module;
the fifth judging module is used for judging whether the copy identified by the fourth judging module is quoted, if so, the copy is released and transferred to the fourth processing module, and if not, the copy is directly transferred to the fourth processing module;
the fourth processing module deletes the metadata copy of the expired file;
and the fifth processing module deletes the expired files.
The beneficial effects of the above technical scheme are that:
compared with the technical means of passively waiting for the timing cleaning of the recycle bin in the prior art, the method has the advantages that the position of the file to be deleted is actively judged, the file to be deleted is cleaned in a direct deleting or gradually deleting mode, when the file to be deleted needs to be deleted gradually, one duplicate file is firstly established, the citation of other clients is avoided, and then the duplicate and metadata are deleted after the release of the duplicate, and a part of space of the recycle bin is released.
Similarly, for expired files, the same processing strategy is adopted, the copies and the metadata are all deleted, the storage space of the recycle bin is completely released, and the occupation of the storage space is avoided.
Drawings
In order to illustrate the embodiments of the present invention or the technical solutions in the prior art more clearly, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious for a person skilled in the art to obtain other drawings without creative efforts.
FIG. 1 is a flow chart of a method provided by one embodiment of the present invention;
FIG. 2 is a schematic diagram of an apparatus provided in accordance with an embodiment of the present invention;
FIG. 3 is a flow chart of a method provided by another embodiment of the present invention;
fig. 4 is a schematic diagram of an apparatus according to another embodiment of the present invention.
Detailed Description
In order to make those skilled in the art better understand the technical solution of the present invention, the technical solution in the embodiment of the present invention will be clearly and completely described below with reference to the drawings in the embodiment of the present invention, and it is obvious that the described embodiment is only a part of the embodiment of the present invention, and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Example 1
Referring to fig. 1, as an embodiment that can be implemented by the present invention, there is provided a recycle bin file cleaning method based on a distributed file system, the method including the following steps:
1) and storing the files to be deleted in the recycle bin, wherein the step is realized on the premise that a certain person deletes a certain file to the recycle bin.
2) Judging whether the file to be deleted is on the main MDS, wherein the judgment principle of the step is as follows: the recycle bin is typically deployed on the primary MDS, mds.0. Therefore, only when the file to be deleted is on the main MDS, the recycle bin can execute a normal cleaning process to delete the file to be deleted completely.
3) And when the file to be deleted is not found on the main MDS, the file metadata master is migrated to the main MDS, and a metadata copy is generated on the original MDS.
4) Checking whether the metadata copy is being referred, and if not, cleaning the metadata copy; if the file is being quoted, the metadata copy cannot be cleaned, otherwise normal file reading and writing of the client is influenced, and the metadata copy can be cleaned only after the file is released by the client.
5) When the copy is removed, the metadata of the file to be deleted can be deleted.
In order to implement the foregoing method, embodiment 1 further provides a recycle bin file cleaning apparatus based on a distributed file system, where the apparatus includes:
and the first judgment module is used for judging whether the file to be deleted is on the main MDS, if so, switching to the first processing module, and if not, switching to the second judgment module.
And the first processing module is used for migrating the metadata master of the file to be deleted to the master MDS and generating a metadata copy in the original storage position.
And the second judgment module is used for judging whether the metadata copy in the first processing module is quoted, if so, the copy is released to be transferred into the second processing module, and if not, the copy is directly transferred into the second processing module.
The second processing module is used for deleting the original data copy, and after the deletion is completed, the original data copy is transferred to the third processing module, and the file to be deleted is deleted by the third processing module.
Example 2
Unlike embodiment 1, in order to ensure that a file is completely deleted, referring to fig. 3, the method includes the following implementation processes:
1) and storing the files to be deleted in the recycle bin, wherein the step is realized on the premise that a certain person deletes a certain file to the recycle bin.
2) Judging whether the file to be deleted is on the main MDS, wherein the judgment principle of the step is as follows: the recycle bin is typically deployed on the primary MDS, mds.0. Therefore, only when the file to be deleted is on the main MDS, the recycle bin can execute a normal cleaning process to delete the file to be deleted completely.
3) And when the file to be deleted is not found on the main MDS, the file metadata master is migrated to the main MDS, and a metadata copy is generated on the original MDS.
4) Checking whether the metadata copy is being referred, and if not, cleaning the metadata copy; if the file is being quoted, the metadata copy cannot be cleaned, otherwise normal file reading and writing of the client is influenced, and the metadata copy can be cleaned only after the file is released by the client.
5) When the copy is removed, the metadata of the file to be deleted can be deleted.
After the above process, the metadata is deleted to a free directory of the MDS, that is, a prompt is returned to indicate that the deletion is successful, and at this time, the metadata and the data of the file are not really deleted in the disk, so that the following operations are also required to be added:
6) and detecting whether an uncleaned file exists in the free directory, if so, judging that an expired file exists in the recycle bin, wherein the expired file not only comprises the file actively deleted by the user, but also comprises the expired file regularly cleaned by the recycle bin.
7) For the detected expired file, whether a copy exists in metadata of the expired file needs to be judged, when the copy exists in the metadata, whether the copy is quoted is checked, and when the copy does not exist in the metadata, the metadata copy is deleted from other MDS, and then deletion is continuously executed.
Also, in order to implement the method of embodiment 2, there is provided a recycle bin file cleaning apparatus based on a distributed file system, where the apparatus includes:
and the first judgment module is used for judging whether the file to be deleted is on the main MDS, if so, switching to the first processing module, and if not, switching to the second judgment module.
And the first processing module is used for migrating the metadata master of the file to be deleted to the master MDS and generating a metadata copy in the original storage position.
And the second judgment module is used for judging whether the metadata copy in the first processing module is quoted, if so, the copy is released to be transferred into the second processing module, and if not, the copy is directly transferred into the second processing module.
The second processing module is used for deleting the original data copy, and after the deletion is completed, the original data copy is transferred to the third processing module, and the file to be deleted is deleted by the third processing module.
The third judging module is used for judging whether the expired files exist in the recycle bin or not, if yes, the fourth judging module is switched to, and if not, the first judging module is returned.
The fourth judging module is used for judging whether the metadata of the expired file has a copy, if so, switching to the fifth judging module, and if not, switching to the fifth processing module;
the fifth judging module is used for judging whether the copy identified by the fourth judging module is quoted, if so, the copy is released and transferred to the fourth processing module, and if not, the copy is directly transferred to the fourth processing module;
and the fourth processing module deletes the metadata copy of the expired file, and transfers the metadata copy to the fifth processing module after the metadata copy is deleted, and the expired file is deleted by the fifth processing module.
The foregoing are merely exemplary embodiments of the present invention, which enable those skilled in the art to understand or practice the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (6)
1. The recycle bin file cleaning method based on the distributed file system is characterized by comprising the following steps of:
s1: storing the file to be deleted in the recycle bin;
s2: judging whether the file is on the main MDS, if so, switching to S7, and if not, switching to S3;
s3: migrating the file metadata master to a master MDS, and generating a metadata copy at an original storage position;
s4: judging whether the copy in the S3 is quoted, if so, switching to S5, and if not, switching to S6;
s5: waiting for the referenced copy to be released;
s6: deleting the original data copy;
s7: deleting the file;
s8: judging whether expired files exist in the recycle bin, if yes, switching to S9, and if not, returning to S1;
s9: judging whether the metadata of the expired file has a copy, if so, switching to S10, and if not, switching to S13;
s10: judging whether the copy in the S9 is quoted, if so, switching to S11, and if not, switching to S12;
s11: waiting for the referenced copy to be released;
s12: deleting the metadata copy of the expired file;
s13: and deleting the expired file.
2. The distributed file system-based recycle bin file cleaning method according to claim 1, wherein the expired files in the step S8 include files actively deleted by executing the steps S1-S7 and expired files regularly cleaned by the recycle bin.
3. The method for clearing recycle bin files based on the distributed file system of claim 1, wherein in step S8, the specific method for determining whether the expired files exist in the recycle bin is as follows: and detecting whether the uncleaned files exist under the free directory, and if so, judging that the expired files exist in the recycle bin.
4. The recycle bin file cleaning device based on the distributed file system is characterized by comprising the following components:
the first judgment module is used for judging whether the file to be deleted is on the main MDS, if so, switching to the first processing module, and if not, switching to the second judgment module;
the first processing module is used for migrating the metadata master of the file to be deleted to the master MDS and generating a metadata copy in an original storage position;
the second judgment module is used for judging whether the metadata copy in the first processing module is quoted, if so, the copy is released to be transferred to the second processing module, and if not, the copy is directly transferred to the second processing module;
the second processing module is used for deleting the original data copy;
and the third processing module is used for deleting the file to be deleted.
5. The distributed file system based recycle bin file clean-up apparatus of claim 4 further comprising:
the third judging module is used for judging whether the expired files exist in the recycle bin or not, if yes, the fourth judging module is switched to, and if not, the fourth judging module returns to the first judging module;
the fourth judging module is used for judging whether the metadata of the expired file has a copy, if so, switching to the fifth judging module, and if not, switching to the fifth processing module;
the fifth judging module is used for judging whether the copy identified by the fourth judging module is quoted, if so, the copy is released and transferred to the fourth processing module, and if not, the copy is directly transferred to the fourth processing module;
the fourth processing module deletes the metadata copy of the expired file;
and the fifth processing module deletes the expired files.
6. The file cleaning device of the recycle bin based on the distributed file system as claimed in claim 5, wherein the expired files comprise files actively deleted by the third processing module and expired files periodically cleaned by the recycle bin.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010081957.9A CN111309682B (en) | 2020-02-06 | 2020-02-06 | Recycle bin file cleaning method and device based on distributed file system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010081957.9A CN111309682B (en) | 2020-02-06 | 2020-02-06 | Recycle bin file cleaning method and device based on distributed file system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111309682A CN111309682A (en) | 2020-06-19 |
| CN111309682B true CN111309682B (en) | 2022-04-19 |
Family
ID=71159140
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010081957.9A Active CN111309682B (en) | 2020-02-06 | 2020-02-06 | Recycle bin file cleaning method and device based on distributed file system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111309682B (en) |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9390109B2 (en) * | 2012-12-21 | 2016-07-12 | Commvault Systems, Inc. | Systems and methods to detect deleted files |
| CN106354884B (en) * | 2016-10-09 | 2020-02-07 | 苏州浪潮智能科技有限公司 | Data object deleting method and device based on distributed file system |
| CN106407465A (en) * | 2016-10-14 | 2017-02-15 | 郑州云海信息技术有限公司 | File deletion method and system based on distributed file system |
| CN108197270B (en) * | 2018-01-04 | 2021-05-28 | 中科边缘智慧信息科技（苏州）有限公司 | Distributed file system data recovery method |
| CN110413577A (en) * | 2019-06-27 | 2019-11-05 | 无锡华云数据技术服务有限公司 | Data-erasure method, device, electronic equipment and computer readable storage medium |
-
2020
- 2020-02-06 CN CN202010081957.9A patent/CN111309682B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN111309682A (en) | 2020-06-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20220413706A1 (en) | Data Storage Method, Apparatus and Storage System | |
| US8250033B1 (en) | Replication of a data set using differential snapshots | |
| US8856469B2 (en) | Apparatus and method for logging optimization using non-volatile memory | |
| CN102945278B (en) | A kind of method and apparatus of data-base recording redo log | |
| JP4304194B2 (en) | File system management apparatus and method | |
| CN107729558B (en) | Method, system and device for defragmenting file system and computer storage medium | |
| JP2003177947A5 (en) | ||
| CN109614377A (en) | File delet method, device, equipment and the storage medium of distributed file system | |
| CN102339321A (en) | Network file system with version control and method using same | |
| CN107391774A (en) | The rubbish recovering method of JFS based on data de-duplication | |
| CN107066349A (en) | A kind of method and system of cluster file system data protection | |
| CN110612516A (en) | Improving backup performance after backup failure | |
| CN105787037A (en) | Repeated data deleting method and device | |
| CN102567323A (en) | Application program file moving method and application program file moving system | |
| CN108446187B (en) | Data backup method and data recovery method | |
| CN106648457A (en) | Method of updating reverse mapping metadata and device | |
| WO2013170702A1 (en) | File processing method based on snapshot and solid state disk with snapshot function | |
| CN111177105B (en) | Mass file writing method, device, system and medium of distributed file system | |
| CN110968417A (en) | Method, apparatus, system and computer program product for managing storage units | |
| CN111309682B (en) | Recycle bin file cleaning method and device based on distributed file system | |
| CN105740098A (en) | Determination method and system for stale data among backup data | |
| WO2023241528A1 (en) | Data processing method and apparatus | |
| CN104991934A (en) | Directory deletion method and apparatus | |
| CN110196788B (en) | Data reading method, device and system and storage medium | |
| CN111625500B (en) | File snapshot method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |