CN115658619A

CN115658619A - Process processing method and device, processor and electronic equipment

Info

Publication number: CN115658619A
Application number: CN202211461444.6A
Authority: CN
Inventors: 赵岩; 刘名欣; 瞿天善; 张旭明; 王豪迈; 胥昕
Original assignee: Beijing Xingchen Tianhe Technology Co ltd
Current assignee: Beijing Xingchen Tianhe Technology Co ltd
Priority date: 2022-11-17
Filing date: 2022-11-17
Publication date: 2023-01-31
Anticipated expiration: 2042-11-17
Also published as: CN115658619B

Abstract

The application discloses a process processing method, a process processing device, a processor and electronic equipment. The method comprises the following steps: before the execution of the deleting operation of the target directory is completed, detecting whether the process of the deleting operation is terminated; under the condition that the process is terminated, acquiring cursor information of a target directory; determining a knee point set of the process from the cursor information; determining the directory entry deleted last before the process is terminated according to the inflection point set; and determining the current directory item to be deleted after the process is terminated according to the last deleted directory item, recovering the process according to the current directory item to be deleted, and continuously executing the deletion operation in the process. By the method and the device, the problem that in the prior art, when the process for deleting the file is recovered, the file which is deleted before the process is terminated needs to be traversed again, so that the file deleting efficiency is low is solved.

Description

Process processing method and device, processor and electronic equipment

Technical Field

The application relates to the field of file recovery, in particular to a process processing method, a process processing device, a processor and electronic equipment.

Background

In the prior art, when directories in a file system are deleted, a client/server architecture is used, a request for directory traversal is sent to a server through the client, after the server traverses the directories, traversal results are returned to the client in batches, finally, the client sends a deletion request for each file in the directories to the server one by one, and the server deletes the files in the directories one by one.

However, the operation flow of deleting the directory based on the client/server architecture has a defect that multiple network interactions between the client and the server are required, and the efficiency of deleting the directory by the server is low because the client sends the files to the server one by one for deletion. When the directory to be deleted contains a large number of sub-directories and files, it takes a long time to delete the entire directory.

In addition, if the operation process for deleting the directory is accidentally exited by mistake in the middle, the directory deleting operation is terminated. When the directory terminating the delete operation subsequently re-executes the delete operation, it may still traverse the intermediate state residual records of the partial directory entries in the database that have been deleted before being terminated (but the final traversal result does not list the deleted directory entries), and it is time consuming to traverse the intermediate state residual records of the deleted directory entries, so that the overall efficiency of deleting the directory is very low.

Aiming at the problem that the file deleting efficiency is low because the deleted file is required to be traversed again before the process is terminated when the process for deleting the file is recovered in the related technology, an effective solution is not provided at present.

Disclosure of Invention

The application mainly aims to provide a process processing method, a process processing device, a processor and electronic equipment, so as to solve the problem that file deletion efficiency is low due to the fact that files which are deleted before a process is terminated need to be traversed again when the file deletion process is recovered in the related art.

To achieve the above object, according to one aspect of the present application, a processing method of a process is provided. The method comprises the following steps: before the execution of the deleting operation of the target directory is completed, detecting whether the process of the deleting operation is terminated or not, wherein the target directory comprises a plurality of directory entries; under the condition that the process is terminated, acquiring cursor information of the target directory, wherein the cursor information is used for recording the deletion progress of directory items in the target directory; determining an inflection point set of the process from the cursor information, wherein the inflection point set comprises one or more inflection points, and the inflection points are used for marking the deleted directory item; determining the directory entry deleted last before the process is terminated according to the inflection point set; and determining the current directory item to be deleted after the process is terminated according to the last deleted directory item, recovering the process according to the current directory item to be deleted, and continuously executing the deletion operation in the process.

Optionally, determining the directory entry deleted last before the process terminates according to the inflection point set includes: acquiring a current processing level in the cursor information, wherein the current processing level is a level where a finally deleted directory entry is located; determining an inflection point in the inflection point set, which is the same as the hierarchy number of the current processing hierarchy, as a target inflection point, wherein the hierarchy number is used for marking the hierarchy of each directory item in the target directory; and determining the directory entry corresponding to the target inflection point as the directory entry deleted last.

Optionally, determining, according to the last deleted directory entry, a current directory entry to be deleted after the process is terminated includes: detecting whether a same-layer directory item exists in a target parent directory of the directory item corresponding to the target inflection point, wherein the same-layer directory item is a directory item in the target parent directory except the directory item corresponding to the target inflection point; under the condition that at least one same-layer directory entry exists in the target parent directory, determining the same-layer directory entry traversed first in the at least one same-layer directory entry as a current directory entry to be deleted; and under the condition that the same-layer directory entry does not exist in the target parent directory, determining the target parent directory as the current directory entry to be deleted.

Optionally, restoring the process according to the current directory entry to be deleted, and continuing to execute the deletion operation in the process includes: judging whether other inflection points exist in the inflection point set, wherein the other inflection points are inflection points in the inflection point set except the target inflection point; under the condition that other inflection points do not exist in the inflection point set, traversing directory items in the target directory by taking the current directory item to be deleted as a starting point to obtain at least one directory item to be processed, wherein the number of the directory items to be processed is less than a preset threshold value; under the condition that other inflection points exist in the inflection point set, skipping over directory items in subdirectories traversing other inflection points, and traversing directory items in the target directory by taking the current directory item to be deleted as a starting point to obtain at least one directory item to be processed; deleting at least one directory entry to be processed in sequence according to a target sequence; and after the current directory item to be deleted is deleted, obtaining the current deleted directory item, determining the next traversed directory item as the updated current directory item to be deleted, and updating the inflection point set.

Optionally, sequentially deleting at least one to-be-processed directory entry according to the target order includes: under the condition that the number of the directory items to be processed is larger than or equal to the preset number, packing the directory items to be processed and the cursor information in the preset number into a transaction, and persisting the transaction in a metadata system; and the metadata system controls the thread to execute the operation of deleting the directory entries in the transaction in sequence according to the target sequence.

Optionally, updating the set of inflection points comprises: deleting the inflection point corresponding to the directory entry deleted last from the inflection point set; and determining the current deleted directory entry as an updated target inflection point, and adding the updated target inflection point to the inflection point set.

Optionally, after updating the set of inflection points, the method further comprises: judging whether the updated current directory item to be deleted is the same as the hierarchy number of the current deleted directory item; under the condition that the hierarchy number of the updated current directory item to be deleted is different from the hierarchy number of the current deleted directory item, updating the current processing hierarchy in the cursor information into a hierarchy corresponding to the hierarchy number of the updated current directory item to be deleted; and under the condition that the updated current directory entry to be deleted is the same as the hierarchy number of the current deleted directory entry, keeping the current processing hierarchy in the cursor information unchanged.

To achieve the above object, according to another aspect of the present application, there is provided a processing apparatus of a process. The device includes: the device comprises a detection unit, a processing unit and a processing unit, wherein the detection unit is used for detecting whether the process of the deletion operation is terminated before the execution of the deletion operation of a target directory is completed, and the target directory comprises a plurality of directory entries; the system comprises an acquisition unit, a processing unit and a processing unit, wherein the acquisition unit is used for acquiring cursor information of a target directory under the condition that a process is terminated, and the cursor information is used for recording the deletion progress of a directory item in the target directory; a first determining unit, configured to determine a set of inflection points of a process from the cursor information, where the set of inflection points includes one or more inflection points, and the inflection points are used to mark a currently deleted directory entry; the second determining unit is used for determining the directory item deleted last before the process is terminated according to the inflection point set; and the recovery unit is used for determining the current directory item to be deleted after the process is terminated according to the last deleted directory item, recovering the process according to the current directory item to be deleted, and continuously executing the deletion operation in the process.

Through the application, the following steps are adopted: before the execution of the deleting operation of the target directory is completed, detecting whether the process of the deleting operation is terminated or not, wherein the target directory comprises a plurality of directory entries; under the condition that the process is terminated, acquiring cursor information of the target directory, wherein the cursor information is used for recording the deletion progress of directory items in the target directory; determining a inflection point set of a process from the cursor information, wherein the inflection point set comprises one or more inflection points, and the inflection points are used for marking the deleted directory items; determining the directory entry deleted last before the process is terminated according to the inflection point set; the method comprises the steps of determining a current directory item to be deleted after a process is terminated according to a last deleted directory item, recovering the process according to the current directory item to be deleted, and continuously executing deletion operation in the process, so that the problem of low file deletion efficiency caused by the fact that the deleted file is required to traverse the file before the process is terminated when the file deleting process is recovered in the related art is solved. By recording the cursor information of the deletion process and determining the current directory item to be deleted after the process is terminated according to the inflection point in the cursor information, the file which is traversed before the process is terminated is prevented from being traversed again, and the effect of improving the file deletion efficiency is achieved.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this application, illustrate embodiments of the application and, together with the description, serve to explain the application and are not intended to limit the application. In the drawings:

FIG. 1 is a flow chart of a processing method of a process provided according to an embodiment of the present application;

FIG. 2 is a schematic diagram of a hierarchical structure of a directory set provided according to an embodiment of the present application;

FIG. 3 is a schematic flow chart illustrating traversal of a target directory according to an embodiment of the present application;

fig. 4 is a schematic diagram of a processing device of a process provided according to an embodiment of the present application.

Detailed Description

It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict. The present application will be described in detail below with reference to the accompanying drawings in conjunction with embodiments.

In order to make the technical solutions of the present application better understood by those skilled in the art, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only some embodiments of the present application, and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

It should be noted that the terms "first," "second," and the like in the description and claims of this application and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It should be understood that the data so used may be interchanged under appropriate circumstances such that embodiments of the application described herein may be used. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.

It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for presentation, analyzed data, etc.) referred to in the present disclosure are information and data authorized by the user or sufficiently authorized by each party.

For convenience of description, some terms or expressions referred to in the embodiments of the present application are explained below:

directory structure: the file system externally shows a file organization mode and a tree structure;

and (3) directory traversal: scanning all subdirectories and files under the specified directory according to a certain sequence;

and (3) breakpoint connection: after the operation is terminated, the same operation triggered subsequently can be continuously executed as before the preamble operation is not terminated, without generating additional influence, and the execution result is completely the same as the preamble operation.

The present invention is described below with reference to preferred implementation steps, and fig. 1 is a flowchart of a processing method of a process provided according to an embodiment of the present application, and as shown in fig. 1, the method includes the following steps:

step S101, before the deletion operation of the target directory is completed, it is detected whether the process of the deletion operation is terminated, where the target directory includes a plurality of directory entries.

Specifically, the target directory may be a designated directory in the file system on which a delete operation is performed, in a case where there are many directory entries in the target directory, it takes a period of time for an operating process that deletes the target directory, and in a case where the delete process is performed, there is a case where the process is terminated, for example, a master metadata system in a metadata cluster that controls the file system fails, so that the process is terminated unexpectedly.

It should be noted that the deletion operation of the target directory is that the client sends a request for deleting the target directory to the metadata system by setting a standard file system interface command of the extended attribute of the directory to be deleted, the metadata system executes the deletion operation after receiving the request, and actively generates a record for deleting the target directory, persists the record in the metadata system, and returns a persistency result to the client after the persistence is completed. Meanwhile, when the target directory is marked to be deleted, the directory, the subdirectories and the files thereof are still visible to the outside, but the files or the subdirectories are not allowed to be created and deleted in the directory, the files or the subdirectories under the directory are not allowed to be modified, and the operations of setting, deleting and the like on the directory are not allowed.

After the recording and the persistence are completed, a thread which is specially responsible for processing the internal request in the background in the metadata system scans the persistent request, then traverses the target directory to be deleted, and deletes the files and subdirectories under the directory in a batch mode at the same time until all the files and subdirectories are deleted, and then deletes the persistent target directory deletion request.

Step S102, under the condition that the process is terminated, cursor information of the target directory is obtained, wherein the cursor information is used for recording the deletion progress of the directory items in the target directory.

Specifically, the cursor information may be information for recording a deletion progress of the target directory, and the cursor information may include an inflection point set and a current processing level, where the inflection point set records inflection points (deleted directory entries) in each layer of directory structure in a traversal process of the target directory from top to bottom by using a chain recording method, and the current processing level is a level where a last deleted directory entry in the process of deleting the target directory is located.

It should be noted that the types of directory traversal include depth traversal and breadth traversal, where the breadth traversal is to traverse the next layer of directory after completing the traversal of a single layer of directory, and the breadth traversal has the advantages of continuous traversal of a single directory, and fully playing the performance of local cache of a database.

In the process of traversing the single-layer directory, once a first subdirectory is found, the first subdirectory is entered, and the next layer starts to be traversed; and after the traversal is finished at the last layer of directory, tracing back upwards and continuing the traversal. The advantage of deep traversal is that the number of temporary directories is small, at most the temporary maximum directory depth is multiplied by the number of directories that are traversed at a single time, and at the same time, because the maximum directory depth of the file system is controlled, the consumed memory resources are not excessive and controllable. The drawback is that the single directory traversal is not continuous and may have some impact on performance. Based on the consideration of the controllability of memory resource consumption, the method selects and realizes the process processing method based on the depth traversal algorithm.

And step S103, determining an inflection point set of the process from the cursor information, wherein the inflection point set comprises one or more inflection points, and the inflection points are used for marking the deleted directory entry.

Specifically, there may be only one inflection point in the set of inflection points, or there may be multiple inflection points, where a fileid of an upper parent directory, a name of a directory entry corresponding to the inflection point, a file name of the directory entry corresponding to the inflection point, and a type of the directory entry corresponding to the inflection point are recorded in the inflection point, such as a normal file, a directory, or a soft link. There is only one inflection point at each level, and each inflection point marks the directory entry deleted at the last level where the inflection point is located. For example, fig. 2 is a schematic diagram of a hierarchical structure of a directory set provided according to an embodiment of the present application, and as shown in fig. 2, the target directory includes 4 layers,/path/to/dir is the target directory, which is located at layer 0, and from top to bottom are layer 1, layer 2, and layer 3. If the files deleted last by the process are both deleted files file _3 \, dir \1 \0and file _2 \2, at this time, there are 3 inflection points in the inflection point set, the inflection point at layer 1 is dir _1 \0, the inflection point at layer 2 is file _2 \2, and the inflection point at layer 3 is file _3 \3. If the file deleted last by the process is file _3_0, only layer 3 in the inflection point set has one inflection point, i.e., file _3_0.

And step S104, determining the directory item deleted last before the process is terminated according to the inflection point set.

Specifically, by determining a target inflection point in the inflection point set, the directory entry corresponding to the target inflection point is the directory entry deleted last before the process is terminated. The target inflection point is an inflection point of the current processing level, namely, an inflection point corresponding to the directory entry deleted last before the process is terminated.

And step S105, determining the current directory item to be deleted after the process is terminated according to the last deleted directory item, recovering the process according to the current directory item to be deleted, and continuously executing the deletion operation in the process.

Specifically, after determining the last deleted directory entry before the process is terminated, the next directory entry to be deleted after the process is resumed, that is, the directory entry to be deleted currently, may be determined according to the last deleted directory entry. And taking the current directory entry to be deleted as the deletion starting directory entry after the process is recovered, and continuously executing the deletion operation of the process.

It should be noted that, after the deletion process is resumed until all directory entries in the target directory are deleted, the target directory itself is deleted, and the request for deleting the target directory and the corresponding cursor information are deleted from the metadata system.

According to the process processing method provided by the embodiment of the application, before the execution of the deleting operation of the target directory is completed, whether the process of the deleting operation is terminated is detected, wherein the target directory comprises a plurality of directory entries; under the condition that the process is terminated, acquiring cursor information of the target directory, wherein the cursor information is used for recording the deletion progress of the directory entries in the target directory; determining an inflection point set of the process from the cursor information, wherein the inflection point set comprises one or more inflection points, and the inflection points are used for marking the deleted directory item; determining the directory entry deleted last before the process is terminated according to the inflection point set; the method comprises the steps of determining a current directory item to be deleted after a process is terminated according to a last deleted directory item, recovering the process according to the current directory item to be deleted, and continuously executing deletion operation in the process, so that the problem of low file deletion efficiency caused by the fact that the deleted file is required to traverse the file before the process is terminated when the file deleting process is recovered in the related art is solved. By recording the cursor information of the deletion process and determining the current directory item to be deleted after the process is terminated according to the inflection point in the cursor information, the deleted file before the process is terminated is prevented from being traversed again, and the effect of improving the file deletion efficiency is achieved.

Optionally, in the processing method of a process provided in this embodiment of the present application, determining, according to the inflection point set, a directory entry to be deleted last before the process is terminated includes: acquiring a current processing level in the cursor information, wherein the current processing level is a level where a finally deleted directory entry is located; determining an inflection point in the inflection point set, which is the same as the hierarchy number of the current processing hierarchy, as a target inflection point, wherein the hierarchy number is used for marking the hierarchy of each directory item in the target directory; and determining the directory entry corresponding to the target inflection point as the directory entry deleted last.

Specifically, the cursor information records the hierarchy where the currently deleted directory entry is located in real time, that is, the current processing hierarchy, and the currently deleted directory entry is updated to the target inflection point. The current processing level is also the level of the target inflection point. And determining the directory entry corresponding to the target inflection point as the directory entry deleted last. And finding the breakpoint when the process is terminated by determining the directory entry deleted last, and further directly recovering the deletion operation of the process from the breakpoint.

It should be noted that, according to the algorithm of depth traversal, the directory entries of the top-level directory are traversed from the top-level directory, and the number of the traversed directory entries in each search does not exceed a certain threshold, so as to reduce the memory resources consumed by temporarily storing the information of the directory entries in one search. In the process, not all directory entries in the target directory are directly traversed, but a part of directory entries are traversed, then a part of directory entries are deleted, and then a part of directory entries are traversed, a directory set dirs _ set (a memory structure does not need to be distributed and persisted among metadata systems) which is traversed by the process is recorded from top to bottom according to the target directory hierarchy, wherein the directory set comprises the hierarchy, the name and the file name of each directory entry, the file name of a parent directory, the information (stored according to the traversed sequence) of each currently traversed directory entry, the information of the starting position of the next traversal and the like.

As shown in fig. 2, the directory file is prefixed with "dir _", and the normal file is prefixed with "file _". The/path/to/dir is the target directory. The rest of the directory entry name is the index of the directory hierarchy in which the directory entry is located (numbered from 0, with level 0 representing the target directory), and the index of the location within the directory hierarchy in which the directory entry is located (numbered from 0). For example, dir _1_0 indicates a hierarchy number of 1, the 0 th directory entry in layer 1, which is a directory, and file _3_2 indicates a hierarchy number of 3, the 2 nd directory entry in layer 3, which is a file.

Optionally, in the method for processing a process provided in the embodiment of the present application, determining, according to the last deleted directory entry, the current directory entry to be deleted after the termination of the process includes: detecting whether a same-layer directory item exists in a target parent directory of the directory item corresponding to the target inflection point, wherein the same-layer directory item is a directory item in the target parent directory except the directory item corresponding to the target inflection point; under the condition that at least one same-layer directory entry exists in the target parent directory, determining the same-layer directory entry traversed first in the at least one same-layer directory entry as a current directory entry to be deleted; and under the condition that the same-layer directory entry does not exist in the target parent directory, determining the target parent directory as the current directory entry to be deleted.

Specifically, the same-level directory entry refers to a directory entry located under the target parent directory with the same hierarchical number as the target inflection point, for example, if the directory entry corresponding to the target inflection point is file _3_0, then file _3_1 and file _3_2 are same-level directory entries. Under the condition of deleting one target item every time, if the directory item to be deleted is file _3_0, the directory item to be deleted is file _3_1. If the directory entries corresponding to the target inflection point are file _3_2, and file _3 _0and file _3 _1are all deleted, and the target inflection point does not have the same-layer directory entry at this time, the parent directory dir _2 _0of file _3 _2is determined as the directory entry to be currently deleted. And recovering the deleting process with the current directory item to be deleted as the starting point by determining the current directory item to be deleted.

It should be noted that, each time the preset number of directory entries are deleted, the number of transactions that can be concurrently processed by the process cannot exceed a certain threshold, and if the number of transactions exceeds the certain threshold, the subsequent traversal deletion operation cannot be continued until the pre-preamble transaction is completed. Therefore, the total number of the concurrent transactions needs to be recorded in the memory, and when a new transaction is sent, the total number of the transactions is added with 1; when the preorder transaction is completed, subtracting 1 from the total number of the transactions; before traversing the deleted file, judging whether the total number of the current transactions exceeds a threshold value of the transactions. For example, an X process that deletes the target directory can process 3 transactions simultaneously, each transaction can delete two directory entries simultaneously.

Optionally, in the method for processing a process provided in the embodiment of the present application, the recovering the process according to the current directory entry to be deleted, and continuing to perform the deletion operation in the process includes: judging whether other inflection points exist in the inflection point set, wherein the other inflection points are inflection points in the inflection point set except the target inflection point; under the condition that other inflection points do not exist in the inflection point set, traversing directory items in the target directory by taking the current directory item to be deleted as a starting point to obtain at least one directory item to be processed, wherein the number of the directory items to be processed is less than a preset threshold value; under the condition that other inflection points exist in the inflection point set, skipping over directory items in subdirectories traversing other inflection points, and traversing directory items in the target directory by taking the current directory item to be deleted as a starting point to obtain at least one directory item to be processed; deleting at least one directory entry to be processed in sequence according to a target sequence; and after the current directory item to be deleted is deleted, obtaining the current deleted directory item, determining the next traversed directory item as the updated current directory item to be deleted, and updating the inflection point set.

Specifically, there may be one or more inflection points in the set of inflection points, and if there is only one inflection point, it indicates that there is no deleted directory entry in other layers except the current processing layer, and the deep traversal continues directly from the directory entry corresponding to the target inflection point. If other inflection points exist in the inflection point set, it is indicated that other layers except the current processing layer have deleted directory items, in order to avoid traversing the deleted directory items, the deleted directory items of other layers are marked by other inflection points, when the process is recovered, and when the file to be deleted is traversed after the process is recovered, for example, when the layer 1 is traversed to one inflection point, it is indicated that all sub-files under the inflection point and the file corresponding to the inflection point are deleted, so that the file corresponding to the inflection point and the sub-files thereof are skipped to be traversed, the inflection point of each layer is confirmed layer by layer, and when the current processing level is traversed, the current directory item to be deleted is taken as a starting point, and the traversal is continued in a deep traversal mode. The directory entries traversed in the current processing layer may be multiple, the directory entries are determined as directory entries to be processed, the directory entries to be processed are deleted in sequence according to the sequence of traversal, and the directory entries to be processed are provided with preset thresholds, so that the traversing efficiency is prevented from being influenced by too much cache. And simultaneously updating the directory entry to be deleted and the inflection point set. By taking the current directory item to be deleted as a starting point, the directory item recovery process corresponding to other inflection points is skipped over and traversed, so that the directory item which is deleted before the repeated traversal process is terminated is avoided, and the deletion efficiency is improved.

For example, as shown in FIG. 2, the last deleted directory entry before the process terminated is file _3_1, the target directory is traversed in depth after the process is resumed, with no inflection points set, starting from/path/to/dir, level 1 is traversed to 2 directory entries dir _1 _0and dir _1_1, then dir _1_0, level 2 is traversed to two directory entries dir _2 _0and file _2_1, then dir _2 _0is traversed because file _3 _0and file _3 _1have been deleted, finally 1 directory entry file _3 _2is traversed, and then the resume process continues to delete file _3 u 2.

In the case of the inflection point being set, the target inflection point is file _3_1, the target directory is subjected to a depth traversal after the process is resumed, the traversal is started from/path/to/dir, the 1 st layer is traversed to 2 directory entries dir _1 _0and dir _1_1, then dir _1_0, the 2 nd layer is traversed to two directory entries dir _2 _0and file _2_1, then dir _2 _0is traversed, the presence of the target inflection point file _3 _1is detected at the 3 rd layer, the traversal is started at the 3 rd layer with file _3 _1as a starting point, and the directory entry traversed to the 3 rd layer is file _3_2. During traversal, the deleted file _3_0 and file _3 _1before the process is terminated are skipped, and the traversal efficiency is improved.

If the last deleted directory entry before the process is terminated is file _3 \, the target inflection point is file _3 \, the inflection point of the 1 st layer is dir _1 \0, the inflection point of the 2 nd layer is file _2 \2, the depth traversal is performed on the target directory after the process is resumed, traversal is started from/path/to/dir, the inflection point dir _1 \0is detected at the 1 st layer, traversal is started from dir _1 \0at the 1 st layer, the traversed directory entry of the 1 st layer is dir _1 \1, then dir _1 \1is traversed, the file inflection point file _2 \2is detected at the second layer, traversal is started from file _2 \2at the 2 nd layer, the traversed directory entry of the 2 nd layer is dir _2 \3, then dir _2 \3is traversed, the target inflection point 3 is detected at the 3 rd layer, the traversal is started from file _3 \ _3, and the traversal is started from dir _3 rd layer 3 \ _ 4. During traversal, the deleted dir _1 \, its subfiles, file _2 \, and file _3 \, before the process is terminated, are skipped, which improves the traversal efficiency.

Optionally, in the processing method of the process provided in the embodiment of the present application, sequentially deleting at least one to-be-processed directory entry according to a target order includes: packing the directory entries to be processed and cursor information into a transaction in a preset number under the condition that the number of the directory entries to be processed is greater than or equal to the preset number, and persisting the transaction in a metadata system; and the metadata system controls the thread to execute the operation of sequentially deleting the directory entries in the transaction according to the target sequence.

For example, fig. 3 is a schematic flowchart of a process of traversing the target directory according to an embodiment of the present application, where the preset number may be 2 as shown in fig. 3, and when the number of traversed to-be-processed directory entries is greater than 2, the traversed to-be-processed directory entries and the cursor information are packed into one transaction according to the traversal order. The current processing level of the directory items file _3 \, 1 to be processed and the cursor information at this time is 3, the target inflection point is file _3 \, 1 packed as transaction 1, after the transaction 1 is processed, file _3 \, 2 is deleted, and after the deletion of file 3 \, 2 is completed, the target inflection point is updated to file _3 \, 2 and the current processing level is still 3, then the deletion of dir _2 \, 0 and dir \, 2 \, 0 is continued, the target inflection point is updated to dir _2 \, 0, the current processing level is changed from 3 to 2, at this time, the deleted directory items file _3 u \, 2 \, 0 and the cursor information at this time are 2 at the current processing level, and the target inflection point is dir _2 \, packed 0 is transaction 2. Transaction 2 processing is complete.

Optionally, in the processing method of the process provided in the embodiment of the present application, updating the inflection point set includes: deleting the inflection point corresponding to the directory entry deleted last from the inflection point set; and determining the current deleted directory entry as an updated target inflection point, and adding the updated target inflection point to the inflection point set.

Specifically, the currently deleted directory entry is marked as a target inflection point, after the deletion of the currently to-be-deleted directory entry is completed, the currently to-be-deleted directory entry is changed into the currently deleted directory entry, the target inflection point corresponding to the last deleted directory entry is deleted from the inflection point set, and the currently deleted directory entry is added to the inflection point set as a new target inflection point. And simultaneously updating the inflection point information in the inflection point set. And deleting the old target inflection point and the inflection point information in the subdirectory thereof. By updating the inflection point set, the progress of deleting the target directory can be recorded in time.

For example, as shown in FIG. 3, dir _2 \0is the old target inflection point and file _2_1 is the new target inflection point. Both dir _2 \, 0 and the inflection point information file _3 \, 2 of the third layer under the directory thereof are deleted from the inflection point set, and file _2 \, 1 is recorded in the inflection point set as the inflection point information of the second layer. At this time, the third layer has no inflection point.

As the set of inflection points is updated, the current processing hierarchy is also updated, and optionally after updating the set of inflection points, the method further comprises: judging whether the updated current directory item to be deleted is the same as the hierarchy number of the current deleted directory item; under the condition that the hierarchy number of the updated current directory item to be deleted is different from the hierarchy number of the current deleted directory item, updating the current processing hierarchy in the cursor information into a hierarchy corresponding to the hierarchy number of the updated current directory item to be deleted; and under the condition that the updated current directory entry to be deleted is the same as the hierarchy number of the current deleted directory entry, keeping the current processing hierarchy in the cursor information unchanged.

For example, as shown in fig. 3, the currently deleted directory item is file _3_1, the currently to-be-deleted directory items are file _3_2, and the tier numbers of file_3_1 and file _3_2 are all 3, at this time, the currently to-be-deleted directory item and the currently deleted directory item have the same tier number, and when the target inflection point is updated from file _3_1 to file _3_2, the current processing tier in the cursor information is still 3. If the currently deleted directory entry is file _3_2, the currently to-be-deleted directory entry is dir _2_0, the hierarchy number of file _3 _2is 3, and the hierarchy number of dir _2 _0is 2, at this time, the currently to-be-deleted directory entry is different from the hierarchy number of the currently deleted directory entry, and when the target inflection point is updated from file _3 _2to dir _2_0, the currently processed hierarchy in the cursor information is changed from 3 to 2.

According to another embodiment of the present application, an alternative method for deleting a target directory is provided, as shown in fig. 3: before the target directory is traversed, recording each layer of directory structure of the target directory into the directory item set being traversed, then starting to traverse the target directory "/path/to/dir", setting the threshold value of batch deletion to be 2, and setting the threshold value of the number of directory items traversed by one search to be 2. The inflection point of the cursor at the layer (layer 0) of the target directory is recorded as null, so two directory entries, dir _1_0 and dir _1_1, are obtained from the theoretical minimum starting position, i.e. from/path/to/dir traversal.

When the first directory item dir _1 \0is continuously traversed downwards in a deep mode, the first directory item dir _1 \0is found to be a directory, the first directory item is recorded into the directory item set being traversed before the traversal, and because the inflection point of the cursor at the layer (layer 1) is recorded to be empty at the moment, two directory items, dir _2 \0and file _2 \1are obtained by the traversal from the theoretical minimum starting position (dir _1 \0).

When the first directory item dir _2 \, 0 is continuously traversed downwards in a deep mode, the first directory item dir _2 \, 0 is found to be a directory, the first directory item is recorded into the directory item set being traversed before the traversal, and because the inflection point of the cursor at the layer (layer 2) is recorded to be empty at the moment, two directory items, file _3 \, 0 and file _3 \, 1 are obtained by the traversal from the theoretical minimum starting position (dir _2 \, 0).

The directory entries file _3 \, 1 are traversed, and both files can be directly deleted as normal files. At this time, the maximum traversal level of the cursor is set to 3, and the inflection point information of layer 2 is set to file _3_1 (although file _3 _1is located at layer 3, the inflection point information belongs to layer 2). The two files to be deleted (file _3 \, 1) and the cursor information at this time are integrated into one transaction (the box numbered 1 in fig. 3), and the total number of transactions in the memory of the main metadata system is updated after the completion of the persistent execution of each metadata system, that is, the total number of transactions in the memory of the main metadata system is reduced by 1.

At this time, the original master metadata system process exits by an unexpected error, and one of the metadata systems is selected as a new master metadata system. The thread in the new main metadata system, which is specially responsible for background processing of internal requests, scans the persistent delete target directory request and the corresponding cursor information from the disk (database), and then re-traverses the/path/to/dir according to the cursor information and resumes executing the delete operation that is terminated unexpectedly.

The above steps are executed again by traversal, and since the cursor information records the corresponding inflection point information, so as to skip searching the directory entry that has been deleted before, the directory dir _2 \0is traversed from file _3_1 to obtain a directory entry, file _3_2 (because only this directory entry is left). Next, the parent directory dir _2 \, 0 is deleted, and dir _2 \, 0 is deleted from the set of directory items being traversed. At this time, the maximum traversal level of the cursor is set to 2, and the inflection point information of the 1 st layer is set to dir _2 \0 (although dir _2 \0is located at the 2 nd layer, the inflection point information belongs to the 1 st layer), and all inflection point information equal to or greater than the maximum traversal level, that is, the inflection point information file _3 \1of the 2 nd layer, needs to be deleted according to the update rule of the cursor information. The file and directory (file _3 \, dir _2 \, 0) to be deleted and the current cursor information are integrated into one transaction (box numbered 2 in fig. 2), and the total number of transactions in the main metadata system memory is updated after the completion of the persistent execution of each metadata system, that is, the total number of transactions in the main metadata system memory is reduced by 1.

The similar traversal and deletion operations described above are continued, i.e., the transactions are generated and persisted across the metadata systems in the order of the numbering of the blocks in FIG. 3 until the target directory itself (/ path/to/dir) is deleted. And deleting the request for deleting the target directory and the corresponding cursor information in the last transaction, and finally completing the whole request for deleting the target directory after the last transaction updates the total number of the transactions in the memory of the main metadata system after the persistent execution of each metadata system is completed.

It should be noted that the steps illustrated in the flowcharts of the figures may be performed in a computer system such as a set of computer-executable instructions and that, although a logical order is illustrated in the flowcharts, in some cases, the steps illustrated or described may be performed in an order different than presented herein.

The embodiment of the present application further provides a processing apparatus of a process, and it should be noted that the processing apparatus of the process according to the embodiment of the present application may be used to execute the processing method for the process provided in the embodiment of the present application. The following describes a processing apparatus for processing a process provided in an embodiment of the present application.

Fig. 4 is a schematic diagram of a processing device of a process according to an embodiment of the present application. As shown in fig. 4, the apparatus includes:

a detecting unit 10, configured to detect whether a process of a deletion operation is terminated before the deletion operation of a target directory is completed, where the target directory includes a plurality of directory entries;

an obtaining unit 20, configured to obtain cursor information of the target directory in a case that the process is terminated, where the cursor information is used to record a deletion progress of a directory entry in the target directory;

a first determining unit 30, configured to determine a set of inflection points of a process from the cursor information, where the set of inflection points includes one or more inflection points, and the inflection points are used to mark currently deleted directory entries;

a second determining unit 40, configured to determine, according to the inflection point set, a directory entry that is deleted last before the process is terminated;

and the recovery unit 50 is configured to determine, according to the last deleted directory entry, a current directory entry to be deleted after the process is terminated, recover the process according to the current directory entry to be deleted, and continue to execute the deletion operation in the process.

In the processing apparatus for a process provided in the embodiment of the present application, by using the detection unit 10, before a deletion operation of a target directory is completed, whether the process of the deletion operation is terminated is detected, where the target directory includes a plurality of directory entries; the acquisition unit 20 is used for acquiring cursor information of the target directory under the condition that the process is terminated, wherein the cursor information is used for recording the deletion progress of the directory items in the target directory; a first determining unit 30, which determines a set of inflection points of the process from the cursor information, wherein the set of inflection points includes one or more inflection points, and the inflection points are used for marking the currently deleted directory entry; a second determining unit 40, which determines the directory entry deleted last before the process is terminated according to the inflection point set; the recovery unit 50 determines the current directory item to be deleted after the process is terminated according to the last deleted directory item, recovers the process according to the current directory item to be deleted, and continues to execute the deletion operation in the process, so that the problem that the file deletion efficiency is low because the deleted file is required to be traversed again before the process is terminated when the process for deleting the file is recovered in the related art is solved, the current directory item to be deleted after the process is terminated is determined according to the inflection point in the cursor information by recording the cursor information of the deleted process, the traversed file before the process is terminated is prevented from being traversed again, and the effect of improving the file deletion efficiency is achieved.

Optionally, in the processing apparatus of the process provided in the embodiment of the present application, the second determining unit 40 includes: the acquisition module is used for acquiring a current processing level in the cursor information, wherein the current processing level is the level where the directory entry deleted at last is located; a first determining module, configured to determine an inflection point in the set of inflection points, which is the same as a hierarchy number of a current processing hierarchy, as a target inflection point, where the hierarchy number is used to mark a hierarchy of each directory entry in a target directory; and the second determining module is used for determining the directory entry corresponding to the target inflection point as the directory entry deleted at last.

Optionally, in the processing apparatus of the process provided in the embodiment of the present application, the recovery unit 50 includes: the detection module is used for detecting whether the same-layer directory item exists in a target parent directory of the directory item corresponding to the target inflection point, wherein the same-layer directory item is the directory item except the directory item corresponding to the target inflection point in the target parent directory; the third determining module is used for determining the same-layer directory entry traversed first in the at least one same-layer directory entry as the current directory entry to be deleted under the condition that the target parent directory has the at least one same-layer directory entry; and the fourth determining module is used for determining the target parent directory as the current directory item to be deleted under the condition that the same-layer directory item does not exist in the target parent directory.

Optionally, in the processing apparatus of the process provided in the embodiment of the present application, the recovery unit 50 includes: the judging module is used for judging whether other inflection points exist in the inflection point set, wherein the other inflection points are inflection points in the inflection point set except the target inflection point; the first traversal module is used for traversing the directory items in the target directory by taking the current directory items to be deleted as a starting point under the condition that other inflection points do not exist in the inflection point set to obtain at least one directory item to be processed, wherein the number of the directory items to be processed is smaller than a preset threshold value; the second traversal module is used for skipping and traversing directory entries in subdirectories of other inflection points under the condition that other inflection points exist in the inflection point set, and traversing directory entries in the target directory by taking the current directory entry to be deleted as a starting point to obtain at least one directory entry to be processed; the deleting module is used for sequentially deleting at least one directory item to be processed according to a target sequence; and the fifth determining module is used for obtaining the current deleted directory item after the current to-be-deleted directory item is deleted, determining the next traversed directory item as the updated current to-be-deleted directory item, and updating the inflection point set.

Optionally, in the processing apparatus for a process provided in an embodiment of the present application, the deleting module includes: the persistence sub-module is used for packing the directory items to be processed and the cursor information into a transaction in a preset number under the condition that the number of the directory items to be processed is greater than or equal to the preset number, and persisting the transaction in the metadata system; and the execution submodule is used for controlling the thread to execute the operation of deleting the directory entries in the transaction in sequence according to the target sequence by the metadata system.

Optionally, in the processing apparatus of the process provided in the embodiment of the present application, the fifth determining module includes: the deleting submodule is used for deleting the inflection point corresponding to the directory entry deleted at last from the inflection point set; and the adding submodule is used for determining the current deleted directory entry as an updated target inflection point and adding the updated target inflection point to the inflection point set.

Optionally, in a processing apparatus of a process provided in an embodiment of the present application, the apparatus further includes: the judging unit is used for judging whether the updated current directory item to be deleted is the same as the hierarchy number of the current deleted directory item; the first updating unit is used for updating the current processing level in the cursor information to the level corresponding to the level number of the updated current directory item to be deleted under the condition that the level numbers of the updated current directory item to be deleted and the current deleted directory item are different; and the second updating unit is used for keeping the current processing level in the cursor information unchanged under the condition that the updated current directory item to be deleted is the same as the level number of the current deleted directory item.

The processing device of the process comprises a processor and a memory, wherein the detection unit 10, the acquisition unit 20, the first determination unit 30, the second determination unit 40, the recovery unit 50 and the like are stored in the memory as program units, and the processor executes the program units stored in the memory to realize corresponding functions.

The processor comprises a kernel, and the kernel calls the corresponding program unit from the memory. One or more kernels can be set, and file deleting efficiency is improved by adjusting kernel parameters.

The memory may include volatile memory in a computer readable medium, random Access Memory (RAM) and/or nonvolatile memory such as Read Only Memory (ROM) or flash memory (flash RAM), and the memory includes at least one memory chip.

An embodiment of the present invention provides a computer-readable storage medium on which a program is stored, the program implementing a processing method of a process when executed by a processor.

The embodiment of the invention provides a processor, which is used for running a program, wherein the program runs to execute a processing method of a process.

The embodiment of the invention provides electronic equipment, which comprises a processor, a memory and a program which is stored on the memory and can run on the processor, wherein the processor executes the program and realizes the following steps: a process processing method. The device herein may be a server, a PC, a PAD, a mobile phone, etc.

The present application further provides a computer program product adapted to perform a program for initializing the following method steps when executed on a data processing device: a process processing method.

As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.

The memory may include forms of volatile memory in a computer readable medium, random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). The memory is an example of a computer-readable medium.

Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, a computer readable medium does not include a transitory computer readable medium such as a modulated data signal and a carrier wave.

It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrases "comprising a," "8230," "8230," or "comprising" does not exclude the presence of additional identical elements in the process, method, article, or apparatus comprising the element.

The above are merely examples of the present application and are not intended to limit the present application. Various modifications and changes may occur to those skilled in the art to which the present application pertains. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A method for processing a process, comprising:

before the execution of the deleting operation of the target directory is completed, detecting whether the process of the deleting operation is terminated or not, wherein the target directory comprises a plurality of directory entries;

under the condition that the process is terminated, acquiring cursor information of the target directory, wherein the cursor information is used for recording the deletion progress of directory entries in the target directory;

determining a set of inflection points for the process from the cursor information, wherein the set of inflection points includes one or more inflection points used to mark a currently deleted directory entry;

determining the directory entry deleted last before the process is terminated according to the inflection point set;

and determining the current directory item to be deleted after the process is terminated according to the last deleted directory item, recovering the process according to the current directory item to be deleted, and continuously executing the deletion operation in the process.

2. The method of claim 1, wherein determining the directory entry that was last deleted before the process terminated based on the set of inflection points comprises:

acquiring a current processing level in the cursor information, wherein the current processing level is a level where the directory entry deleted last is located;

determining an inflection point in the set of inflection points that is the same as a tier number of the current processing tier as a target inflection point, wherein the tier number is used to mark a tier of each directory entry in the target directory;

and determining the directory entry corresponding to the target inflection point as the directory entry deleted at last.

3. The method of claim 2, wherein determining the current directory entry to be deleted after the process terminates according to the last deleted directory entry comprises:

detecting whether a same-layer directory item exists in a target parent directory of the directory item corresponding to the target inflection point, wherein the same-layer directory item is a directory item in the target parent directory except the directory item corresponding to the target inflection point;

determining a peer directory entry traversed first in the at least one peer directory entry as the current directory entry to be deleted under the condition that the target parent directory has the at least one peer directory entry;

and under the condition that the same-layer directory entry does not exist in the target parent directory, determining the target parent directory as the current directory entry to be deleted.

4. The method according to claim 2, wherein recovering the process according to the current directory entry to be deleted, and continuing to perform the deletion operation in the process comprises:

judging whether other inflection points exist in the inflection point set, wherein the other inflection points are inflection points in the inflection point set except the target inflection point;

traversing the directory items in the target directory by taking the current directory item to be deleted as a starting point to obtain at least one directory item to be processed under the condition that the other inflection points do not exist in the inflection point set, wherein the number of the directory items to be processed is smaller than a preset threshold value;

under the condition that the other inflection points exist in the inflection point set, skipping to traverse directory entries in subdirectories of the other inflection points, and traversing directory entries in the target directory by taking the current directory entry to be deleted as a starting point to obtain at least one directory entry to be processed;

deleting the at least one directory item to be processed in sequence according to a target sequence;

and after the current directory item to be deleted is deleted, obtaining the current deleted directory item, determining the next traversed directory item as the updated current directory item to be deleted, and updating the inflection point set.

5. The method of claim 4, wherein sequentially deleting the at least one pending directory entry in the target order comprises:

under the condition that the number of the directory items to be processed is greater than or equal to a preset number, packaging the preset number of the directory items to be processed and the cursor information into a transaction, and persisting the transaction in a metadata system;

and the metadata system control thread executes the operation of deleting the directory entries in the transaction in sequence according to the target sequence.

6. The method of claim 4, wherein updating the set of inflection points comprises:

deleting the inflection point corresponding to the finally deleted directory entry from the inflection point set;

determining the current deleted directory entry as an updated target inflection point, and adding the updated target inflection point to the inflection point set.

7. The method of claim 4, wherein after updating the set of inflection points, the method further comprises:

judging whether the updated current to-be-deleted directory entry is the same as the hierarchy number of the current deleted directory entry;

under the condition that the updated directory item to be deleted and the current deleted directory item have different hierarchy numbers, updating the current processing hierarchy in the cursor information to a hierarchy corresponding to the hierarchy number of the updated directory item to be deleted;

and under the condition that the updated directory entry to be deleted is the same as the current deleted directory entry in the hierarchy number, keeping the current processing hierarchy in the cursor information unchanged.

8. An apparatus for processing a process, comprising:

the device comprises a detection unit, a processing unit and a processing unit, wherein the detection unit is used for detecting whether the process of the deletion operation is terminated or not before the execution of the deletion operation of a target directory is completed, and the target directory comprises a plurality of directory entries;

the acquisition unit is used for acquiring cursor information of the target directory under the condition that the process is terminated, wherein the cursor information is used for recording the deletion progress of directory items in the target directory;

a first determining unit, configured to determine a set of inflection points of the process from the cursor information, where the set of inflection points includes one or more inflection points, and the inflection points are used to mark a currently deleted directory entry;

a second determining unit, configured to determine, according to the inflection point set, a directory entry that is deleted last before the process is terminated;

and the recovery unit is used for determining the current directory entry to be deleted after the process is terminated according to the last deleted directory entry, recovering the process according to the current directory entry to be deleted, and continuously executing the deletion operation in the process.

9. A processor, characterized in that the processor is configured to execute a program, wherein the program executes a processing method of executing the process of any one of claims 1 to 7.

10. An electronic device, comprising one or more processors and memory for storing one or more programs, wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the process of any of claims 1-7.