CN110874182B - Processing method, device and equipment for strip index - Google Patents

Processing method, device and equipment for strip index Download PDF

Info

Publication number
CN110874182B
CN110874182B CN201811015213.6A CN201811015213A CN110874182B CN 110874182 B CN110874182 B CN 110874182B CN 201811015213 A CN201811015213 A CN 201811015213A CN 110874182 B CN110874182 B CN 110874182B
Authority
CN
China
Prior art keywords
index
stripe
recorded
deleted
determining
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201811015213.6A
Other languages
Chinese (zh)
Other versions
CN110874182A (en
Inventor
黄华东
夏伟强
王伟
林起芊
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hangzhou Hikvision System Technology Co Ltd
Original Assignee
Hangzhou Hikvision System Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hangzhou Hikvision System Technology Co Ltd filed Critical Hangzhou Hikvision System Technology Co Ltd
Priority to CN201811015213.6A priority Critical patent/CN110874182B/en
Publication of CN110874182A publication Critical patent/CN110874182A/en
Application granted granted Critical
Publication of CN110874182B publication Critical patent/CN110874182B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/0608—Saving storage space on storage systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0646—Horizontal data movement in storage systems, i.e. moving data in between storage devices or systems
    • G06F3/0652—Erasing, e.g. deleting, data cleaning, moving of data to a wastebasket
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/062—Securing storage systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0638—Organizing or formatting or addressing of data
    • G06F3/064—Management of blocks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/0671—In-line storage system
    • G06F3/0683—Plurality of storage devices
    • G06F3/0689—Disk arrays, e.g. RAID, JBOD
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/0614—Improving the reliability of storage systems
    • G06F3/0619—Improving the reliability of storage systems in relation to data integrity, e.g. data losses, bit errors

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The application provides a processing method, a device and equipment for a stripe index, comprising the following steps: determining a target object to be deleted; acquiring a found strip index of the strip storing the target object; setting the state of the target object as a specified mark when determining that other object identifiers except the target object identifier are recorded in the searched strip index; and deleting the searched strip index when the state of the object corresponding to each object identifier recorded in the searched strip index is determined to be a designated mark. The method provided by the application can prevent the problem of metadata loss of the undeleted object caused by deleting the stripe index.

Description

Processing method, device and equipment for strip index
Technical Field
The present invention relates to the field of storage, and in particular, to a method, an apparatus, and a device for processing a stripe index.
Background
An object storage system generally includes a storage node and a management node. A storage node, also called OSD (Object Storage Device ) device, is used to store objects. The management node, also called as a Metadata Server (MDS) device, is configured to store Metadata of an object stored in the OSD device, where the Metadata of the object may represent object information such as a location where the object is stored, an input time of the object, and the like.
In general, metadata for an object may include an object index and a stripe index. The object index includes at least: object identification, stripe identification of a stripe storing the object, object write time, and the like. The stripe index at least comprises information such as block identification on the stripe and object identification of the object stored by the block.
The object storage system periodically deletes some designated objects, and when deleting, the storage node and the management node execute the deletion policy at the same time, the storage node deletes the object to be deleted, and the management node deletes the metadata of the object to be deleted. Such as the management node deleting the object index of the object and the stripe index of the stripe storing the object. However, when the deleted stripe index also stores other objects, metadata of the other objects is lost.
Disclosure of Invention
In view of this, the present application provides a method, apparatus and device for processing a stripe index, so as to prevent the problem of metadata loss of an undeleted object caused by deleting the stripe index.
Specifically, the application is realized by the following technical scheme:
according to a first aspect of the present application, there is provided a method of processing a stripe index, the method being applied to a management node in an object storage system, comprising:
determining a target object to be deleted;
searching a stripe index of a stripe storing the target object;
detecting whether other object identifiers except the target object identifier are recorded in the searched strip index;
if yes, setting a designated mark for the target object identification;
when each object identification recorded in the stripe index is determined to be provided with a designated mark, the stripe index is deleted.
Optionally, the method further comprises:
and if the found stripe index only records the target object identification, deleting the stripe index.
Optionally, the determining the target object to be deleted includes:
determining an object index meeting specified conditions in the locally recorded object indexes;
and taking the object indicated by the determined object index as the object to be deleted.
Optionally, the object index records at least the writing time of the object;
the determining the object index meeting the specified condition in the locally recorded object indexes comprises the following steps:
subtracting a preset time length from the current time to determine a deleting time;
searching the object index before the deleting time at the object writing time, and determining the searched object index as the object index meeting the specified condition.
Optionally, the stripe index records at least an object identifier;
the searching the stripe index of the stripe storing the target object includes:
and searching the recorded stripe index by taking the object identification of the target object as a keyword, wherein the stripe index comprises the keyword.
According to a second aspect of the present application, there is provided a processing apparatus of a stripe index, the apparatus being applied to a management node in an object storage system, comprising:
a determining unit configured to determine a target object to be deleted;
a searching unit for searching a stripe index of a stripe storing the target object;
the detection unit is used for detecting whether other object identifiers except the target object identifier are recorded in the searched strip index;
a setting unit configured to set a designation flag for the target object identifier if yes;
and the processing unit is used for deleting the strip index when determining that each object identifier recorded in the strip index is provided with the index mark.
Optionally, the processing unit is further configured to delete the stripe index if the found stripe index only records the target object identifier.
Optionally, the determining unit is specifically configured to determine, among locally recorded object indexes, an object index that meets a specified condition; and taking the object indicated by the determined object index as the object to be deleted.
Optionally, the object index records at least the writing time of the object;
the determining unit is specifically configured to determine a deletion time by subtracting a preset time from a current time when determining an object index satisfying a specified condition from locally recorded object indexes; searching the object index before the deleting time at the object writing time, and determining the searched object index as the object index meeting the specified condition.
Optionally, the stripe index records at least an object identifier;
the searching unit is specifically configured to search, in the recorded stripe indexes, the stripe index containing the keyword by using the object identifier of the target object as the keyword.
According to a third aspect of the present application, there is provided an object management apparatus comprising a processor and a machine-readable storage medium storing machine-executable instructions executable by the processor to cause the processing method of the stripe index provided in the first aspect to be performed.
According to a fourth aspect of the present application there is provided a machine-readable storage medium storing machine-executable instructions which, when invoked and executed by a processor, cause the processor to perform the method of processing a stripe index provided in the first aspect.
The application provides a processing method of a stripe index, wherein a management node can set a designated mark for an object to be deleted, and when the stripe index is deleted, the management node deletes the stripe index when determining that each object of a stripe index record is set with the designated mark. Otherwise, the stripe index is reserved.
Since the management node deletes the stripe index when it is determined that each object recorded on the stripe index is provided with the designation flag, it is possible to prevent the problem of metadata loss of an object not deleted due to deletion of the stripe index.
Drawings
FIG. 1 is a schematic view of a strip shown in an exemplary embodiment of the present application;
FIG. 2 is a flowchart illustrating a stripe index processing method according to an exemplary embodiment of the present application;
FIG. 3 is a hardware block diagram of an object management device according to an exemplary embodiment of the present application;
fig. 4 is a block diagram of a stripe index processing device according to an exemplary embodiment of the present application.
Detailed Description
Reference will now be made in detail to exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, the same numbers in different drawings refer to the same or similar elements, unless otherwise indicated. The implementations described in the following exemplary examples are not representative of all implementations consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with some aspects of the present application as detailed in the accompanying claims.
The terminology used in the present application is for the purpose of describing particular embodiments only and is not intended to be limiting of the present application. As used in this application and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used herein refers to and encompasses any or all possible combinations of one or more of the associated listed items.
It should be understood that although the terms first, second, third, etc. may be used herein to describe various information, these information should not be limited by these terms. These terms are only used to distinguish one type of information from another. For example, a first message may also be referred to as a second message, and similarly, a second message may also be referred to as a first message, without departing from the scope of the present application. The word "if" as used herein may be interpreted as "at … …" or "at … …" or "responsive to a determination", depending on the context.
Before describing the stripe index processing method provided by the application, concepts related to the application are described.
1) Stripe, object, block
In an object storage system, stored data, files, etc. are referred to as objects.
In an object storage system, storage nodes OSD and management nodes MDS are typically included. A storage node refers to a storage device for storing objects. The storage node may comprise several disks. N of the number of disks may form a disk array. The total storage space of the N disks is divided into N stripes, each stripe comprising N blocks, the N blocks being used to store objects.
As shown in fig. 1, the disk array shown in fig. 1 is a disk array composed of 6 disks. Each column in fig. 1 represents a disk, each row represents a stripe, and each stripe has 6 blocks thereon.
When the object is stored, if the size of the object is larger than one block on the stripe, splitting the object into N sub-objects, and using the N blocks on the stripe to store the N sub-objects respectively. If the size of the object is smaller than a block on the stripe, then a block on the stripe is used to store the object.
2) Object index
An object index table is configured on a management node in the object storage system. The object index table includes a plurality of object indexes, each object index corresponding to an object stored in a storage node. The object index may indicate object information such as a storage location of the object.
The object index records at least the object identifier of the object, the writing time of the object, the stripe identifier for storing the object, and the like, and of course, other contents, such as the size of the object, and the like, may also be included in the object index, which is only exemplified herein and not specifically limited thereto.
When the object storage system needs to store a received object, the storage node may store the object in a specified stripe and the management node may generate an object index for the object.
For example, assuming that object 1 needs to be stored, assuming that the storage node stores object 1 on blocks 1 and 2 of stripe 1, the write time is 2018/05/16 19:00.
The management node may generate an object index 1 for object 1, the generated object index 1 being shown in table 1.
Object identification Storing stripe identification of the object Write time
Object 1 Strip 1 2018/05/16 19:00
TABLE 1
Of course, other contents, such as the size of the object, etc., may be included in table 1, and the object index is only exemplarily described herein, and is not specifically limited.
3) Stripe index
To facilitate stripe management, a stripe index table is also configured on a management node in the object storage system. The stripe index table records the stripe indexes of all stripes. At least the stripe index records: the block identification of the block on the stripe, the object identification of the object stored by the block.
For example, assume that object 1 needs to be stored, assume that the storage node stores object 1 on blocks 1 and 2 of stripe 1. It is assumed that object 2 is also needed to be stored, and that the storage node stores object 2 on block 3 on stripe 1.
At this time, the stripe index of stripe 1 on the management node is shown in table 2.
TABLE 2
Of course, the stripe index shown in Table 2 may also include other content, which is only exemplary and not specifically limited.
Typically, the object storage system will delete some objects periodically to free up space to store newly received objects. When deleting an object, the storage node OSD may delete the object from the blocks of the stripe. The management node MDS may delete metadata of the object to be deleted, such as the MDS may delete the object index and stripe index of the object to be deleted.
However, when the deleted stripe index also contains other object identifiers, deleting the stripe index directly results in metadata loss of other objects.
For example, assuming the storage system is to delete object 1, object 1 is stored on blocks 1 and 2 of stripe 1, and object 2 is also stored on block 3 of stripe 1. While object 2 is an object that does not need to be deleted.
When deleting object 1, the OSD may delete object 1 from blocks 1 and 2 of stripe 1. Meanwhile, the MDS may delete the object index of object 1 and delete the stripe index of stripe 1. However, since the stripe index of the stripe 1 also stores the object 2, and the object 2 is an object that does not need to be deleted, the stripe index of the stripe 1 is deleted, so that the stripe index of the object 2 is lost, and the metadata of the object 2 is lost.
In view of this, the present application provides a processing method of a stripe index, where a management node may set a designation flag for an object to be deleted, and when deleting the stripe index, the management node deletes the stripe index when determining that each object of the stripe index record is set with the designation flag. Otherwise, the stripe index is reserved.
Since the management node deletes the stripe index when it is determined that each object recorded on the stripe index is provided with the designation flag, it is possible to prevent the problem of metadata loss of an object not deleted due to deletion of the stripe index.
Referring to fig. 2, fig. 2 is a flowchart illustrating a stripe index processing method according to an exemplary embodiment of the present application. The processing method of the stripe index can be applied to a management node of the object storage system and can comprise the following steps. The management node may periodically perform steps 201 to 203.
Step 201: the management node may determine the target object to be deleted.
In an object storage system, the object storage system periodically deletes some data that is earlier in time to free up some space to store newly written objects.
In the implementation, the management node may search for an object index satisfying the specified condition in the locally recorded object index list, and use the object indicated by the object identifier recorded in the searched object index as the object to be deleted.
When searching for the object index meeting the specified condition, an alternative implementation manner is that the management node can subtract the preset time length from the current time to obtain the deletion time. Then, the management node may search the object index before the deletion time at the writing time, and then use the object indicated by the object identifier recorded in the searched object index as the target object to be deleted.
It should be noted that, the above-mentioned preset time period is a time period set by a developer according to actual experience, and the preset time period is not specifically limited here.
It should be noted that, the management node may also search some specified object indexes as object indexes satisfying the specified condition, and only one example of searching the object indexes satisfying the specified condition is described herein, and the object indexes searching the specified condition are not specifically limited.
Step 202: the management node may look up a stripe index that stores stripes of the target object.
In implementation, the management node may use the object identifier of the target object as a key, and search the stripe index recorded with the key in the stripe index list.
Step 203: the management node may detect whether other object identifiers than the target object identifier are recorded in the searched stripe index.
Step 204: if the found stripe index records other object identifiers except the target object identifier, the management node can set a specified mark for the target object identifier.
If the found stripe index also records other object identifications than the target object identification, the management node may set a specified flag for the target object identification.
For example, the management node may add a specific flag to the target object identification in the found stripe index, where the specific flag indicates that the target object is an object to be deleted.
If the found stripe index only records the target object identification, the management node may delete the found stripe index from the stripe index list. In addition, the management node may delete the object index corresponding to the target object.
Of course, the management node may delete the object index corresponding to the target object when determining the target object to be deleted, and the present application does not specifically limit the object index time of deleting the target object, so long as the management node deletes the object index corresponding to the target object after determining the target object to be deleted.
Step 205: when the management node determines that each object identification recorded in the searched stripe index is provided with a designation, the searched stripe index is deleted.
In an alternative implementation, after performing steps 202 and 203 on each target object to be deleted, the management node may traverse the stripe index list and sequentially read the stripe index containing the object identifier of the object to be deleted.
The management node may perform the following steps for each stripe index read.
Here, the read stripe is taken as an example of the found stripe index
The management node may detect whether the object identification of each object of the found stripe index record is set with a specified flag.
If each object in the searched stripe index is configured with a designated mark, the management node can delete the stripe index. In addition, the management node may delete the object index of the target object.
If some object identifiers are configured with the specified marks or none of the object identifiers recorded in the searched strip indexes, the searched strip indexes are reserved. In addition, the management node may delete the object index of the target object.
It should be noted that, the management node may also delete the object index corresponding to the target object when determining the target object to be deleted, and the present application does not specifically limit the object index time of deleting the target object, as long as the management node deletes the object index corresponding to the target object after determining the target object to be deleted.
It should be noted that, in a general and easily conceivable manner, a monitoring task is started in the background, and when the monitoring task in the background monitors that all object identifiers in the stripe index are configured with a specified flag, the stripe index is deleted. And the current CPU calls the program of the memory to only set the appointed mark for the object identification of the object to be deleted periodically. In short, the setting of the designation flag and the monitoring task of the background do not have a correlation, and the two perform respective tasks. This is disadvantageous in that when there are a large number of stripe indexes in which a plurality of object identifiers are recorded, the background monitoring task becomes a burden on the MDS, and the MDS will work.
Thus, what is used in this application is: after the operations of steps 202 to 204 are completed for the target object to be deleted. And sequentially reading the strip indexes containing the object identifications of the objects to be deleted from the strip index list, and deleting the strip indexes when determining that the object identifications of a plurality of objects recorded by the strip indexes are configured with the index identifications. The deleting operation of the stripe index is not completed by the background, but the CPU calls the program in the memory to execute the operation of deleting the stripe index after the setting of the specified mark on the object identification of all the target objects is completed. Because the background monitoring task is not required to be set independently, the problem that the background task consumes MDS equipment resources is avoided, and the equipment performance can be improved by adopting the method provided by the application.
The processing method of the stripe index provided in the present application is described in detail below by way of specific examples.
Step 1: the management node determines the object to be deleted.
The management node may determine the object to be deleted by using the method described in step 201, which will not be described in detail here.
Suppose that the management node determines that there are 4 objects to be deleted, object 1, object 2, object 3, and object 4, respectively.
Step 2: the management node can search the stripe index corresponding to each object to be deleted.
Specifically, the management node may search for a stripe index containing the key by using the object identifier of the object to be deleted as a key, and determine the searched stripe index as the stripe index corresponding to the object to be deleted.
For example, assume that blocks 1-3 on stripe 1 store object 1 and blocks 5-6 on stripe 1 store object 2. Only object 1 and object 2 are stored on stripe 1.
The management node may use the stripe index 1 recorded with the object identifiers of the object 1 and the object 2 as the stripe index corresponding to the object 1 and the object 2, where the stripe index 1 is shown in table 3.
TABLE 3 Table 3
Assume that blocks 1-5 on stripe 2 store object 3 and that stripe 2 stores only object 3.
The management node may use the stripe index 2 recorded with the object identifier of the object 3 as the stripe index corresponding to the object 3, where the stripe index 2 is shown in table 4.
TABLE 4 Table 4
Assume that blocks 1 to 2 on stripe 3 store object 4, blocks 4 to 6 on stripe 3 store object 5, and object 5 is an object that does not need to be deleted. Stripe 3 stores only object 4 and object 5.
The management node may identify the stripe index 3 of the object 4 as the stripe index of the object 4 with the object identification stripe index 3 recorded with the object 4. Stripe index 3 is shown in table 5.
TABLE 5
Step 3: for each object to be deleted, the management node may check whether the object identifier of the other objects except the object to be deleted is recorded in the stripe index corresponding to the object to be deleted, and if so, set a specified flag for the object identifier of the object to be deleted. If not, the stripe index is deleted.
For example, for the object 1 to be deleted, the management node may check whether the stripe index 1 corresponding to the object 1 further records other object identifiers except the object 1 identifier, in this example, since the stripe index 1 further records the identifier of the object 2, the management node adds a specified flag to the object 1 in the stripe index 1.
For the object 2 to be deleted, the management node may check whether other object identifiers except the identifier of the object 2 are also recorded in the stripe index 1 corresponding to the object 2, and in this example, since the identifier of the object 1 is also recorded in the stripe index 1, the management node adds a specified flag to the object 2 in the stripe index 1.
The stripe index 1 after the management node adds the specified tag to the object 1 and the object 2 in the stripe index 1 is shown in table 6.
TABLE 6
For the object 3 to be deleted, the management node may check whether other object identifiers than the identifier of the object 3 are also recorded in the stripe index 2 corresponding to the object 3, and in this example, since only the identifier of the object 3 is recorded in the stripe index 2, the management node may delete the stripe index of the object 3.
For the object 4 to be deleted, the management node may check whether other object identifiers than the identifier of the object 4 are also recorded in the stripe index 3 corresponding to the object 4, and in this example, since the identifier of the object 5 is also recorded in the stripe index 3, the management node adds a specified flag to the object 4 in the stripe index 3. The stripe index 3 to which the specified flag is added for the object 4 is shown in table 7.
TABLE 7
Step 4: after the management node completes the operation of step 3 on the object to be deleted, the management node may search the stripe index containing the object identifier of the object to be deleted. For each stripe index found, the management node may check whether each object identification of the stripe index record is configured with a specified flag, if so, delete the stripe index, and if not, preserve the stripe index.
For example, in this example, the management node may obtain the stripe index recorded with the plurality of object identifiers as stripe index 1 and stripe index 3.
For stripe index 1, the management node may check whether the object identifier of each object in stripe index 1 is configured with a specified flag, in this example, object 1 and object 2 recorded in stripe index 1 are configured with specified flags, so the management node deletes stripe index 1.
For stripe index 3, the management node may check that the management node may check whether each object in stripe index 3 is configured with a specified tag, in this example object 4 recorded in stripe index 3 is configured with a specified tag, but recorded object 5 is not configured with a specified tag, so the management node may retain stripe index 3.
As is clear from the above description, the management node may set a specific flag for an object to be deleted, and when deleting a stripe index, the management node deletes the stripe index when determining that the object identifier of each object of the stripe index record is configured with the specific flag. Otherwise, the stripe index is reserved.
Since the management node deletes the stripe index when determining that the object identifier of each object recorded on the stripe index is configured with the specified flag, it is possible to prevent the problem of metadata loss of an undeleted object due to deletion of the stripe index.
Referring to fig. 3, fig. 3 is a hardware architecture diagram of an object management device where a stripe index processing apparatus is located according to an exemplary embodiment of the present application. The object management apparatus includes: a communication interface 301, a processor 302, a machine-readable storage medium 303, and a bus 304; wherein the communication interface 301, the processor 302 and the machine-readable storage medium 303 perform communication with each other via a bus 304. The processor 302 may perform the stripe index processing method described above by reading and executing machine-executable instructions in the machine-readable storage medium 303 corresponding to the stripe index processing control logic.
The machine-readable storage medium 303 referred to herein may be any electronic, magnetic, optical, or other physical storage device that may contain or store information, such as executable instructions, data, or the like. For example, the machine-readable storage medium 303 may include at least one of the following: volatile memory, nonvolatile memory, other types of storage media. The volatile memory may be RAM (Random Access Memory ), and the nonvolatile memory may be flash memory, a storage drive (e.g., a hard disk drive), a solid state disk, a storage disk (e.g., an optical disk, a DVD, etc.).
Referring to fig. 4, fig. 4 is a block diagram illustrating a stripe index deletion apparatus according to an exemplary embodiment of the present application, which may be applied to a management node in an object storage system, and may include the following units.
A determining unit 401 for determining a target object to be deleted;
a searching unit 402, configured to search a stripe index of a stripe storing the target object;
a detecting unit 403, configured to detect whether other object identifiers except the target object identifier are recorded in the searched stripe index;
a setting unit 404, configured to set a specified flag for the target object identifier, if yes;
a processing unit 405, configured to delete the stripe index when it is determined that each object identifier recorded in the stripe index is set with a designation.
Optionally, the processing unit 405 is further configured to delete the stripe index if the found stripe index only records the target object identifier.
Optionally, the determining unit 401 is specifically configured to determine, among locally recorded object indexes, an object index that meets a specified condition; and taking the object indicated by the determined object index as the object to be deleted.
Optionally, the object index records at least the writing time of the object;
the determining unit 401 is specifically configured to determine, when determining, from among locally recorded object indexes, an object index that satisfies a specified condition, a deletion time by subtracting a preset time period from a current time; searching the object index before the deleting time at the object writing time, and determining the searched object index as the object index meeting the specified condition.
Optionally, the stripe index records at least an object identifier;
the searching unit 402 is specifically configured to search, in the recorded stripe indexes, for a stripe index containing the key by using the object identifier of the target object as the key.
The implementation process of the functions and roles of each unit in the above device is specifically shown in the implementation process of the corresponding steps in the above method, and will not be described herein again.
For the device embodiments, reference is made to the description of the method embodiments for the relevant points, since they essentially correspond to the method embodiments. The apparatus embodiments described above are merely illustrative, wherein the elements illustrated as separate elements may or may not be physically separate, and the elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purposes of the present application. Those of ordinary skill in the art will understand and implement the present invention without undue burden.
The foregoing description of the preferred embodiments of the present invention is not intended to limit the invention to the precise form disclosed, and any modifications, equivalents, improvements and alternatives falling within the spirit and principles of the present invention are intended to be included within the scope of the present invention.

Claims (12)

1. A method for processing a stripe index, the method being applied to a management node in an object storage system, comprising:
determining a target object to be deleted;
searching a stripe index of a stripe storing the target object;
detecting whether other object identifiers except the target object identifier are recorded in the searched strip index;
if yes, setting a designated mark for the target object identification;
when each object identification recorded in the stripe index is determined to be provided with a designated mark, the stripe index is deleted.
2. The method according to claim 1, wherein the method further comprises:
and if the found stripe index only records the target object identification, deleting the stripe index.
3. The method of claim 1, wherein the determining the target object to be deleted comprises:
determining an object index meeting specified conditions in the locally recorded object indexes;
and taking the object indicated by the determined object index as the object to be deleted.
4. A method according to claim 3, wherein the object index records at least an object write time;
the determining the object index meeting the specified condition in the locally recorded object indexes comprises the following steps:
subtracting a preset time length from the current time to determine a deleting time;
searching the object index before the deleting time at the object writing time, and determining the searched object index as the object index meeting the specified condition.
5. The method of claim 1, wherein the stripe index records at least an object identification;
the searching the stripe index of the stripe storing the target object includes:
and searching the recorded stripe index by taking the object identification of the target object as a keyword, wherein the stripe index comprises the keyword.
6. A processing apparatus for stripe indexing, the apparatus being applied to a management node in an object storage system, comprising:
a determining unit configured to determine a target object to be deleted;
a searching unit for searching a stripe index of a stripe storing the target object;
the detection unit is used for detecting whether other object identifiers except the target object identifier are recorded in the searched strip index;
a setting unit configured to set a designation flag for the target object identifier if yes;
and the processing unit is used for deleting the strip index when determining that each object identifier recorded in the strip index is provided with the index mark.
7. The apparatus of claim 6, wherein the processing unit is further configured to delete the stripe index if the found stripe index only records the target object identification.
8. The apparatus according to claim 6, wherein the determining unit is specifically configured to determine, among locally recorded object indexes, an object index that satisfies a specified condition; and taking the object indicated by the determined object index as the object to be deleted.
9. The apparatus of claim 8, wherein the object index records at least an object write time;
the determining unit is specifically configured to determine a deletion time by subtracting a preset time from a current time when determining an object index satisfying a specified condition from locally recorded object indexes; searching the object index before the deleting time at the object writing time, and determining the searched object index as the object index meeting the specified condition.
10. The apparatus of claim 6, wherein the stripe index records at least an object identification;
the searching unit is specifically configured to search, in the recorded stripe indexes, the stripe index containing the keyword by using the object identifier of the target object as the keyword.
11. An object management device comprising a processor and a machine-readable storage medium storing machine-executable instructions executable by the processor to cause the method of any one of claims 1 to 5 to be performed.
12. A machine-readable storage medium storing machine-executable instructions which, when invoked and executed by a processor, cause the processor to perform the method of any one of claims 1 to 5.
CN201811015213.6A 2018-08-31 2018-08-31 Processing method, device and equipment for strip index Active CN110874182B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201811015213.6A CN110874182B (en) 2018-08-31 2018-08-31 Processing method, device and equipment for strip index

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201811015213.6A CN110874182B (en) 2018-08-31 2018-08-31 Processing method, device and equipment for strip index

Publications (2)

Publication Number Publication Date
CN110874182A CN110874182A (en) 2020-03-10
CN110874182B true CN110874182B (en) 2023-12-26

Family

ID=69715476

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201811015213.6A Active CN110874182B (en) 2018-08-31 2018-08-31 Processing method, device and equipment for strip index

Country Status (1)

Country Link
CN (1) CN110874182B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111625502B (en) * 2020-05-28 2024-02-23 浙江大华技术股份有限公司 Data reading method and device, storage medium and electronic device

Citations (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20060067119A (en) * 2004-12-14 2006-06-19 한국전자통신연구원 Crash recovery system and method for a distributed file server using object based storage
CN101393532A (en) * 2007-03-29 2009-03-25 赛门铁克公司 Removal of data from a single instancing data archive
US8549223B1 (en) * 2009-10-29 2013-10-01 Symantec Corporation Systems and methods for reclaiming storage space on striped volumes
CN103605726A (en) * 2013-11-15 2014-02-26 中安消技术有限公司 Method and system for accessing small files, control node and storage node
US9047312B1 (en) * 2011-03-30 2015-06-02 Amazon Technologies, Inc. System and method for deletion of extraneous delete marker objects
JP2015210815A (en) * 2014-04-25 2015-11-24 ネイバー コーポレーションＮＡＶＥＲ Ｃｏｒｐｏｒａｔｉｏｎ Database management method and database management system
CN105912664A (en) * 2016-04-11 2016-08-31 华为技术有限公司 Method and equipment for file processing
WO2017206436A1 (en) * 2016-06-03 2017-12-07 杭州海康威视数字技术股份有限公司 Method, device and system for recovering space on online hard drive
CN107562368A (en) * 2016-06-30 2018-01-09 杭州海康威视数字技术股份有限公司 A kind of data processing method and device
CN107704203A (en) * 2017-09-27 2018-02-16 郑州云海信息技术有限公司 It polymerize delet method, device, equipment and the computer-readable storage medium of big file
CN107870940A (en) * 2016-09-28 2018-04-03 杭州海康威视数字技术股份有限公司 A kind of file memory method and device
CN107958079A (en) * 2017-12-14 2018-04-24 郑州云海信息技术有限公司 Aggregate file delet method, system, device and readable storage medium storing program for executing
US10599344B1 (en) * 2004-05-13 2020-03-24 Primos Storage Technology Llc Transaction-based storage system and method that uses variable sized objects to store data

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9104717B2 (en) * 2013-01-31 2015-08-11 Futurewei Technologies, Inc. Distributed storage object delete

Patent Citations (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10599344B1 (en) * 2004-05-13 2020-03-24 Primos Storage Technology Llc Transaction-based storage system and method that uses variable sized objects to store data
KR20060067119A (en) * 2004-12-14 2006-06-19 한국전자통신연구원 Crash recovery system and method for a distributed file server using object based storage
CN101393532A (en) * 2007-03-29 2009-03-25 赛门铁克公司 Removal of data from a single instancing data archive
US8549223B1 (en) * 2009-10-29 2013-10-01 Symantec Corporation Systems and methods for reclaiming storage space on striped volumes
US9047312B1 (en) * 2011-03-30 2015-06-02 Amazon Technologies, Inc. System and method for deletion of extraneous delete marker objects
CN103605726A (en) * 2013-11-15 2014-02-26 中安消技术有限公司 Method and system for accessing small files, control node and storage node
JP2015210815A (en) * 2014-04-25 2015-11-24 ネイバー コーポレーションＮＡＶＥＲ Ｃｏｒｐｏｒａｔｉｏｎ Database management method and database management system
CN105912664A (en) * 2016-04-11 2016-08-31 华为技术有限公司 Method and equipment for file processing
WO2017206436A1 (en) * 2016-06-03 2017-12-07 杭州海康威视数字技术股份有限公司 Method, device and system for recovering space on online hard drive
CN107562368A (en) * 2016-06-30 2018-01-09 杭州海康威视数字技术股份有限公司 A kind of data processing method and device
CN107870940A (en) * 2016-09-28 2018-04-03 杭州海康威视数字技术股份有限公司 A kind of file memory method and device
CN107704203A (en) * 2017-09-27 2018-02-16 郑州云海信息技术有限公司 It polymerize delet method, device, equipment and the computer-readable storage medium of big file
CN107958079A (en) * 2017-12-14 2018-04-24 郑州云海信息技术有限公司 Aggregate file delet method, system, device and readable storage medium storing program for executing

Also Published As

Publication number Publication date
CN110874182A (en) 2020-03-10

Similar Documents

Publication Publication Date Title
US10331349B2 (en) Shrinking virtual hard disk image
US8224875B1 (en) Systems and methods for removing unreferenced data segments from deduplicated data systems
CN110399310B (en) Method and device for recovering storage space
CN107015985B (en) Data storage and acquisition method and device
US7577808B1 (en) Efficient backup data retrieval
CN110888837B (en) Object storage small file merging method and device
US20080162591A1 (en) Method of Logging Transactions and a Method of Reversing a Transaction
US7895164B1 (en) Efficient checkpoint process
CN110874182B (en) Processing method, device and equipment for strip index
CN110716924B (en) Method and device for deleting expired data
CN113157600A (en) Space allocation method of shingled hard disk, file storage system and server
CN109271097A (en) Data processing method, data processing equipment and server
US8825600B2 (en) File backup apparatus and method for backing up one or more directories
CN109165305B (en) Characteristic value storage and retrieval method and device
CN114281242B (en) Memory load balancing method, device and equipment
CN109783274B (en) Disk snapshot management method and device and readable storage medium
CN106648550B (en) Method and device for concurrently executing tasks
CN106528590B (en) Query method and device
CN110471623B (en) Hard disk file writing method, device, computer equipment and storage medium
CN108984343B (en) Virtual machine backup and storage management method based on content analysis
CN109992220B (en) Lock release method, device, equipment and medium
CN110196785B (en) Data backup management method and device and electronic equipment
JP2010044571A (en) Database device, data management method and program therefore
CN107704208B (en) Method, device and medium for repairing metadata
CN112764754A (en) File identifier management method, device and medium based on Unity3D

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant