CN116483271A - Method, system, device and storage medium for object storage life cycle management - Google Patents

Method, system, device and storage medium for object storage life cycle management Download PDF

Info

Publication number
CN116483271A
CN116483271A CN202310452373.1A CN202310452373A CN116483271A CN 116483271 A CN116483271 A CN 116483271A CN 202310452373 A CN202310452373 A CN 202310452373A CN 116483271 A CN116483271 A CN 116483271A
Authority
CN
China
Prior art keywords
objects
expired
memory database
bucket
traversing
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202310452373.1A
Other languages
Chinese (zh)
Inventor
程晓煜
陶桐桐
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Jinan Inspur Data Technology Co Ltd
Original Assignee
Jinan Inspur Data Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Jinan Inspur Data Technology Co Ltd filed Critical Jinan Inspur Data Technology Co Ltd
Priority to CN202310452373.1A priority Critical patent/CN116483271A/en
Publication of CN116483271A publication Critical patent/CN116483271A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0646—Horizontal data movement in storage systems, i.e. moving data in between storage devices or systems
    • G06F3/0647—Migration mechanisms
    • G06F3/0649—Lifecycle management
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0646—Horizontal data movement in storage systems, i.e. moving data in between storage devices or systems
    • G06F3/0652—Erasing, e.g. deleting, data cleaning, moving of data to a wastebasket
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The invention provides a method, a system, equipment and a storage medium for managing the life cycle of object storage, wherein the method comprises the following steps: detecting whether the in-memory database is empty in response to beginning the execution lifecycle; traversing all objects in the memory database and deleting the expired objects in response to the memory database not being empty; traversing a bucket on a bucket index shard in response to the memory database being empty; and responding to the state of the barrel as not executing, traversing the objects in the barrel, deleting the objects which have expired, and adding the objects which expire within a preset number of days into the memory database. The invention reduces the pressure of the metadata pool during the execution of the life cycle, reduces the influence on other services, improves the efficiency of the execution of the life cycle, and effectively improves the deletion speed of the expired object.

Description

Method, system, device and storage medium for object storage life cycle management
Technical Field
The present invention relates to the field of distributed object storage, and more particularly, to a method, system, device, and storage medium for object storage lifecycle management.
Background
The distributed object storage is suitable for storing unstructured data such as pictures, videos and the like, and simultaneously has the characteristics of high-speed direct access of block storage to a disk and distributed sharing of file storage. The bucket life cycle refers to that by setting a specified expiration rule for an object with a specified prefix or label in the bucket, the automatic expiration deleting operation for the object in the bucket can be realized. However, the life cycle of the bucket in the current object storage needs to traverse the objects in the bucket every day, and whether the objects are deleted or not is determined by comparing the uploading time of the objects. It is necessary to traverse the objects within the bucket every day when the bucket lifecycle setting expires. When the current life cycle is executed, the cluster pressure is high, and when objects in the cluster are more, the problem that the objects are not deleted because the objects are not traversed in time exists.
In the current object storage, the objects meeting the conditions in the bucket can be automatically deleted through the bucket life cycle. The lifecycle thread checks the barrel of the open lifecycle, scans the object which accords with the condition according to the set lifecycle rule, compares the current time and the object uploading time for the object which accords with the condition, and deletes the expired object. In the prior art, only objects which have expired are processed, the scanned objects which are about to expire are not recorded and processed, objects in the barrel are continuously traversed when the threads are executed next time, for example, the life cycle expiration time is set to be N, and the objects in the barrel are deleted on the N th day after the objects are uploaded after being enumerated N times because the life cycle thread execution period is one day. When the life cycle rule is set longer, serious cluster resource waste is caused, and cluster metadata pool pressure is increased.
Disclosure of Invention
In view of this, an object of the embodiments of the present invention is to provide a method, a system, a computer device and a computer readable storage medium for managing an object storage life cycle, which are capable of modifying a method that an object to be expired needs to be traversed every day before to be executed only once in a preset number of days (the preset number of days can be dynamically adjusted according to a cluster and an object size in the bucket) by caching the object to be expired in a memory, thereby reducing the pressure of a metadata pool during the execution of the life cycle, reducing the influence on other services, improving the efficiency of the execution of the life cycle, and effectively improving the deletion speed of the expired object.
Based on the above objects, an aspect of the embodiments of the present invention provides a method for object storage lifecycle management, including the steps of: detecting whether the in-memory database is empty in response to beginning the execution lifecycle; traversing all objects in the memory database and deleting the expired objects in response to the memory database not being empty; traversing a bucket on a bucket index shard in response to the memory database being empty; and responding to the state of the barrel as not executing, traversing the objects in the barrel, deleting the objects which have expired, and adding the objects which expire within a preset number of days into the memory database.
In some embodiments, the step of traversing all objects in the in-memory database and deleting objects that have expired comprises: and comparing the uploading time and the current time of each object in turn, deleting the expired object, and simultaneously deleting the expired object recorded in the memory.
In some embodiments, the step of traversing all objects in the in-memory database and deleting objects that have expired comprises: and in response to processing all the objects which have expired in the current bucket, modifying the state of the current bucket on the bucket index shard to be completed.
In some embodiments, the step of traversing all objects in the in-memory database and deleting objects that have expired comprises: and updating the caching days of the objects recorded in the memory in response to the fact that the number of the objects recorded in the memory is not zero.
In some embodiments, the adding the object that expires within a preset number of days to the in-memory database comprises: and adding the key value of the object expired in the preset days into a memory database in a mode of naming the expiration time, and modifying the execution state of the barrel to be finished.
In some embodiments, adding the key value of the expired object in the preset number of days to the memory database according to the manner of naming the expiration time includes: and sequencing the objects in the memory database according to the sequence of the expiration time.
In some implementations, the traversing the objects in the bucket includes: in response to the current object not expiring, traversing of objects arranged behind the object is stopped.
In another aspect of an embodiment of the present invention, there is provided a system for object storage lifecycle management, including: a detection module configured to detect whether the memory database is empty in response to starting execution of the lifecycle; the first traversing module is configured to respond to the fact that the memory database is not empty, traverse all objects in the memory database and delete objects which have expired; a second traversing module configured to traverse a bucket on a bucket index shard in response to the memory database being empty; and the execution module is configured to traverse the objects in the bucket, delete the objects which have expired and add the objects which have expired within a preset number of days to the memory database in response to the status of the bucket being unexecuted.
In yet another aspect of the embodiment of the present invention, there is also provided a computer apparatus, including: at least one processor; and a memory storing computer instructions executable on the processor, which when executed by the processor, perform the steps of the method as above.
In yet another aspect of the embodiments of the present invention, there is also provided a computer-readable storage medium storing a computer program which, when executed by a processor, implements the method steps as described above.
The invention has the following beneficial technical effects: by caching the object to be expired into the memory, the method that the expired object in the barrel needs to be traversed every day before is modified to be executed once in a preset number of days, so that the pressure of a metadata pool during the execution of the life cycle is reduced, the influence on other services is reduced, the execution efficiency of the life cycle is improved, and the deletion speed of the expired object is effectively improved.
Drawings
In order to more clearly illustrate the embodiments of the invention or the technical solutions in the prior art, the drawings that are necessary for the description of the embodiments or the prior art will be briefly described, it being obvious that the drawings in the following description are only some embodiments of the invention and that other embodiments may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic diagram of an embodiment of a method for object storage lifecycle management provided by the present invention;
FIG. 2 is a schematic diagram of an embodiment of a system for object store lifecycle management provided by the present invention;
FIG. 3 is a schematic hardware architecture diagram of an embodiment of a computer device for object storage lifecycle management provided by the present invention;
FIG. 4 is a schematic diagram of an embodiment of a computer storage medium for object storage lifecycle management provided by the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the following embodiments of the present invention will be described in further detail with reference to the accompanying drawings.
It should be noted that, in the embodiments of the present invention, all the expressions "first" and "second" are used to distinguish two entities with the same name but different entities or different parameters, and it is noted that the "first" and "second" are only used for convenience of expression, and should not be construed as limiting the embodiments of the present invention, and the following embodiments are not described one by one.
In a first aspect of the embodiments of the present invention, an embodiment of a method for object storage lifecycle management is presented. FIG. 1 is a schematic diagram of an embodiment of a method for object store lifecycle management provided by the present invention. As shown in fig. 1, the embodiment of the present invention includes the following steps:
s1, detecting whether an in-memory database is empty or not in response to starting an execution life cycle;
s2, responding to the fact that the memory database is not empty, traversing all objects in the memory database, and deleting the objects which have expired;
s3, responding to the fact that the memory database is empty, traversing the barrel on the barrel index partition; and
and S4, traversing the objects in the barrel in response to the condition that the barrel is not executed, deleting the objects which have expired, and adding the objects which have expired within a preset number of days into the memory database.
The object storage is a new network storage architecture, has the characteristics of SAN (Storage Area Network, storage local area network) high-speed direct access disk and the distributed sharing characteristics of NAS (Network Attached Storage ), and has wider application prospect and application value. Lifecycle management refers to the implementation of timed deletion of objects in a bucket or timed conversion of storage categories of objects by configuring specified rules. Bucket sharding refers to sharding of index objects of a bucket in an RGW (object storage gateway system) for recording objects in the bucket and object related information.
Aiming at the problem that objects are frequently enumerated in the current scheme, the embodiment of the invention provides the problems that the objects to be expired in the preset days are recorded in the memory database according to the object traversal result in the barrel on the basis of the previous life cycle thread, and only the memory database is required to be accessed when the life cycle thread is executed in the preset days, so that the objects in the barrel are not required to be traversed, the problem of frequent enumeration of the objects is reduced, and the efficiency of the object expiration deletion is improved.
In response to beginning an execution lifecycle, it is detected whether the in-memory database is empty.
And traversing all objects in the memory database and deleting the objects which have expired in response to the memory database not being empty.
In some embodiments, the step of traversing all objects in the in-memory database and deleting objects that have expired comprises: and comparing the uploading time and the current time of each object in turn, deleting the expired object, and simultaneously deleting the expired object recorded in the memory.
In some embodiments, the step of traversing all objects in the in-memory database and deleting objects that have expired comprises: and in response to processing all the objects which have expired in the current bucket, modifying the state of the current bucket on the bucket index shard to be completed.
In some embodiments, the step of traversing all objects in the in-memory database and deleting objects that have expired comprises: and updating the caching days of the objects recorded in the memory in response to the fact that the number of the objects recorded in the memory is not zero.
If the memory database is not empty, traversing the object to be deleted, comparing the uploading time and the current time of the object, deleting the object and the record in the memory if the object is expired, and deleting all expired objects of the barrel if the object is not expired. After the bucket is processed, the bucket state on the bucket index partition is modified to be finished, and if the number of recorded objects is not empty, the caching days of the objects to be deleted are updated at the same time.
And traversing the bucket on the bucket index shard in response to the in-memory database being empty.
And traversing the objects in the bucket, deleting the objects which have expired and adding the objects which expire within a preset number of days into the memory database in response to the status of the bucket being unexecuted.
In some embodiments, the adding the object that expires within a preset number of days to the in-memory database comprises: and adding the key value of the object expired in the preset days into a memory database in a mode of naming the expiration time, and modifying the execution state of the barrel to be finished.
In some embodiments, adding the key value of the expired object in the preset number of days to the memory database according to the manner of naming the expiration time includes: and sequencing the objects in the memory database according to the sequence of the expiration time.
In some implementations, the traversing the objects in the bucket includes: in response to the current object not expiring, traversing of objects arranged behind the object is stopped.
If the memory database is empty or the barrel in the memory is processed, traversing the barrel on the barrel index fragment, checking the state of the barrel, if the barrel is not executed and the barrel has no cache record of the object to be deleted in other nodes, traversing the object in the barrel, deleting the expired object, adding the expired object key (key value) in the preset days into the node memory database in a mode of naming the expiration time, modifying the execution state of the barrel to be finished, recording the cache node as the current node and the cache days as the preset days.
To reduce memory usage, the object to be deleted for each bucket cache is recorded on only one node, and each node processes the bucket of the node cache data as the lifecycle thread executes. The keys in the memory database are named according to the expiration time of the objects, so that the objects in the database are ordered according to the expiration time, when the objects in the database are traversed, and when the current objects are not expired, the following objects are not expired, and the traversal time is reduced.
When the life cycle thread is executed, after the barrel life cycle configuration is obtained, checking whether an object to be expired of the barrel is reserved in the memory database, and if the object to be expired exists, deleting the object to be expired according to a result in the memory database; if the object is not present, scanning the object conforming to the rule in the barrel according to the barrel index fragment, comparing the uploading time of the object, if the object is expired, directly deleting the object, if the object is not expired and the expiration time is less than N days, adding the object into a memory database of the node, and simultaneously recording the node where the object to be deleted is cached and the caching days in the barrel life cycle related metadata.
According to the embodiment of the invention, the method that the expired object in the barrel needs to be traversed every day before is modified by caching the expired object in the memory, so that each barrel needs to be executed only once in a preset number of days, the pressure of a metadata pool during the execution of the life cycle is reduced, the influence on other services is reduced, the execution efficiency of the life cycle is improved, and the deletion speed of the expired object is effectively improved.
It should be noted that, the steps in the embodiments of the method for managing the object storage lifecycle may be intersected, replaced, added and deleted, so that the method for managing the object storage lifecycle by the reasonable permutation and combination should also fall within the protection scope of the present invention, and the protection scope of the present invention should not be limited to the embodiments.
Based on the above object, a second aspect of the embodiments of the present invention proposes a system for object storage lifecycle management. As shown in fig. 2, the system 200 includes the following modules: a detection module configured to detect whether the memory database is empty in response to starting execution of the lifecycle; the first traversing module is configured to respond to the fact that the memory database is not empty, traverse all objects in the memory database and delete objects which have expired; a second traversing module configured to traverse a bucket on a bucket index shard in response to the memory database being empty; and the execution module is configured to traverse the objects in the bucket, delete the objects which have expired and add the objects which have expired within a preset number of days to the memory database in response to the status of the bucket being unexecuted.
In some embodiments, the first traversal module is configured to: and comparing the uploading time and the current time of each object in turn, deleting the expired object, and simultaneously deleting the expired object recorded in the memory.
In some embodiments, the first traversal module is configured to: and in response to processing all the objects which have expired in the current bucket, modifying the state of the current bucket on the bucket index shard to be completed.
In some embodiments, the first traversal module is configured to: and updating the caching days of the objects recorded in the memory in response to the fact that the number of the objects recorded in the memory is not zero.
In some embodiments, the execution module is configured to: and adding the key value of the object expired in the preset days into a memory database in a mode of naming the expiration time, and modifying the execution state of the barrel to be finished.
In some embodiments, the execution module is configured to: and sequencing the objects in the memory database according to the sequence of the expiration time.
In some embodiments, the execution module is configured to: in response to the current object not expiring, traversing of objects arranged behind the object is stopped.
According to the embodiment of the invention, the method that the expired object in the barrel needs to be traversed every day before is modified by caching the expired object in the memory, so that each barrel needs to be executed only once in a preset number of days, the pressure of a metadata pool during the execution of the life cycle is reduced, the influence on other services is reduced, the execution efficiency of the life cycle is improved, and the deletion speed of the expired object is effectively improved.
In view of the above object, a third aspect of the embodiments of the present invention provides a computer device, including: at least one processor; and a memory storing computer instructions executable on the processor, the instructions being executable by the processor to perform the steps of: s1, detecting whether an in-memory database is empty or not in response to starting an execution life cycle; s2, responding to the fact that the memory database is not empty, traversing all objects in the memory database, and deleting the objects which have expired; s3, responding to the fact that the memory database is empty, traversing the barrel on the barrel index partition; and S4, traversing the objects in the barrel in response to the condition that the barrel is not executed, deleting the objects which have expired, and adding the objects which have expired within a preset number of days into the memory database.
In some embodiments, the step of traversing all objects in the in-memory database and deleting objects that have expired comprises: and comparing the uploading time and the current time of each object in turn, deleting the expired object, and simultaneously deleting the expired object recorded in the memory.
In some embodiments, the step of traversing all objects in the in-memory database and deleting objects that have expired comprises: and in response to processing all the objects which have expired in the current bucket, modifying the state of the current bucket on the bucket index shard to be completed.
In some embodiments, the step of traversing all objects in the in-memory database and deleting objects that have expired comprises: and updating the caching days of the objects recorded in the memory in response to the fact that the number of the objects recorded in the memory is not zero.
In some embodiments, the adding the object that expires within a preset number of days to the in-memory database comprises: and adding the key value of the object expired in the preset days into a memory database in a mode of naming the expiration time, and modifying the execution state of the barrel to be finished.
In some embodiments, adding the key value of the expired object in the preset number of days to the memory database according to the manner of naming the expiration time includes: and sequencing the objects in the memory database according to the sequence of the expiration time.
In some implementations, the traversing the objects in the bucket includes: in response to the current object not expiring, traversing of objects arranged behind the object is stopped.
According to the embodiment of the invention, the method that the expired object in the barrel needs to be traversed every day before is modified by caching the expired object in the memory, so that each barrel needs to be executed only once in a preset number of days, the pressure of a metadata pool during the execution of the life cycle is reduced, the influence on other services is reduced, the execution efficiency of the life cycle is improved, and the deletion speed of the expired object is effectively improved.
As shown in fig. 3, a hardware structure diagram of an embodiment of the computer device for object storage lifecycle management according to the present invention is shown.
Taking the example of the device shown in fig. 3, a processor 301 and a memory 302 are included in the device.
The processor 301 and the memory 302 may be connected by a bus or otherwise, for example in fig. 3.
The memory 302 is used as a non-volatile computer readable storage medium for storing non-volatile software programs, non-volatile computer executable programs, and modules, such as program instructions/modules corresponding to the methods of object storage lifecycle management in the embodiments of the present application. The processor 301 executes various functional applications of the server and data processing, i.e., a method of implementing object storage lifecycle management, by running nonvolatile software programs, instructions, and modules stored in the memory 302.
Memory 302 may include a storage program area that may store an operating system, at least one application program required for functionality, and a storage data area; the storage data area may store data created according to the use of the method of object storage lifecycle management, and the like. In addition, memory 302 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid-state storage device. In some embodiments, memory 302 may optionally include memory located remotely from processor 301, which may be connected to the local module via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
Computer instructions 303 corresponding to one or more methods of object storage lifecycle management are stored in memory 302 that, when executed by processor 301, perform the methods of object storage lifecycle management in any of the method embodiments described above.
The method for managing the life cycle of the object storage comprises the following steps: detecting whether the in-memory database is empty in response to beginning the execution lifecycle; traversing all objects in the memory database and deleting the expired objects in response to the memory database not being empty; traversing a bucket on a bucket index shard in response to the memory database being empty; and responding to the state of the barrel as not executing, traversing the objects in the barrel, deleting the objects which have expired, and adding the objects which expire within a preset number of days into the memory database.
In some embodiments, the step of traversing all objects in the in-memory database and deleting objects that have expired comprises: and comparing the uploading time and the current time of each object in turn, deleting the expired object, and simultaneously deleting the expired object recorded in the memory.
In some embodiments, the step of traversing all objects in the in-memory database and deleting objects that have expired comprises: and in response to processing all the objects which have expired in the current bucket, modifying the state of the current bucket on the bucket index shard to be completed.
In some embodiments, the step of traversing all objects in the in-memory database and deleting objects that have expired comprises: and updating the caching days of the objects recorded in the memory in response to the fact that the number of the objects recorded in the memory is not zero.
In some embodiments, the adding the object that expires within a preset number of days to the in-memory database comprises: and adding the key value of the object expired in the preset days into a memory database in a mode of naming the expiration time, and modifying the execution state of the barrel to be finished.
In some embodiments, adding the key value of the expired object in the preset number of days to the memory database according to the manner of naming the expiration time includes: and sequencing the objects in the memory database according to the sequence of the expiration time.
In some implementations, the traversing the objects in the bucket includes: in response to the current object not expiring, traversing of objects arranged behind the object is stopped.
Any one embodiment of a computer device that performs the above method of object store lifecycle management may achieve the same or similar effects as any of the previously described method embodiments corresponding thereto.
The present invention also provides a computer readable storage medium storing a computer program which when executed by a processor performs a method of object storage lifecycle management.
As shown in fig. 4, a schematic diagram of an embodiment of the computer storage medium for managing the life cycle of the object storage according to the present invention is provided. Taking a computer storage medium as shown in fig. 4 as an example, the computer readable storage medium 401 stores a computer program 402 that when executed by a processor performs the above method.
Finally, it should be noted that, as will be understood by those skilled in the art, implementing all or part of the above-described methods in the embodiments may be implemented by a computer program to instruct related hardware, and the program of the method for managing the lifecycle of the object storage may be stored in a computer readable storage medium, where the program may include the steps of the embodiments of the methods described above when executed. The storage medium of the program may be a magnetic disk, an optical disk, a read-only memory (ROM), a random-access memory (RAM), or the like. The computer program embodiments described above may achieve the same or similar effects as any of the method embodiments described above.
The foregoing is an exemplary embodiment of the present disclosure, but it should be noted that various changes and modifications could be made herein without departing from the scope of the disclosure as defined by the appended claims. The functions, steps and/or actions of the method claims in accordance with the disclosed embodiments described herein need not be performed in any particular order. Furthermore, although elements of the disclosed embodiments may be described or claimed in the singular, the plural is contemplated unless limitation to the singular is explicitly stated.
It should be understood that as used herein, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly supports the exception. It should also be understood that "and/or" as used herein is meant to include any and all possible combinations of one or more of the associated listed items.
The foregoing embodiment of the present invention has been disclosed with reference to the number of embodiments for the purpose of description only, and does not represent the advantages or disadvantages of the embodiments.
It will be understood by those skilled in the art that all or part of the steps for implementing the above embodiments may be implemented by hardware, or may be implemented by a program for instructing relevant hardware, and the program may be stored in a computer readable storage medium, where the storage medium may be a read-only memory, a magnetic disk or an optical disk, etc.
Those of ordinary skill in the art will appreciate that: the above discussion of any embodiment is merely exemplary and is not intended to imply that the scope of the disclosure of embodiments of the invention, including the claims, is limited to such examples; combinations of features of the above embodiments or in different embodiments are also possible within the idea of an embodiment of the invention, and many other variations of the different aspects of the embodiments of the invention as described above exist, which are not provided in detail for the sake of brevity. Therefore, any omission, modification, equivalent replacement, improvement, etc. of the embodiments should be included in the protection scope of the embodiments of the present invention.

Claims (10)

1. A method of object store lifecycle management, comprising the steps of:
detecting whether the in-memory database is empty in response to beginning the execution lifecycle;
traversing all objects in the memory database and deleting the expired objects in response to the memory database not being empty;
traversing a bucket on a bucket index shard in response to the memory database being empty; and
and traversing the objects in the bucket, deleting the objects which have expired and adding the objects which expire within a preset number of days into the memory database in response to the status of the bucket being unexecuted.
2. The method of object storage lifecycle management as recited in claim 1, wherein traversing all objects in the in-memory database and deleting objects that have expired comprises:
and comparing the uploading time and the current time of each object in turn, deleting the expired object, and simultaneously deleting the expired object recorded in the memory.
3. The method of object storage lifecycle management as recited in claim 2, wherein traversing all objects in the in-memory database and deleting objects that have expired comprises:
and in response to processing all the objects which have expired in the current bucket, modifying the state of the current bucket on the bucket index shard to be completed.
4. The method of object storage lifecycle management as recited in claim 2, wherein traversing all objects in the in-memory database and deleting objects that have expired comprises:
and updating the caching days of the objects recorded in the memory in response to the fact that the number of the objects recorded in the memory is not zero.
5. The method of claim 1, wherein adding an object that expires within a preset number of days to the in-memory database comprises:
and adding the key value of the object expired in the preset days into a memory database in a mode of naming the expiration time, and modifying the execution state of the barrel to be finished.
6. The method of claim 5, wherein adding the key value of the expired object in the preset number of days to the memory database according to the manner of naming the expiration time comprises:
and sequencing the objects in the memory database according to the sequence of the expiration time.
7. The method of object storage lifecycle management as recited in claim 6, wherein traversing the objects in the bucket comprises:
in response to the current object not expiring, traversing of objects arranged behind the object is stopped.
8. A system for object storage lifecycle management, comprising:
a detection module configured to detect whether the memory database is empty in response to starting execution of the lifecycle;
the first traversing module is configured to respond to the fact that the memory database is not empty, traverse all objects in the memory database and delete objects which have expired;
a second traversing module configured to traverse a bucket on a bucket index shard in response to the memory database being empty; and
and the execution module is configured to traverse the objects in the bucket, delete the objects which have expired and add the objects which have expired within a preset number of days to the memory database in response to the status of the bucket being unexecuted.
9. A computer device, comprising:
at least one processor; and
a memory storing computer instructions executable on the processor, which when executed by the processor, perform the steps of the method of any one of claims 1-7.
10. A computer readable storage medium storing a computer program, characterized in that the computer program when executed by a processor implements the steps of the method of any one of claims 1-7.
CN202310452373.1A 2023-04-21 2023-04-21 Method, system, device and storage medium for object storage life cycle management Pending CN116483271A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310452373.1A CN116483271A (en) 2023-04-21 2023-04-21 Method, system, device and storage medium for object storage life cycle management

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310452373.1A CN116483271A (en) 2023-04-21 2023-04-21 Method, system, device and storage medium for object storage life cycle management

Publications (1)

Publication Number Publication Date
CN116483271A true CN116483271A (en) 2023-07-25

Family

ID=87217359

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310452373.1A Pending CN116483271A (en) 2023-04-21 2023-04-21 Method, system, device and storage medium for object storage life cycle management

Country Status (1)

Country Link
CN (1) CN116483271A (en)

Similar Documents

Publication Publication Date Title
US11574290B2 (en) Data processing method and apparatus, computer device, and storage medium
US11397538B2 (en) Data migration method and apparatus
CN107943718B (en) Method and device for cleaning cache file
CN111198856B (en) File management method, device, computer equipment and storage medium
CN110727685B (en) Data compression method, equipment and storage medium based on Cassandra database
CN105306465A (en) Website secure access realization method and apparatus
CN111339033A (en) Method, system and terminal for realizing object storage life cycle
US10509767B2 (en) Systems and methods for managing snapshots of a file system volume
CN111737254B (en) Method and device for improving application cache utilization rate
CN114968845A (en) Cache processing method, system, equipment and storage medium
CN112528098A (en) Data query method, system, electronic equipment and storage medium
CN115952195A (en) Block chain data tracing query optimization method
CN116483271A (en) Method, system, device and storage medium for object storage life cycle management
US20230252029A1 (en) On-board data storage method and system
CN111309264B (en) Method, system, device and medium for making directory quota compatible with snapshot
CN107341203A (en) The access control and optimiged index method and apparatus of a kind of distributed search engine
CN115827702B (en) Software white list query method based on bloom filter
CN105653367A (en) Method and device for traversing timer
CN112131234A (en) Method, system, equipment and medium for optimizing data cache concurrency
CN113609160A (en) B + tree traversal method, system, storage medium and equipment
CN112579006A (en) Data storage life cycle management method and system
CN113177031B (en) Processing method and device for database shared cache, electronic equipment and medium
CN111881092B (en) Method and device for merging files based on cassandra database
CN115150469B (en) Domain name resolution result storage method and device, electronic equipment and storage medium
CN114221934B (en) Domain name resolution caching method, DNS server and computer readable storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination