CN117349255A - Object data processing method and device - Google Patents
Object data processing method and device Download PDFInfo
- Publication number
- CN117349255A CN117349255A CN202311183826.1A CN202311183826A CN117349255A CN 117349255 A CN117349255 A CN 117349255A CN 202311183826 A CN202311183826 A CN 202311183826A CN 117349255 A CN117349255 A CN 117349255A
- Authority
- CN
- China
- Prior art keywords
- object data
- request message
- life cycle
- identification
- storage device
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003672 processing method Methods 0.000 title abstract description 9
- 238000003860 storage Methods 0.000 claims abstract description 452
- 238000000034 method Methods 0.000 claims abstract description 76
- 238000012217 deletion Methods 0.000 claims abstract description 66
- 230000037430 deletion Effects 0.000 claims abstract description 66
- 239000012634 fragment Substances 0.000 claims description 29
- 238000012545 processing Methods 0.000 claims description 25
- 230000004044 response Effects 0.000 claims description 8
- 238000012544 monitoring process Methods 0.000 abstract description 5
- 238000007726 management method Methods 0.000 description 178
- 238000004590 computer program Methods 0.000 description 15
- 230000008569 process Effects 0.000 description 13
- 238000010586 diagram Methods 0.000 description 12
- 230000006870 function Effects 0.000 description 9
- 206010011906 Death Diseases 0.000 description 8
- 230000003287 optical effect Effects 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 5
- 239000004065 semiconductor Substances 0.000 description 4
- 230000002776 aggregation Effects 0.000 description 3
- 238000004220 aggregation Methods 0.000 description 3
- 238000013500 data storage Methods 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 125000004122 cyclic group Chemical group 0.000 description 2
- 230000003203 everyday effect Effects 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 239000004973 liquid crystal related substance Substances 0.000 description 2
- 230000009471 action Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000000295 complement effect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 229910044991 metal oxide Inorganic materials 0.000 description 1
- 150000004706 metal oxides Chemical class 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000008520 organization Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/182—Distributed file systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/16—File or folder operations, e.g. details of user interfaces specifically adapted to file systems
- G06F16/162—Delete operations
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/061—Improving I/O performance
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0646—Horizontal data movement in storage systems, i.e. moving data in between storage devices or systems
- G06F3/0647—Migration mechanisms
- G06F3/0649—Lifecycle management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0646—Horizontal data movement in storage systems, i.e. moving data in between storage devices or systems
- G06F3/0652—Erasing, e.g. deleting, data cleaning, moving of data to a wastebasket
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/067—Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application discloses an object data processing method and device, which are used for realizing more convenient monitoring of an object storage system on the life cycle of object data, further realizing quick and batch deletion of the object data with the end of the life cycle, and improving the deletion efficiency of the object data and the system performance. The method provided by the application comprises the following steps: determining that object data ending the life cycle exists based on a preset life cycle management index; the life cycle management index and the identification of the object data have a corresponding relation, and the identification of the object data corresponding to the same life cycle management index is the identification of the object data written into the object storage device in the same unit time; acquiring an identifier of the object data of the end of the life cycle corresponding to the life cycle management index; and executing deleting operation for the object data of the life cycle ending based on the identification of the object data of the life cycle ending corresponding to the life cycle management index.
Description
Technical Field
The application relates to the technical field of distributed cloud storage, in particular to an object data processing method and device.
Background
A distributed file storage system (Ceph) is a widely adopted, automatically re-balanced and automatically recoverable distributed file storage system, and is capable of providing block device storage, file system storage and object storage functions, wherein the Ceph can provide an object storage interface through an object storage Gateway (RGW) thereof, provide a user with convenient and quick object data operation, and write and delete object data into an object storage device (Object Storage Device, OSD) through the RGW. In the Ceph system, all object data are stored in a storage Bucket (Bucket), and for a client, the Bucket and the object data are only two layers, and the object storage has good storage performance, so that a file system has good sharing performance, and the operation of the object data is simpler, thereby being widely applied.
Disclosure of Invention
The embodiment of the application provides an object data processing method and device, which are used for realizing more convenient monitoring of an object storage system on the life cycle of object data, further realizing quick and batch deletion of the object data with the end of the life cycle, and improving the deletion efficiency of the object data and the system performance.
At the object storage gateway side, the method for processing object data provided by the embodiment of the application includes:
determining that object data ending the life cycle exists based on a preset life cycle management index; the life cycle management index and the identification of the object data have a corresponding relation, and the identification of the object data corresponding to the same life cycle management index is the identification of the object data written into the object storage device in the same unit time;
acquiring an identifier of the object data of the end of the life cycle corresponding to the life cycle management index;
and executing deleting operation for the object data of the life cycle ending based on the identification of the object data of the life cycle ending corresponding to the life cycle management index.
In the embodiment of the application, based on a preset life cycle management index, determining that object data ending the life cycle exists; the life cycle management index and the identification of the object data have a corresponding relation, and the identification of the object data corresponding to the same life cycle management index is the identification of the object data written into the object storage device in the same unit time; after the identification of the object data of the lifecycle ending corresponding to the lifecycle management index is obtained, the deleting operation of the object data of the lifecycle ending is executed based on the identification of the object data of the lifecycle ending corresponding to the lifecycle management index. Therefore, the embodiment of the application can realize more convenient monitoring of the life cycle of the object data through the life cycle management index, and when the object data with the life cycle ended exists, the object data with the life cycle ended can be rapidly deleted in batches based on the identification of the object data with the life cycle ended corresponding to the life cycle management index, so that the object data deleting efficiency is improved, the system performance occupied by the operation of deleting the object data is reduced, and the performance of writing the object data is further improved.
In some embodiments, obtaining the identification of the object data of the end of the lifecycle corresponding to the lifecycle management index includes:
sending a first request message for reading a life cycle management index to the object storage device, wherein the first request message is used for requesting the object storage device and providing an identification of object data corresponding to the life cycle management index;
and receiving the identification of the object data corresponding to the life cycle management index returned by the object storage device aiming at the first request message.
In some embodiments, based on the identification of the end-of-life object data corresponding to the life-cycle management index, performing a delete operation for the end-of-life object data includes:
sending a second request message for deleting the head objects of the object data in batches to the object storage device, wherein the second request message is used for requesting the object storage device to delete the head objects of the object data of which the life cycle is finished;
receiving a header object completion message returned by the object storage device for deleting the object data of which the life cycle is finished aiming at the second request message;
Sending a third request message for deleting the corresponding relation between the identification of the object data and the storage bucket index to the object storage device, wherein the third request message is used for requesting the object storage device to delete the corresponding relation between the identification of the object data ending in the life cycle and the storage bucket index, and the object data ending in the life cycle is stored in the storage bucket corresponding to the storage bucket index;
receiving a corresponding relation completion message between the identifier of the returned deleted object data and the bucket index of the object storage device aiming at the third request message;
sending a fourth request message for deleting the fragmented object of the object data to the object storage device, wherein the fourth request message is used for requesting the object storage device to delete the fragmented object of the object data of which the life cycle is finished;
receiving a fragment object completion message of deleting object data returned by the object storage device aiming at the fourth request message;
and sending a fifth request message for deleting the corresponding relation between the identification of the object data and the life cycle management index to the object storage device, wherein the fifth request message is used for requesting the object storage device to delete the corresponding relation between the identification of the object data ending the life cycle and the life cycle management index.
In some embodiments, the method further comprises:
when object data to be written exist, a sixth request message for writing the object data is sent to the object storage device, wherein the sixth request message is used for requesting the object storage device to execute a storage operation of a head object and a fragment object of the object data to be written;
receiving an object data writing completion message returned by the object storage device aiming at the sixth request message;
a seventh request message for establishing the corresponding relation between the identification of the object data and the storage bucket index is sent to the object storage equipment, and the seventh request message is used for requesting the object storage equipment to establish the corresponding relation between the storage bucket index and the identification of the object data;
receiving a corresponding relation completion message between the identification of the returned establishment object data and the storage bucket index of the object storage device aiming at the seventh request message;
an eighth request message for establishing the corresponding relation between the identification of the object data and the life cycle management index is sent to the object storage device, and the eighth request message is used for requesting the object storage device to establish the corresponding relation between the life cycle management index and the identification of the object data;
And receiving a corresponding relation completion message between the identification of the returned establishment object data and the life cycle management index by the object storage device aiming at the eighth request message.
Accordingly, on the object storage device side, the method for processing object data provided in the embodiment of the present application includes:
receiving a first request message for reading a life cycle management index sent by an object storage gateway, wherein the first request message is used for requesting a first object storage device and providing an identification of object data corresponding to the life cycle management index; the life cycle management index and the identification of the object data have a corresponding relation, and the identification of the object data corresponding to the same life cycle management index is the identification of the object data written into the object storage device in the same unit time;
and returning the identification of the object data corresponding to the life cycle management index to the object storage gateway aiming at the first request message.
In some embodiments, the method further comprises:
receiving a second request message of batch deleting the head objects of the object data sent by the object storage gateway, wherein the second request message is used for requesting the first object storage device to delete the head objects of the object data with the end of the life cycle;
Executing the operation of deleting the head object of the object data aiming at the second request message, and returning a head object completion message for deleting the object data of which the life cycle is finished to the object storage gateway;
receiving a third request message sent by the object storage gateway for deleting the corresponding relation between the identification of the object data and the storage bucket index, wherein the third request message is used for requesting the first object storage device to delete the corresponding relation between the identification of the object data ending in the life cycle and the storage bucket index, and the object data ending in the life cycle is stored in the storage bucket corresponding to the storage bucket index;
executing the operation of deleting the corresponding relation between the identification of the object data and the storage bucket index aiming at the third request message, and returning a corresponding relation completion message between the identification of the object data and the storage bucket index to the object storage gateway;
receiving a fourth request message of deleting the fragmented object of the object data sent by the object storage gateway, wherein the fourth request message is used for requesting the first object storage device to delete the fragmented object of the object data with the end of the life cycle;
Executing the operation of deleting the fragmented object of the object data aiming at the fourth request message, and returning a fragmented object completion message of deleting the object data to the object storage gateway;
receiving a fifth request message which is sent by the object storage gateway and used for deleting the corresponding relation between the identification of the object data and the lifecycle management index, wherein the fifth request message is used for requesting the object storage device to delete the corresponding relation between the identification of the object data ending the lifecycle and the lifecycle management index;
and executing the operation of deleting the corresponding relation between the identification of the object data and the life cycle management index aiming at the fifth request message.
In some embodiments, the method further comprises:
sending a batch deletion slave request message to a second object storage device, wherein the second object storage device is used for storing backup data of related contents of object data stored in the first object storage device, and the slave request message is used for requesting the second object storage device to delete the backup data of related contents of the object data which needs to be deleted by the first object storage device; the related content comprises one or a combination of the following contents: the method comprises the steps of enabling a head object of object data, a fragment object of the object data, a corresponding relation between an identification of the object data and a storage bucket index, and a corresponding relation between an identification of the object data and a life cycle management index;
And receiving a response message returned by the second object storage device for the request message.
In some embodiments, the method further comprises:
receiving a sixth request message for writing object data sent by the object storage gateway, wherein the sixth request message is used for requesting the first object storage device to execute a storage operation for a head object and a fragment object of the object data to be written;
executing operations of writing the head object and the fragment object of the object data aiming at the sixth request message, and returning an object data writing completion message to the object storage gateway;
receiving a seventh request message which is sent by the object storage gateway and used for establishing the corresponding relation between the identification of the object data and the storage bucket index, wherein the seventh request message is used for requesting the first object storage equipment to establish the corresponding relation between the storage bucket index and the identification of the object data;
executing the operation of establishing the corresponding relation between the identification of the object data and the storage bucket index aiming at the seventh request message, and returning a corresponding relation completion message between the identification of the object data and the storage bucket index to the object storage gateway;
Receiving an eighth request message which is sent by the object storage gateway and used for establishing the corresponding relation between the identification of the object data and the life cycle management index, wherein the eighth request message is used for requesting the first object storage device to establish the corresponding relation between the life cycle management index and the identification of the object data;
and executing the operation of establishing the corresponding relation between the identification of the object data and the life cycle management index aiming at the eighth request message, and returning a corresponding relation completion message between the identification of the object data and the life cycle management index to the object storage gateway.
Another embodiment of the present application provides an electronic device, including a memory, where the memory is configured to store program instructions, and a processor configured to invoke the program instructions stored in the memory, and execute any of the methods according to the obtained program.
Furthermore, according to an embodiment, for example, a computer program product for a computer is provided, comprising software code portions for performing the steps of the method defined above, when said product is run on a computer. The computer program product may include a computer-readable medium having software code portions stored thereon. Furthermore, the computer program product may be directly loaded into the internal memory of the computer and/or transmitted via the network by at least one of an upload procedure, a download procedure and a push procedure.
Another embodiment of the present application provides a computer-readable storage medium storing computer-executable instructions for causing a computer to perform any of the methods described above.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the description of the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a schematic diagram of a celp distributed object storage system architecture according to an embodiment of the present application;
fig. 2 is a schematic diagram of an architecture between a master OSD and a slave OSD according to an embodiment of the present application;
FIG. 3 is a schematic diagram of a flow chart of writing object data into an OSD by RGW according to an embodiment of the present application;
fig. 4 is a schematic flow chart of deleting object data in OSD by RGW control according to the embodiment of the present application;
fig. 5 is a schematic flow chart of main OSD and deleting object data from OSD according to an embodiment of the present application;
FIG. 6 is a general flow chart of an object data processing method on the RGW side according to an embodiment of the present application;
Fig. 7 is a general flow chart of an object data processing method on the OSD side according to the embodiment of the present application;
fig. 8 is a schematic structural diagram of an electronic device according to an embodiment of the present application;
FIG. 9 is a schematic structural diagram of an object data processing device on the RGW side according to an embodiment of the present application;
fig. 10 is a schematic structural diagram of an object data processing device on the OSD side according to an embodiment of the present application.
Detailed Description
The following description of the technical solutions in the embodiments of the present application will be made clearly and completely with reference to the accompanying drawings in the embodiments of the present application, and it is apparent that the described embodiments are only some embodiments of the present application, but not all embodiments. All other embodiments, which can be made by one of ordinary skill in the art without undue burden from the present disclosure, are within the scope of the present disclosure.
The embodiment of the application provides an object data processing method and device, which are used for realizing more convenient monitoring of an object storage system on the life cycle of object data, further realizing quick and batch deletion of the object data with the end of the life cycle, and improving the deletion efficiency of the object data and the system performance.
The method and the device are based on the same application, and because the principles of solving the problems by the method and the device are similar, the implementation of the device and the method can be referred to each other, and the repetition is not repeated.
The terms first, second and the like in the description and in the claims of the embodiments and in the above-described figures, if any, are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments described herein may be implemented in other sequences than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
The following examples and embodiments are to be construed as illustrative only. Although the specification may refer to "an", "one", or "some" example or embodiment(s) at several points, this does not mean that each such reference is related to the same example or embodiment, nor that the feature is applicable to only a single example or embodiment. Individual features of different embodiments may also be combined to provide further embodiments. Furthermore, terms such as "comprising" and "including" should be understood not to limit the described embodiments to consist of only those features already mentioned; such examples and embodiments may also include features, structures, units, modules, etc. that are not specifically mentioned.
Various embodiments of the present application are described in detail below with reference to the drawings attached hereto. It should be noted that, the display sequence of the embodiments of the present application only represents the sequence of the embodiments, and does not represent the advantages or disadvantages of the technical solutions provided by the embodiments.
The Ceph distributed object storage system employs a decentralised storage architecture, see fig. 1, comprising an object storage Gateway (RGW) and an object storage device (Object Storage Device, OSD), the system performance can be linearly increased with the number of nodes. Object storage is very widely used in cloud computing, network disks, mirror warehouses, and view storage. In view business scenes, pictures are generally generated by capturing through a monitoring bayonet, the pictures can be continuously and stably written into an object storage device, and the storage time of the pictures also has specific requirements, generally 1 month. In this business scenario, the object storage device is generally at a constant capacity water level, 80% or 90%, the picture writing speed is also relatively constant, there is an equal amount of picture writing per second for a single node, and after the storage time of the picture expires, there is an equal proportion of pictures to be deleted. The system resources consumed by deleting the pictures are basically consistent with the system resources consumed by writing the pictures, and the performance of the IOPS (Input/Output Operations Per Second) of the whole cluster is fixed, so that the picture writing speed of the storage system is greatly reduced due to the equal proportion of deleting the pictures. Performance optimization for deleted pictures greatly improves the performance specification of objects stored in a full specification loop overlay scene. Both the writing and deleting operations of the object are provided by the object storage service, and the performance of the object storage service is constant, and the deleting also consumes the performance of the object storage service, so that when the writing of the object operation is performed under full pressure, the deleting operation competes with the performance of the object storage service after the equal proportion deleting operation is performed, and the writing performance is reduced.
For example, for an S3 (Simple Storage Service ) object, as a small object, the S3 object storage is to store data in units of the S3 object, a common hard disk 1TiB is required to store about 1 million small objects if a 1KiB object is stored, however, the metadata pressure of a large object and a small object is the same, and if a 1GiB object is stored, the metadata pressure of a 1KiB small object is 10 ten thousand times that of a 1GiB object, so that the small object has higher performance requirements on storage service and is more likely to generate performance bottlenecks.
If the performance of the delete operation can be optimized, i.e., the performance of the delete operation consuming the object store service is greatly reduced, the released object store service performance can be utilized by the write operation, so that the performance of the write operation can be improved.
Thus, embodiments of the present application optimize small object deletion performance for a distributed file storage system (Ceph).
According to the embodiment of the application, for storing object data, a life cycle management index is introduced, for example, the life cycle management index is generated by taking a day as a time unit, names of objects written into the object storage device on the same day are all established, namely, a mapping relation between the life cycle management index and an identification of the object data is established, wherein the life cycle management index is a date, the identification of the object data corresponding to the life cycle management index is the identification of the object written into the OSD on the date, and the identification of the object can be the name or the ID of the object. In some embodiments, a corresponding lifecycle management index may be set for each bucket, for example, bucket 1 corresponds to 10 lifecycle management indexes, where each of the 10 lifecycle management indexes is specifically set for bucket 1, each of the lifecycle management indexes corresponds to 10 object names, and object data corresponding to the 10 object names is stored in bucket 1, which indicates that 100 object data are stored in bucket 1. Of course, the lifecycle management index may be set separately, for example, 1 lifecycle management index corresponds to 10 object names, and then the object data corresponding to the 10 object names may be stored in the same bucket, or may be stored in different buckets, that is, 1 lifecycle management index may also correspond to different buckets.
After the life cycle management index is introduced, expired object records can be read from the life cycle management index in batches. Because the life cycle management index reflects the life cycle of the object data, whether the life cycle of the object data is finished or not can be monitored very conveniently through the life cycle management index, once the life cycle is finished, the identification of the object data corresponding to the life cycle management index can be directly found, the object data corresponding to the identifications of the object data are deleted in batches, for example, one life cycle management index corresponds to the identification of 10000 pieces of object data, the identification of the 10000 pieces of object data corresponds to 10000 pieces of outdated object data, and then batch deletion operation is carried out on the 10000 outdated object data.
In the embodiment of the application, the same RGW may send a batch deletion request of the object data to different OSDs, see fig. 1. The batch deletion request sent to any OSD includes an identifier of object data corresponding to an object whose life cycle is overtime stored in the target OSD.
The OSD in the embodiment of the present application is an entity of bearing data at the bottom layers of the data pool and the metadata pool, which is essentially a daemon of a disk, and it can be understood that the data pool and the metadata pool are logical concepts, while the OSD is a physically existing hard disk daemon, and the object data is written into the hard disk through the OSD, so as to achieve the purpose of data storage, and achieve the corresponding deletion operation through the OSD.
In this embodiment of the present application, the OSD directly receives the batch deletion request sent by the RGW, and after receiving the batch deletion request sent by the RGW, the main OSD sends a batch deletion request to the slave OSD as a main OSD, so as to delete the object data backed up on the slave OSD and having a life cycle overtime, see fig. 2. The distributed storage is provided with redundancy, and 3 copies are taken as an example, namely, one piece of object data is written into 3 OSDs, 1 piece of object data is arranged on each OSD, and 1 main OSD and 2 standby OSDs are arranged in the 3 OSDs.
After the batch deletion request sent by the RGW is processed by the main OSD and enters the main OSD, the main OSD generates batch deletion local object transactions according to the identification of the object data to be deleted carried in the batch deletion request sent by the RGW, and then the transactions are uniformly executed by a BlueTope module (object storage engine), namely, the deletion operation is executed on the related content of the local object to be deleted. The Bluetooth module is a module arranged in each OSD and used for executing the writing and deleting operations of the related content of the object data.
The master OSD transmits a bulk delete request to the slave OSD in response to the bulk delete request transmitted from the RGW (the bulk delete request transmitted from the master OSD to the slave OSD is hereinafter referred to as a bulk delete slave request in order to distinguish it from the bulk delete request transmitted from the RGW).
Similarly, after receiving a batch deletion slave request from the OSD, according to the identification of the object data to be deleted carried in the batch deletion slave request sent by the main OSD, a batch deletion local object transaction is generated, and then the transaction is executed by the bluetooth module, namely, the deletion operation is executed on the local object to be deleted.
An example of a processing flow of the write operation and the delete operation of the object data provided in the embodiment of the present application is given below.
The flow of the write operation with respect to the object data is as shown in fig. 3, including:
s301, RGW sends a write object data request message to OSD.
The write object data request message is used for requesting the OSD to perform a storage operation for a header object and a fragment object of object data to be written.
The object data described in the embodiments of the present application includes 1 head object and at least one slice object, and one head object and at least one slice object constitute one object data.
S302, the OSD sends a target data writing completion message to the RGW.
S303, RGW sends a corresponding relation request message between the identification of the object data and the storage bucket index to the OSD.
In some embodiments, this step establishes a correspondence between the name of the currently written object data and the bucket index, where the object data identified by the name of the object data is stored in the bucket identified by the bucket index.
S304, the OSD sends a corresponding relation completion message between the identification of the object data and the storage bucket index to the RGW.
S305, the RGW transmits a correspondence request message between the identification of the creation object data and the lifecycle management index to the OSD.
In some embodiments, this step establishes a correspondence between the name of the currently written object data and the lifecycle management index, where the object data identified by the name of the object data is written to the OSD on the date identified by the lifecycle management index.
It can be seen that in the embodiment of the present application, the RGW introduces a lifecycle management index in the lifecycle management of the object, to record the object information written into the object storage system every day, for example: 10 objects are written in the 2023 8-14 th year, and the identifiers of object data of the 10 objects are obj1 to obj10 respectively, so that a corresponding relationship between the life cycle management index 20230814 and the identifiers of the object data [ obj1, obj2, to obj10] is established, and the object information written in the object storage system every day is recorded.
The RGW can acquire the object information of all the life cycles overtime only by traversing the life cycle management index through the life cycle management thread.
In some embodiments, the lifecycle management thread may further sort objects with a lifecycle timeout according to the identifiers of the object data, and aggregate the identifiers of the object data with the lifecycle timeout belonging to the same OSD (Object Storage Device ), so as to facilitate the deletion operation of related content of the object data to be uniformly sent to the same OSD.
S306, the OSD sends a corresponding relation completion message between the identification of the object data and the life cycle management index to the RGW.
In some embodiments, in step S306, in the correspondence completing message sent by the OSD to the RGW for establishing the correspondence between the identifier of the object data and the identifier of the object data, the indication information of the correspondence between the established lifecycle management index and the identifier of the object data may be carried, and further the correspondence between the specific lifecycle management index and the identifier of the object data may be carried, so that the RGW may learn the correspondence between the lifecycle management index and the identifier of the object data, and be used to monitor whether the lifecycle of the object data is ended.
It can be seen that since there is a separate index record for expired objects, the object storage gateway can quickly determine the object data to be expired. In the writing flow of the object data provided by the embodiment of the application, a life cycle management index is added, and for the expired object data (namely the object data of which the life cycle is ended), the name of the expired object data can be directly read from the life cycle management index, and batch deletion for the expired object data is executed after aggregation.
In some embodiments, the object storage gateway generates a lifecycle management index in days, which is 1 special data structure that generates 1 per day, for example: 2023, 8, 14, then a lifecycle management index of 20230814 is generated;
when the object storage gateway writes the object data, writing an object record (such as the name of the object data) of the object data into a corresponding storage bucket index and a lifecycle management index, wherein the object record written in the current day is only written into the lifecycle management index generated in the current day;
for example: 10 object data are written in 2023, 8 and 14 days, the names of the 10 object data are assumed to be obj1 to obj10 respectively, and object records [ obj1, obj2, to obj10] can be obtained by reading 20230814 indexes. Assuming that a life cycle of 5 days is set, until 2023, 8, 20 days, 20230814 life cycle management index corresponds to [ obj1, obj2, to obj10], these object data are all out of date.
Accordingly, as shown in fig. 4, the deletion operation flow regarding the object data includes:
s401, the RGW sends a read lifecycle management index request message to the OSD.
S402, the OSD returns the identification of the object data corresponding to the life cycle management index to the RGW.
That is, through steps S401 and S402, the RGW can acquire the lifecycle management index, and further can determine which object data have their lifecycle ended and need to be deleted according to the lifecycle management index.
According to the setting of the life cycle (for example, 5 days), the expired life cycle management index can be read, the identifiers of the object data corresponding to the life cycle management index are read in batches, and the object data corresponding to the identifiers are prepared to be deleted in batches.
For example, two object data, whose names are obj1 and obj2, are written for the lifecycle management index 20230814, i.e., 2023, 8, 14. Then, when the object data written in 2023, 8, and 14 expires, the RGW carries the lifecycle management index 20230814 in the request message sent to the OSD in S401, and the OSD returns the lifecycle management index to the RGW in S402, the two records of the object data corresponding to the lifecycle management index 20230814 are carried [ obj1, obj2].
Therefore, according to the embodiment of the application, whether each traversed object exceeds the life cycle or not is judged without traversing the storage bucket list, and then the object deleting action is sequentially carried out, so that the IOPS performance of the object storage system is effectively prevented from being greatly consumed in the deleting operation of the small object after the life cycle arrives. Therefore, the small object quick release scheme provided by the embodiment of the application can effectively improve the life cycle management efficiency of the Ceph object storage gateway on small object data, and improve the performance specification of the Ceph object storage system in a full-specification cyclic coverage scene.
S403, the RGW transmits a header object request message of the bulk deletion object data to the OSD.
For example, the RGW requests OSD to delete the head object corresponding to the two object data of obj1 and obj2.
In some embodiments, the RGW aggregates the head object information to be deleted by the PG, e.g., places the head object information in an aggregation queue of the PG. The PG is a logical concept of data storage organization on the OSD, which is equivalent to several tens of PGs on one OSD, and object data is stored on the several tens of PGs. For example: if the object data [ obj1, obj2, to obj10] are stored in the same PG, the head object information of the outdated object data [ obj1, obj2, to obj10] is aggregated, for example, the aggregated head object information is [ obj1.Head, obj2.Head, …, obj10.Head ]. Further, when the number of the head objects to be deleted in the aggregation queue of the PG reaches a preset threshold, triggering the operation of deleting the head objects in batches, namely sending aggregated head object information to the OSD to delete the head objects in batches.
The head objects to be deleted are aggregated at the object storage gateway side, and a request for deleting the head objects in batches can be simultaneously used for requesting to delete dozens of head objects, so that the deleting efficiency can be greatly improved, and the influence of deleting operation on the performance of a storage system is reduced.
S404, the OSD sends a header object completion message of the batch deletion object data to the RGW.
For example, this step OSD notifies the RGW that the deletion operation of the head object corresponding to the two object data of obj1 and obj2 has been completed.
S405, the RGW sends a correspondence request message between the identification of the deletion object data and the bucket index to the OSD.
For example, the present step RGW requests OSD to delete the correspondence between obj1 and obj2 and bucket index.
S406, the OSD sends a corresponding relation completion message of the identification of the deletion object data and the storage bucket index to the RGW.
For example, this step OSD informs the RGW that the deletion operation of the correspondence between obj1 and obj2 and the bucket index has been completed.
The related index information of the object data is stored in the metadata pool, and thus, the RGW sends a request to delete the related index to the metadata pool in the OSD, so that the related record is deleted in the metadata pool, for example: delete the [ obj1, …, obj10] record in the lifecycle management index 20230814 and delete the [ obj1, …, obj10] record in the bucket index bucket 1. That is, the lifecycle of the delete record manages the correspondence of index 20230814 to [ obj1, …, obj10], and the correspondence of bucket index bucket 1 to [ obj1, …, obj10 ]. Further, when the 20230814 lifecycle management index is empty (all object records corresponding to the 20230814 index are deleted), the 20230814 lifecycle management index is deleted, and similarly, when the bucket index of the bucket 1 is empty (all object records corresponding to the bucket 1 index are deleted), the bucket index of the bucket 1 is deleted.
S407, the RGW transmits a slice object request message for deleting the object data to the OSD.
For example, the RGW requests OSD to delete the tile object corresponding to the two object data of obj1 and obj 2.
S408, the OSD sends a slice object completion message of deleting the object data to the RGW.
For example, this step OSD notifies the RGW that the deletion operation of the tile object corresponding to the two object data of obj1 and obj2 has been completed.
In some embodiments, the fragmented objects of the object data may be put into the GC queue for deletion, that is, there are many fragmented objects of the object data to be deleted in the GC queue, and the deletion operations of the fragmented objects are sequentially performed according to the order in the queue.
S409, the RGW transmits a correspondence request message between the identification of the deletion target data and the lifecycle management index to the OSD.
For example, the present step RGW requests OSD to delete the correspondence between obj1 and obj2 and the lifecycle management index.
After the OSD completes the deletion operation of the correspondence between obj1 and obj2 and the lifecycle management index, the deletion of the related contents of the two object data, namely, the two object data, namely obj1 and obj2, are not present on the OSD, the correspondence between obj1 and obj2 and the bucket index is not present, and the correspondence between obj1 and obj2 and the lifecycle management index is not present.
In some embodiments, it may further include: the OSD sends a correspondence completion message of the identification of the deletion object data and the lifecycle management index to the RGW. For example, the OSD informs the RGW that the deletion operation of the correspondence between obj1 and obj2 and the lifecycle management index has been completed.
In some embodiments, the OSDs in the flows shown in fig. 3 and 4 are all primary OSDs, i.e., the OSDs that interact with the RGW are all primary OSDs.
After the OSD receives a batch deletion request (which may be a deletion request for a header object or a fragment object of the object data, or a deletion request for a correspondence between an identifier of the object data and a bucket index, or a deletion request for a correspondence between an identifier of the object data and a lifecycle management index) about the object data sent by the object storage gateway, the OSD needs to perform the batch deletion request about the object data as a main OSD, where deletion of the locally stored object data is included, and in some embodiments, deletion of the object data backed up in the slave OSD is also included. Referring to fig. 5, a specific process flow after the osd receives a batch deletion request about object data sent by the object storage gateway includes, for example:
S501, the main OSD generates a batch deletion slave request according to the object to be deleted indicated in the batch deletion request, and generates a batch deletion local object data transaction.
The batch deletion slave request is used for indicating the batch deletion of the object data backed up in the slave OSD from the OSD, and the batch deletion slave request also carries the indication information of the object to be deleted.
And deleting the local object data transaction in batches, namely deleting tasks of related contents (corresponding relation between a header object, a fragment object, an identification of the object data and a storage bucket index and corresponding relation between an identification of the object data and a life cycle management index) of a plurality of outdated object data locally stored for the main OSD to be executed.
S502, the main OSD processes the batch deleting local object data transaction through a local Bluetooth module;
that is, the blustore module in the main OSD of this step specifically performs a deletion operation for an object to be deleted, deleting the related content of the object data from the local hard disk.
The Bluetooth module supports the operation of deleting local object data in batches, and merges KV (Key-Value) of the object data to be deleted, so that the deletion is unified and fast, the resource cost of the deletion operation in a full-specification cyclic coverage scene is greatly reduced, and the performance of object storage service is improved. The KV is metadata information of object data stored in the OSD, corresponding metadata information is generated after the object data is stored on the OSD, and the metadata information is used for recording which sector of a magnetic disk the object data is stored in, so that when the object data are deleted in batches, KV of all the object data to be deleted is gathered, and the object data of storage positions corresponding to the KV are deleted uniformly and quickly.
S503, the master OSD sends a batch deletion slave request to the slave OSD;
s504, the slave OSD deletes the objects to be deleted indicated in the slave request according to the batch, and generates a batch deletion local object data transaction;
and deleting the local object data transaction in batches, namely deleting tasks of related contents (corresponding relation between a header object, a fragment object, an identification of the object data and a storage bucket index and corresponding relation between an identification of the object data and a life cycle management index) of a plurality of outdated object data locally stored for the main OSD to be executed.
S505, processing the batch deleting local object data transaction from the OSD through a local BlueTorE module;
that is, this step specifically performs a delete operation for an object to be deleted from the bluetooth module in the OSD, deleting the relevant content of the object data from the local hard disk.
S506, the Bluetooth module in the main OSD processes the batch deletion local object data transaction to finish;
s507, deleting local object data transaction from the BlueTore module in the OSD in batches;
s508, sending a response for the batch deletion slave request from the OSD to the master OSD;
s509, the main OSD determines that the process of the batch deletion request sent for the RGW is completed, and returns a corresponding response message to the RGW.
In the above description, the interaction process between the master OSD and one slave OSD is taken as an example, and if one master OSD corresponds to a plurality of slave OSDs, the interaction process between the master OSD and other slave OSDs is similar, and will not be described again.
In summary, referring to fig. 6, on the RGW side, the method for processing object data provided in the embodiment of the present application includes:
s101, determining that object data ending the life cycle exists based on a preset life cycle management index (such as date); wherein, the life cycle management index has a corresponding relation with the identification (such as name) of the object data, and the identification of the object data corresponding to the same life cycle management index is the identification of the object data written into the object storage device in the same unit time (such as day);
s102, obtaining an identification of object data of the end of the life cycle corresponding to the life cycle management index;
and S103, based on the identification of the object data of the life cycle ending corresponding to the life cycle management index, executing the deleting operation of the object data of the life cycle ending.
In some embodiments, obtaining the identification of the object data of the end of the lifecycle corresponding to the lifecycle management index includes:
Transmitting a first request message (for example, the request message transmitted in the step S401) for reading a lifecycle management index to the object storage device, where the first request message is used to request the object storage device, and provide an identifier of object data corresponding to the lifecycle management index;
and receiving the identification of the object data corresponding to the life cycle management index returned by the object storage device aiming at the first request message.
In some embodiments, based on the identification of the end-of-life object data corresponding to the life-cycle management index, performing a delete operation for the end-of-life object data includes:
sending a second request message (for example, the request message sent in the step S403) for bulk deleting the head object of the object data to the object storage device, where the second request message is used to request the object storage device to delete the head object of the object data whose lifecycle ends;
receiving a header object completion message returned by the object storage device for deleting the object data of which the life cycle is finished aiming at the second request message;
transmitting a third request message (for example, the request message transmitted in the step S405) for deleting the correspondence between the identifier of the object data and the bucket index to the object storage device, where the third request message is used to request the object storage device to delete the correspondence between the identifier of the object data ending in the lifecycle and the bucket index, and the object data ending in the lifecycle is stored in the bucket corresponding to the bucket index;
Receiving a corresponding relation completion message between the identifier of the returned deleted object data and the bucket index of the object storage device aiming at the third request message;
transmitting a fourth request message (for example, the request message transmitted in the step S407) for deleting the fragmented object of the object data to the object storage device, the fourth request message being used for requesting the object storage device to delete the fragmented object of the object data of which the lifecycle ends;
receiving a fragment object completion message of deleting object data returned by the object storage device aiming at the fourth request message;
and sending a fifth request message (for example, the request message sent in the step S409) for deleting the correspondence between the identifier of the object data and the lifecycle management index to the object storage device, where the fifth request message is used to request the object storage device to delete the correspondence between the identifier of the object data ending the lifecycle and the lifecycle management index.
In some embodiments, the method further comprises:
when there is object data to be written, a sixth request message (for example, the request message sent in the step S301) for writing the object data is sent to the object storage device, where the sixth request message is used to request the object storage device to perform a storage operation for a header object and a fragment object of the object data to be written;
Receiving an object data writing completion message returned by the object storage device aiming at the sixth request message;
a seventh request message (for example, the request message sent in the step S303) for requesting the object storage device to establish a correspondence between the bucket index and the identification of the object data is sent to the object storage device;
receiving a corresponding relation completion message between the identification of the returned establishment object data and the storage bucket index of the object storage device aiming at the seventh request message;
an eighth request message (for example, the request message sent in the step S305) for requesting the object storage device to establish a correspondence between the identification of the object data and the identification of the object data is sent to the object storage device;
and receiving a corresponding relation completion message between the identification of the returned establishment object data and the life cycle management index by the object storage device aiming at the eighth request message.
Accordingly, referring to fig. 7, on the OSD side, the object data processing method provided in the embodiment of the present application includes:
S201, receiving a first request message for reading a life cycle management index sent by an object storage gateway, wherein the first request message is used for requesting a first object storage device (such as the main OSD) and providing an identifier of object data corresponding to the life cycle management index; the life cycle management index and the identification of the object data have a corresponding relation, and the identification of the object data corresponding to the same life cycle management index is the identification of the object data written into the object storage device in the same unit time;
s202, for the first request message, returning an identification of object data corresponding to the life cycle management index to the object storage gateway.
In some embodiments, the method further comprises:
receiving a second request message of batch deleting the head objects of the object data sent by the object storage gateway, wherein the second request message is used for requesting the first object storage device to delete the head objects of the object data with the end of the life cycle;
executing the operation of deleting the head object of the object data aiming at the second request message, and returning a head object completion message for deleting the object data of which the life cycle is finished to the object storage gateway;
Receiving a third request message sent by the object storage gateway for deleting the corresponding relation between the identification of the object data and the storage bucket index, wherein the third request message is used for requesting the first object storage device to delete the corresponding relation between the identification of the object data ending in the life cycle and the storage bucket index, and the object data ending in the life cycle is stored in the storage bucket corresponding to the storage bucket index;
executing the operation of deleting the corresponding relation between the identification of the object data and the storage bucket index aiming at the third request message, and returning a corresponding relation completion message between the identification of the object data and the storage bucket index to the object storage gateway;
receiving a fourth request message of deleting the fragmented object of the object data sent by the object storage gateway, wherein the fourth request message is used for requesting the first object storage device to delete the fragmented object of the object data with the end of the life cycle;
executing the operation of deleting the fragmented object of the object data aiming at the fourth request message, and returning a fragmented object completion message of deleting the object data to the object storage gateway;
receiving a fifth request message which is sent by the object storage gateway and used for deleting the corresponding relation between the identification of the object data and the lifecycle management index, wherein the fifth request message is used for requesting the object storage device to delete the corresponding relation between the identification of the object data ending the lifecycle and the lifecycle management index;
And executing the operation of deleting the corresponding relation between the identification of the object data and the life cycle management index aiming at the fifth request message.
In some embodiments, the method further comprises:
a slave request message is sent to a second object storage device (such as the slave OSD) for storing backup data of related content of object data stored in the first object storage device, and the slave request message is used for requesting the second object storage device to delete the backup data of related content of object data to be deleted by the first object storage device; the related content comprises one or a combination of the following contents: the method comprises the steps of enabling a head object of object data, a fragment object of the object data, a corresponding relation between an identification of the object data and a storage bucket index, and a corresponding relation between an identification of the object data and a life cycle management index;
and receiving a response message returned by the second object storage device for the request message.
In some embodiments, the method further comprises:
receiving a sixth request message for writing object data sent by the object storage gateway, wherein the sixth request message is used for requesting the first object storage device to execute a storage operation for a head object and a fragment object of the object data to be written;
Executing operations of writing the head object and the fragment object of the object data aiming at the sixth request message, and returning an object data writing completion message to the object storage gateway;
receiving a seventh request message which is sent by the object storage gateway and used for establishing the corresponding relation between the identification of the object data and the storage bucket index, wherein the seventh request message is used for requesting the first object storage equipment to establish the corresponding relation between the storage bucket index and the identification of the object data;
executing the operation of establishing the corresponding relation between the identification of the object data and the storage bucket index aiming at the seventh request message, and returning a corresponding relation completion message between the identification of the object data and the storage bucket index to the object storage gateway;
receiving an eighth request message which is sent by the object storage gateway and used for establishing the corresponding relation between the identification of the object data and the life cycle management index, wherein the eighth request message is used for requesting the first object storage device to establish the corresponding relation between the life cycle management index and the identification of the object data;
and executing the operation of establishing the corresponding relation between the identification of the object data and the life cycle management index aiming at the eighth request message, and returning a corresponding relation completion message between the identification of the object data and the life cycle management index to the object storage gateway.
The following describes a device or apparatus provided in the embodiments of the present application, where explanation or illustration of the same or corresponding technical features as those described in the above method is omitted herein.
An electronic device provided in an embodiment of the present application, referring to fig. 8, for example, includes: a processor 600 and a memory 620; in some embodiments, a user interface 630 may also be included;
wherein, when the electronic device is an RGW-side device:
the processor 600, configured to read the program in the memory 620, performs the following procedures:
determining that object data ending the life cycle exists based on a preset life cycle management index; the life cycle management index and the identification of the object data have a corresponding relation, and the identification of the object data corresponding to the same life cycle management index is the identification of the object data written into the object storage device in the same unit time;
acquiring an identifier of the object data of the end of the life cycle corresponding to the life cycle management index;
and executing deleting operation for the object data of the life cycle ending based on the identification of the object data of the life cycle ending corresponding to the life cycle management index.
In some embodiments, obtaining the identification of the object data of the end of the lifecycle corresponding to the lifecycle management index includes:
sending a first request message for reading a life cycle management index to the object storage device, wherein the first request message is used for requesting the object storage device and providing an identification of object data corresponding to the life cycle management index;
and receiving the identification of the object data corresponding to the life cycle management index returned by the object storage device aiming at the first request message.
In some embodiments, based on the identification of the end-of-life object data corresponding to the life-cycle management index, performing a delete operation for the end-of-life object data includes:
sending a second request message for deleting the head objects of the object data in batches to the object storage device, wherein the second request message is used for requesting the object storage device to delete the head objects of the object data of which the life cycle is finished;
receiving a header object completion message returned by the object storage device for deleting the object data of which the life cycle is finished aiming at the second request message;
Sending a third request message for deleting the corresponding relation between the identification of the object data and the storage bucket index to the object storage device, wherein the third request message is used for requesting the object storage device to delete the corresponding relation between the identification of the object data ending in the life cycle and the storage bucket index, and the object data ending in the life cycle is stored in the storage bucket corresponding to the storage bucket index;
receiving a corresponding relation completion message between the identifier of the returned deleted object data and the bucket index of the object storage device aiming at the third request message;
sending a fourth request message for deleting the fragmented object of the object data to the object storage device, wherein the fourth request message is used for requesting the object storage device to delete the fragmented object of the object data of which the life cycle is finished;
receiving a fragment object completion message of deleting object data returned by the object storage device aiming at the fourth request message;
and sending a fifth request message for deleting the corresponding relation between the identification of the object data and the life cycle management index to the object storage device, wherein the fifth request message is used for requesting the object storage device to delete the corresponding relation between the identification of the object data ending the life cycle and the life cycle management index.
In some embodiments, the processor 600 is further configured to read the program in the memory 620, and perform the following procedure:
when object data to be written exist, a sixth request message for writing the object data is sent to the object storage device, wherein the sixth request message is used for requesting the object storage device to execute a storage operation of a head object and a fragment object of the object data to be written;
receiving an object data writing completion message returned by the object storage device aiming at the sixth request message;
a seventh request message for establishing the corresponding relation between the identification of the object data and the storage bucket index is sent to the object storage equipment, and the seventh request message is used for requesting the object storage equipment to establish the corresponding relation between the storage bucket index and the identification of the object data;
receiving a corresponding relation completion message between the identification of the returned establishment object data and the storage bucket index of the object storage device aiming at the seventh request message;
an eighth request message for establishing the corresponding relation between the identification of the object data and the life cycle management index is sent to the object storage device, and the eighth request message is used for requesting the object storage device to establish the corresponding relation between the life cycle management index and the identification of the object data;
And receiving a corresponding relation completion message between the identification of the returned establishment object data and the life cycle management index by the object storage device aiming at the eighth request message.
When the electronic device is an OSD side device:
the processor 600, configured to read the program in the memory 620, performs the following procedures:
receiving a first request message for reading a life cycle management index sent by an object storage gateway, wherein the first request message is used for requesting a first object storage device and providing an identification of object data corresponding to the life cycle management index; the life cycle management index and the identification of the object data have a corresponding relation, and the identification of the object data corresponding to the same life cycle management index is the identification of the object data written into the object storage device in the same unit time;
and returning the identification of the object data corresponding to the life cycle management index to the object storage gateway aiming at the first request message.
In some embodiments, the processor 600 is further configured to read the program in the memory 620, and perform the following procedure:
receiving a second request message of batch deleting the head objects of the object data sent by the object storage gateway, wherein the second request message is used for requesting the first object storage device to delete the head objects of the object data with the end of the life cycle;
Executing the operation of deleting the head object of the object data aiming at the second request message, and returning a head object completion message for deleting the object data of which the life cycle is finished to the object storage gateway;
receiving a third request message sent by the object storage gateway for deleting the corresponding relation between the identification of the object data and the storage bucket index, wherein the third request message is used for requesting the first object storage device to delete the corresponding relation between the identification of the object data ending in the life cycle and the storage bucket index, and the object data ending in the life cycle is stored in the storage bucket corresponding to the storage bucket index;
executing the operation of deleting the corresponding relation between the identification of the object data and the storage bucket index aiming at the third request message, and returning a corresponding relation completion message between the identification of the object data and the storage bucket index to the object storage gateway;
receiving a fourth request message of deleting the fragmented object of the object data sent by the object storage gateway, wherein the fourth request message is used for requesting the first object storage device to delete the fragmented object of the object data with the end of the life cycle;
Executing the operation of deleting the fragmented object of the object data aiming at the fourth request message, and returning a fragmented object completion message of deleting the object data to the object storage gateway;
receiving a fifth request message which is sent by the object storage gateway and used for deleting the corresponding relation between the identification of the object data and the lifecycle management index, wherein the fifth request message is used for requesting the object storage device to delete the corresponding relation between the identification of the object data ending the lifecycle and the lifecycle management index;
and executing the operation of deleting the corresponding relation between the identification of the object data and the life cycle management index aiming at the fifth request message.
In some embodiments, the processor 600 is further configured to read the program in the memory 620, and perform the following procedure:
sending a batch deletion slave request message to a second object storage device, wherein the second object storage device is used for storing backup data of related contents of object data stored in the first object storage device, and the slave request message is used for requesting the second object storage device to delete the backup data of related contents of the object data which needs to be deleted by the first object storage device; the related content comprises one or a combination of the following contents: the method comprises the steps of enabling a head object of object data, a fragment object of the object data, a corresponding relation between an identification of the object data and a storage bucket index, and a corresponding relation between an identification of the object data and a life cycle management index;
And receiving a response message returned by the second object storage device for the request message.
In some embodiments, the processor 600 is further configured to read the program in the memory 620, and perform the following procedure:
receiving a sixth request message for writing object data sent by the object storage gateway, wherein the sixth request message is used for requesting the first object storage device to execute a storage operation for a head object and a fragment object of the object data to be written;
executing operations of writing the head object and the fragment object of the object data aiming at the sixth request message, and returning an object data writing completion message to the object storage gateway;
receiving a seventh request message which is sent by the object storage gateway and used for establishing the corresponding relation between the identification of the object data and the storage bucket index, wherein the seventh request message is used for requesting the first object storage equipment to establish the corresponding relation between the storage bucket index and the identification of the object data;
executing the operation of establishing the corresponding relation between the identification of the object data and the storage bucket index aiming at the seventh request message, and returning a corresponding relation completion message between the identification of the object data and the storage bucket index to the object storage gateway;
Receiving an eighth request message which is sent by the object storage gateway and used for establishing the corresponding relation between the identification of the object data and the life cycle management index, wherein the eighth request message is used for requesting the first object storage device to establish the corresponding relation between the life cycle management index and the identification of the object data;
and executing the operation of establishing the corresponding relation between the identification of the object data and the life cycle management index aiming at the eighth request message, and returning a corresponding relation completion message between the identification of the object data and the life cycle management index to the object storage gateway.
A transceiver 610 for receiving and transmitting data under the control of the processor 600.
Wherein in fig. 8, a bus architecture may comprise any number of interconnected buses and bridges, and in particular, one or more processors represented by processor 600 and various circuits of memory represented by memory 620, linked together. The bus architecture may also link together various other circuits such as peripheral devices, voltage regulators, power management circuits, etc., which are well known in the art and, therefore, will not be described further herein. The bus interface provides an interface. The transceiver 610 may be a number of elements, i.e., including a transmitter and a receiver, providing a means for communicating with various other apparatus over transmission media, including wireless channels, wired channels, optical cables, etc. The user interface 630 may also be an interface capable of interfacing with an inscribed desired device for different user devices, including but not limited to a keypad, display, speaker, microphone, joystick, etc.
The processor 600 is responsible for managing the bus architecture and general processing, and the memory 620 may store data used by the processor 600 in performing operations.
In some embodiments, the processor 600 may be a CPU (Central processing Unit), ASIC (Application Specific Integrated Circuit ), FPGA (Field-Programmable Gate Array, field programmable Gate array) or CPLD (Complex Programmable Logic Device ), and the processor may also employ a multicore architecture.
The processor is configured to execute any of the methods provided in the embodiments of the present application by invoking a computer program stored in a memory in accordance with the obtained executable instructions. The processor and the memory may also be physically separate.
It should be noted that, the above device provided in this embodiment of the present application can implement all the method steps implemented in the method embodiment and achieve the same technical effects, and detailed descriptions of the same parts and beneficial effects as those of the method embodiment in this embodiment are omitted.
Referring to fig. 9, on the RGW side, an object data processing apparatus (may be an RGW or an apparatus inside the RGW) provided in an embodiment of the present application includes:
A first unit 11, configured to determine, based on a preset lifecycle management index, that there is object data for which the lifecycle ends; the life cycle management index and the identification of the object data have a corresponding relation, and the identification of the object data corresponding to the same life cycle management index is the identification of the object data written into the object storage device in the same unit time;
a second unit 12, configured to obtain an identifier of the object data corresponding to the lifecycle management index and ending the lifecycle;
and a third unit 13, configured to perform a deletion operation for the object data of the end of the lifecycle based on the identification of the object data of the end of the lifecycle corresponding to the lifecycle management index.
In some embodiments, obtaining the identification of the object data of the end of the lifecycle corresponding to the lifecycle management index includes:
sending a first request message for reading a life cycle management index to the object storage device, wherein the first request message is used for requesting the object storage device and providing an identification of object data corresponding to the life cycle management index;
and receiving the identification of the object data corresponding to the life cycle management index returned by the object storage device aiming at the first request message.
In some embodiments, based on the identification of the end-of-life object data corresponding to the life-cycle management index, performing a delete operation for the end-of-life object data includes:
sending a second request message for deleting the head objects of the object data in batches to the object storage device, wherein the second request message is used for requesting the object storage device to delete the head objects of the object data of which the life cycle is finished;
receiving a header object completion message returned by the object storage device for deleting the object data of which the life cycle is finished aiming at the second request message;
sending a third request message for deleting the corresponding relation between the identification of the object data and the storage bucket index to the object storage device, wherein the third request message is used for requesting the object storage device to delete the corresponding relation between the identification of the object data ending in the life cycle and the storage bucket index, and the object data ending in the life cycle is stored in the storage bucket corresponding to the storage bucket index;
receiving a corresponding relation completion message between the identifier of the returned deleted object data and the bucket index of the object storage device aiming at the third request message;
Sending a fourth request message for deleting the fragmented object of the object data to the object storage device, wherein the fourth request message is used for requesting the object storage device to delete the fragmented object of the object data of which the life cycle is finished;
receiving a fragment object completion message of deleting object data returned by the object storage device aiming at the fourth request message;
and sending a fifth request message for deleting the corresponding relation between the identification of the object data and the life cycle management index to the object storage device, wherein the fifth request message is used for requesting the object storage device to delete the corresponding relation between the identification of the object data ending the life cycle and the life cycle management index.
In some embodiments, the apparatus further comprises a fourth unit for:
when object data to be written exist, a sixth request message for writing the object data is sent to the object storage device, wherein the sixth request message is used for requesting the object storage device to execute a storage operation of a head object and a fragment object of the object data to be written;
receiving an object data writing completion message returned by the object storage device aiming at the sixth request message;
A seventh request message for establishing the corresponding relation between the identification of the object data and the storage bucket index is sent to the object storage equipment, and the seventh request message is used for requesting the object storage equipment to establish the corresponding relation between the storage bucket index and the identification of the object data;
receiving a corresponding relation completion message between the identification of the returned establishment object data and the storage bucket index of the object storage device aiming at the seventh request message;
an eighth request message for establishing the corresponding relation between the identification of the object data and the life cycle management index is sent to the object storage device, and the eighth request message is used for requesting the object storage device to establish the corresponding relation between the life cycle management index and the identification of the object data;
and receiving a corresponding relation completion message between the identification of the returned establishment object data and the life cycle management index by the object storage device aiming at the eighth request message.
Accordingly, referring to fig. 10, on the OSD side, an object data processing apparatus (may be an OSD or an apparatus inside the OSD) provided in the embodiment of the present application includes:
a receiving request unit 21, configured to receive a first request message sent by an object storage gateway and used for requesting a first object storage device to provide an identifier of object data corresponding to the lifecycle management index; the life cycle management index and the identification of the object data have a corresponding relation, and the identification of the object data corresponding to the same life cycle management index is the identification of the object data written into the object storage device in the same unit time;
And a feedback unit 22, configured to return, to the object storage gateway, for the first request message, an identification of object data corresponding to the lifecycle management index.
In some embodiments:
the reception request unit 21 is further configured to: receiving a second request message of batch deleting the head objects of the object data sent by the object storage gateway, wherein the second request message is used for requesting the first object storage device to delete the head objects of the object data with the end of the life cycle;
the feedback unit 22 is also configured to: executing the operation of deleting the head object of the object data aiming at the second request message, and returning a head object completion message for deleting the object data of which the life cycle is finished to the object storage gateway;
the reception request unit 21 is further configured to: receiving a third request message sent by the object storage gateway for deleting the corresponding relation between the identification of the object data and the storage bucket index, wherein the third request message is used for requesting the first object storage device to delete the corresponding relation between the identification of the object data ending in the life cycle and the storage bucket index, and the object data ending in the life cycle is stored in the storage bucket corresponding to the storage bucket index;
The feedback unit 22 is also configured to: executing the operation of deleting the corresponding relation between the identification of the object data and the storage bucket index aiming at the third request message, and returning a corresponding relation completion message between the identification of the object data and the storage bucket index to the object storage gateway;
the reception request unit 21 is further configured to: receiving a fourth request message of deleting the fragmented object of the object data sent by the object storage gateway, wherein the fourth request message is used for requesting the first object storage device to delete the fragmented object of the object data with the end of the life cycle;
the feedback unit 22 is also configured to: executing the operation of deleting the fragmented object of the object data aiming at the fourth request message, and returning a fragmented object completion message of deleting the object data to the object storage gateway;
the reception request unit 21 is further configured to: receiving a fifth request message which is sent by the object storage gateway and used for deleting the corresponding relation between the identification of the object data and the lifecycle management index, wherein the fifth request message is used for requesting the object storage device to delete the corresponding relation between the identification of the object data ending the lifecycle and the lifecycle management index;
The feedback unit 22 is also configured to: and executing the operation of deleting the corresponding relation between the identification of the object data and the life cycle management index aiming at the fifth request message.
In some embodiments, the method further comprises a bulk delete slave request unit for:
sending a batch deletion slave request message to a second object storage device, wherein the second object storage device is used for storing backup data of related contents of object data stored in the first object storage device, and the slave request message is used for requesting the second object storage device to delete the backup data of related contents of the object data which needs to be deleted by the first object storage device; the related content comprises one or a combination of the following contents: the method comprises the steps of enabling a head object of object data, a fragment object of the object data, a corresponding relation between an identification of the object data and a storage bucket index, and a corresponding relation between an identification of the object data and a life cycle management index;
and receiving a response message returned by the second object storage device for the request message.
In some embodiments:
the reception request unit 21 is further configured to: receiving a sixth request message for writing object data sent by the object storage gateway, wherein the sixth request message is used for requesting the first object storage device to execute a storage operation for a head object and a fragment object of the object data to be written;
The feedback unit 22 is also configured to: executing operations of writing the head object and the fragment object of the object data aiming at the sixth request message, and returning an object data writing completion message to the object storage gateway;
the reception request unit 21 is further configured to: receiving a seventh request message which is sent by the object storage gateway and used for establishing the corresponding relation between the identification of the object data and the storage bucket index, wherein the seventh request message is used for requesting the first object storage equipment to establish the corresponding relation between the storage bucket index and the identification of the object data;
the feedback unit 22 is also configured to: executing the operation of establishing the corresponding relation between the identification of the object data and the storage bucket index aiming at the seventh request message, and returning a corresponding relation completion message between the identification of the object data and the storage bucket index to the object storage gateway;
the reception request unit 21 is further configured to: receiving an eighth request message which is sent by the object storage gateway and used for establishing the corresponding relation between the identification of the object data and the life cycle management index, wherein the eighth request message is used for requesting the first object storage device to establish the corresponding relation between the life cycle management index and the identification of the object data;
The feedback unit 22 is also configured to: and executing the operation of establishing the corresponding relation between the identification of the object data and the life cycle management index aiming at the eighth request message, and returning a corresponding relation completion message between the identification of the object data and the life cycle management index to the object storage gateway.
It should be noted that, in the embodiment of the present application, the division of the units is schematic, which is merely a logic function division, and other division manners may be implemented in actual practice. In addition, each functional unit in each embodiment of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.
The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application may be embodied in essence or a part contributing to the prior art or all or part of the technical solution, in the form of a software product stored in a storage medium, including several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor (processor) to perform all or part of the steps of the methods described in the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
Any of the devices or apparatuses provided in the embodiments of the present application may be specifically a desktop computer, a portable computer, a smart phone, a tablet computer, a personal digital assistant (Personal Digital Assistant, PDA), or the like. May include a central processing unit (Center Processing Unit, CPU), memory, input/output devices, etc., which may include a keyboard, mouse, touch screen, etc., and output devices may include a display device, such as a liquid crystal display (Liquid Crystal Display, LCD), cathode Ray Tube (CRT), etc.
The memory may include Read Only Memory (ROM) and Random Access Memory (RAM) and provides the processor with program instructions and data stored in the memory. In the embodiments of the present application, the memory may be used to store a program of any of the methods provided in the embodiments of the present application.
The processor is configured to execute any of the methods provided in the embodiments of the present application according to the obtained program instructions by calling the program instructions stored in the memory.
Embodiments of the present application also provide a computer program product or computer program comprising computer instructions stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the method of any of the above embodiments. The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium would include the following: an electrical connection having one or more wires, a portable disk, a hard disk, random Access Memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
The present embodiments provide a computer readable storage medium for storing computer program instructions for use with an apparatus provided in the embodiments of the present application described above, which includes a program for executing any one of the methods provided in the embodiments of the present application described above. The computer readable storage medium may be a non-transitory computer readable medium.
The computer-readable storage medium can be any available medium or data storage device that can be accessed by a computer, including, but not limited to, magnetic storage (e.g., floppy disks, hard disks, magnetic tape, magneto-optical disks (MOs), etc.), optical storage (e.g., CD, DVD, BD, HVD, etc.), and semiconductor storage (e.g., ROM, EPROM, EEPROM, nonvolatile storage (NAND FLASH), solid State Disk (SSD)), etc.
It should be understood that:
the access technology via which an entity in the communication network communicates traffic may be any suitable current or future technology, such as WLAN (wireless local access network), wiMAX (worldwide interoperability for microwave access), LTE-a, 5G, bluetooth, infrared, etc. may be used; in addition, embodiments may also apply wired technologies, e.g., IP-based access technologies, such as wired networks or fixed lines.
Embodiments suitable for implementation as software code or portions thereof and for execution using a processor or processing function are software code independent and may be specified using any known or future developed programming language, such as a high-level programming language, such as an objective-C, C, C ++, c#, java, python, javascript, other scripting languages, etc., or a low-level programming language, such as a machine language or assembler.
The implementation of the embodiments is hardware-independent and may be implemented using any known or future developed hardware technology or any hybrid thereof, such as microprocessors or CPUs (central processing units), MOS (metal oxide semiconductors), CMOS (complementary MOS), biMOS (bipolar MOS), biCMOS (bipolar CMOS), ECL (emitter coupled logic), and/or TTL (transistor-transistor logic).
Embodiments may be implemented as a single device, apparatus, unit, component, or function, or in a distributed fashion, e.g., one or more processors or processing functions may be used or shared in a process, or one or more processing segments or portions may be used and shared in a process where one physical processor or more than one physical processor may be used to implement one or more processing portions dedicated to a particular process as described.
The apparatus may be implemented by a semiconductor chip, a chipset, or a (hardware) module comprising such a chip or chipset.
Embodiments may also be implemented as any combination of hardware and software, such as an ASIC (application specific IC (integrated circuit)) component, an FPGA (field programmable gate array) or CPLD (complex programmable logic device) component, or a DSP (digital signal processor) component.
Embodiments may also be implemented as a computer program product comprising a computer usable medium having a computer readable program code embodied therein, the computer readable program code adapted to perform a process as described in the embodiments, wherein the computer usable medium may be a non-transitory medium.
It will be appreciated by those skilled in the art that embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, magnetic disk storage, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
It will be apparent to those skilled in the art that various modifications and variations can be made in the present application without departing from the spirit or scope of the application. Thus, if such modifications and variations of the present application fall within the scope of the claims and the equivalents thereof, the present application is intended to cover such modifications and variations.
Claims (10)
1. A method of object data processing, the method comprising:
determining that object data ending the life cycle exists based on a preset life cycle management index; the life cycle management index and the identification of the object data have a corresponding relation, and the identification of the object data corresponding to the same life cycle management index is the identification of the object data written into the object storage device in the same unit time;
Acquiring an identifier of the object data of the end of the life cycle corresponding to the life cycle management index;
and executing deleting operation for the object data of the life cycle ending based on the identification of the object data of the life cycle ending corresponding to the life cycle management index.
2. The method of claim 1, wherein obtaining the identification of the end-of-lifecycle object data corresponding to the lifecycle management index comprises:
sending a first request message for reading a life cycle management index to the object storage device, wherein the first request message is used for requesting the object storage device and providing an identification of object data corresponding to the life cycle management index;
and receiving the identification of the object data corresponding to the life cycle management index returned by the object storage device aiming at the first request message.
3. The method of claim 1, wherein performing a delete operation for the end-of-lifecycle object data based on an identification of the end-of-lifecycle object data corresponding to the lifecycle management index comprises:
sending a second request message for deleting the head objects of the object data in batches to the object storage device, wherein the second request message is used for requesting the object storage device to delete the head objects of the object data of which the life cycle is finished;
Receiving a header object completion message returned by the object storage device for deleting the object data of which the life cycle is finished aiming at the second request message;
sending a third request message for deleting the corresponding relation between the identification of the object data and the storage bucket index to the object storage device, wherein the third request message is used for requesting the object storage device to delete the corresponding relation between the identification of the object data ending in the life cycle and the storage bucket index, and the object data ending in the life cycle is stored in the storage bucket corresponding to the storage bucket index;
receiving a corresponding relation completion message between the identifier of the returned deleted object data and the bucket index of the object storage device aiming at the third request message;
sending a fourth request message for deleting the fragmented object of the object data to the object storage device, wherein the fourth request message is used for requesting the object storage device to delete the fragmented object of the object data of which the life cycle is finished;
receiving a fragment object completion message of deleting object data returned by the object storage device aiming at the fourth request message;
and sending a fifth request message for deleting the corresponding relation between the identification of the object data and the life cycle management index to the object storage device, wherein the fifth request message is used for requesting the object storage device to delete the corresponding relation between the identification of the object data ending the life cycle and the life cycle management index.
4. The method according to claim 1, wherein the method further comprises:
when object data to be written exist, a sixth request message for writing the object data is sent to the object storage device, wherein the sixth request message is used for requesting the object storage device to execute a storage operation of a head object and a fragment object of the object data to be written;
receiving an object data writing completion message returned by the object storage device aiming at the sixth request message;
a seventh request message for establishing the corresponding relation between the identification of the object data and the storage bucket index is sent to the object storage equipment, and the seventh request message is used for requesting the object storage equipment to establish the corresponding relation between the storage bucket index and the identification of the object data;
receiving a corresponding relation completion message between the identification of the returned establishment object data and the storage bucket index of the object storage device aiming at the seventh request message;
an eighth request message for establishing the corresponding relation between the identification of the object data and the life cycle management index is sent to the object storage device, and the eighth request message is used for requesting the object storage device to establish the corresponding relation between the life cycle management index and the identification of the object data;
And receiving a corresponding relation completion message between the identification of the returned establishment object data and the life cycle management index by the object storage device aiming at the eighth request message.
5. A method of object data processing, the method comprising:
receiving a first request message for reading a life cycle management index sent by an object storage gateway, wherein the first request message is used for requesting a first object storage device and providing an identification of object data corresponding to the life cycle management index; the life cycle management index and the identification of the object data have a corresponding relation, and the identification of the object data corresponding to the same life cycle management index is the identification of the object data written into the object storage device in the same unit time;
and returning the identification of the object data corresponding to the life cycle management index to the object storage gateway aiming at the first request message.
6. The method of claim 5, wherein the method further comprises:
receiving a second request message of batch deleting the head objects of the object data sent by the object storage gateway, wherein the second request message is used for requesting the first object storage device to delete the head objects of the object data with the end of the life cycle;
Executing the operation of deleting the head object of the object data aiming at the second request message, and returning a head object completion message for deleting the object data of which the life cycle is finished to the object storage gateway;
receiving a third request message sent by the object storage gateway for deleting the corresponding relation between the identification of the object data and the storage bucket index, wherein the third request message is used for requesting the first object storage device to delete the corresponding relation between the identification of the object data ending in the life cycle and the storage bucket index, and the object data ending in the life cycle is stored in the storage bucket corresponding to the storage bucket index;
executing the operation of deleting the corresponding relation between the identification of the object data and the storage bucket index aiming at the third request message, and returning a corresponding relation completion message between the identification of the object data and the storage bucket index to the object storage gateway;
receiving a fourth request message of deleting the fragmented object of the object data sent by the object storage gateway, wherein the fourth request message is used for requesting the first object storage device to delete the fragmented object of the object data with the end of the life cycle;
Executing the operation of deleting the fragmented object of the object data aiming at the fourth request message, and returning a fragmented object completion message of deleting the object data to the object storage gateway;
receiving a fifth request message which is sent by the object storage gateway and used for deleting the corresponding relation between the identification of the object data and the lifecycle management index, wherein the fifth request message is used for requesting the object storage device to delete the corresponding relation between the identification of the object data ending the lifecycle and the lifecycle management index;
and executing the operation of deleting the corresponding relation between the identification of the object data and the life cycle management index aiming at the fifth request message.
7. The method of claim 6, wherein the method further comprises:
sending a batch deletion slave request message to a second object storage device, wherein the second object storage device is used for storing backup data of related contents of object data stored in the first object storage device, and the slave request message is used for requesting the second object storage device to delete the backup data of related contents of the object data which needs to be deleted by the first object storage device; the related content comprises one or a combination of the following contents: the method comprises the steps of enabling a head object of object data, a fragment object of the object data, a corresponding relation between an identification of the object data and a storage bucket index, and a corresponding relation between an identification of the object data and a life cycle management index;
And receiving a response message returned by the second object storage device for the request message.
8. The method of claim 5, wherein the method further comprises:
receiving a sixth request message for writing object data sent by the object storage gateway, wherein the sixth request message is used for requesting the first object storage device to execute a storage operation for a head object and a fragment object of the object data to be written;
executing operations of writing the head object and the fragment object of the object data aiming at the sixth request message, and returning an object data writing completion message to the object storage gateway;
receiving a seventh request message which is sent by the object storage gateway and used for establishing the corresponding relation between the identification of the object data and the storage bucket index, wherein the seventh request message is used for requesting the first object storage equipment to establish the corresponding relation between the storage bucket index and the identification of the object data;
executing the operation of establishing the corresponding relation between the identification of the object data and the storage bucket index aiming at the seventh request message, and returning a corresponding relation completion message between the identification of the object data and the storage bucket index to the object storage gateway;
Receiving an eighth request message which is sent by the object storage gateway and used for establishing the corresponding relation between the identification of the object data and the life cycle management index, wherein the eighth request message is used for requesting the first object storage device to establish the corresponding relation between the life cycle management index and the identification of the object data;
and executing the operation of establishing the corresponding relation between the identification of the object data and the life cycle management index aiming at the eighth request message, and returning a corresponding relation completion message between the identification of the object data and the life cycle management index to the object storage gateway.
9. An electronic device, comprising:
a memory for storing program instructions;
a processor for invoking program instructions stored in said memory to perform the method of any of claims 1 to 8 in accordance with the obtained program.
10. A computer-readable storage medium storing computer-executable instructions for causing a computer to perform the method of any one of claims 1 to 8.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202311183826.1A CN117349255A (en) | 2023-09-14 | 2023-09-14 | Object data processing method and device |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202311183826.1A CN117349255A (en) | 2023-09-14 | 2023-09-14 | Object data processing method and device |
Publications (1)
Publication Number | Publication Date |
---|---|
CN117349255A true CN117349255A (en) | 2024-01-05 |
Family
ID=89365825
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202311183826.1A Pending CN117349255A (en) | 2023-09-14 | 2023-09-14 | Object data processing method and device |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN117349255A (en) |
-
2023
- 2023-09-14 CN CN202311183826.1A patent/CN117349255A/en active Pending
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN107729139B (en) | Method and device for concurrently acquiring resources | |
US10235047B2 (en) | Memory management method, apparatus, and system | |
US11010186B2 (en) | Non-transitory computer-readable recording medium, adjustment device, and adjustment method | |
US11347684B2 (en) | Rolling back KUBERNETES applications including custom resources | |
US20150006608A1 (en) | Networked solutions integration using a cloud business object broker | |
CN109857723B (en) | Dynamic data migration method based on expandable database cluster and related equipment | |
EP2778962B1 (en) | Silo-aware databases | |
WO2023142824A1 (en) | Gpu resource utilization monitoring method and apparatus, computer device, and medium | |
CN112685499A (en) | Method, device and equipment for synchronizing process data of work service flow | |
EP4379543A1 (en) | Cloud desktop data migration method, service node, management node, server, electronic device, and computer-readable storage medium | |
CN111192070B (en) | Storage service metering method and system, computer system and readable storage medium | |
WO2022179342A1 (en) | Application deployment in computing environment | |
CN108459910A (en) | A kind of method and apparatus for deleting resource | |
CN108509150B (en) | Data processing method and device | |
CN112416980B (en) | Data service processing method, device and equipment | |
CN109905258B (en) | PaaS management method, device and storage medium | |
US9898490B2 (en) | Systems and methods for supporting multiple database server versions on a database machine | |
CN117349255A (en) | Object data processing method and device | |
CN112905541B (en) | Mirror image warehouse garbage cleaning method and device | |
CN115729693A (en) | Data processing method and device, computer equipment and computer readable storage medium | |
CN113239059A (en) | Switching method and device of distributed lock, server and storage medium | |
CN114676093B (en) | File management method and device, electronic equipment and storage medium | |
CN111782732B (en) | Data processing method, device and computer readable storage medium | |
US12056372B2 (en) | Collecting quality of service statistics for in-use child physical functions of multiple physical function non-volatile memory devices | |
US11758012B1 (en) | Computer service invocation chain monitoring and remuneration optimization |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination |