TWI769796B - Method for performing simple storage service seamless migration using index objects, host device, and storage server - Google Patents

Abstract

A method for performing Simple Storage Service (S3) seamless migration using index objects and associated apparatus are provided. The method includes: in response to a request of migrating user data of a user of the storage server from a remote S3-compatible server into the storage server, during an index stage, utilizing an index-object-based S3 migration management module among multiple program modules running on a host device within the storage server to create and store multiple index objects into a storage device layer of the storage server, and migrate respective Access Control Lists (ACLs) of the multiple normal objects to the storage server; and during a data stage, utilizing the index-object-based S3 migration management module to trigger one or more migration agents to migrate object data.

Description

Method, host device and storage server for seamless migration of simple storage services using index objects

The present invention relates to memory control, and more particularly, to a method and apparatus (apparatus) for seamless migration of simple storage service (S3) using index objects, wherein an example of the apparatus may include: an entire storage server , a host device within the storage server, a processing circuit within the host device, and at least one processor/processor core (eg, a central processing unit) running one or more program modules in the processing circuit corresponding to the method processing unit, CPU)/central processing unit core), but the present invention is not limited thereto.

Servers can be used for cloud computing, for example, storage servers can be used to implement remote storage (eg, a cloud that can store data for users). When a user using an old public cloud finds that the public cloud is almost full or the access speed is slow, the user may need to find a new cloud. A data migration method has been proposed in the prior art to try Copying data from the old public cloud to another public cloud, however, some problems may arise, for example, this data migration method usually copies the data first, including all the data before the start point in the old public cloud, and then carries out the port Point switching, etc., so it is possible to erroneously copy data that should be deleted (such as data that has been deleted by the user after the start point in time) to another public cloud. In addition, according to the suggestion of this data migration method, it may take a long time (for example, several months) to copy the data first. Cloud, it seems that there is no suitable solution for migrating data from old cloud to private cloud in the related art, therefore, a novel architecture is required for seamless migration of simple storage services for private cloud (for example, from A simple storage service seamlessly migrates from any public/private cloud to a private cloud) to allow users to access data normally and smoothly during the data migration process.

Therefore, one of the objectives of the present invention is to disclose a method for seamless migration of simple storage services using index objects, and to disclose related equipment (such as storage servers, host devices in storage servers, etc.) to solve the above problems .

Another object of the present invention is to disclose a method for using index objects to perform seamless migration of simple storage services, and to disclose related devices (such as storage servers, master devices in storage servers, etc.), so as not to introduce side effects Optimal performance is achieved in a way that is less likely to introduce side effects.

At least one embodiment of the present invention discloses a method for seamless migration of simple storage services using index objects, wherein the method is applied to a storage server, and the method may include: in response to compatibility from a remote simple storage service (S3-compatible) server to migrate user data of a user of the storage server to a request to the storage server, in an index stage of a plurality of stages of migration of the user data , utilizes an index-object-based simple storage service migration management module in a plurality of program modules running on a host device in the storage server to create and store a plurality of index objects to a storage device layer of the storage server as respective representatives of the normal objects of the user data in the storage server, and migrate the respective accesses of the normal objects access control lists (ACLs) to the storage server as respective access control lists of the plurality of index objects to minimize any change of any one of the plurality of normal objects affecting the user The probability of correctness of the migration of data, wherein the storage server includes the host device and the storage device layer, the storage device layer including at least one storage device coupled to the host device, the host device for controlling the operation of the storage server, and the at least one storage device is used to store information for the storage server; and in a data stage following the indexing stage of the plurality of stages, using the based an easy storage service migration management module for indexing objects to trigger one or more migration agents to migrate respective ones of a set of normal objects of the plurality of normal objects from the remote easy storage service compatible server to the storage device layer of the storage server as respective replacements of a set of index objects among the plurality of index objects to complete the migration of the user data.

In addition to the above method, the present invention also discloses a host device, the host device may include a processing circuit for controlling the host device to perform simple storage service migration management based on index objects in a storage server, wherein the storage The server includes the host device and a storage device layer, the storage device layer includes at least one storage device coupled to the host device, the host device is used to control the operation of the storage server, and the at least one storage device The device is used to store information for the storage server, eg, in response to a request to migrate a user data of a user of the storage server to the storage server from a remote simple storage service compatible server, at In an indexing phase of a plurality of stages of a migration of the user data, a simple storage service migration management module based on indexing objects among a plurality of program modules running on the processing circuit creates and stores a plurality of indexes objects to the storage device layer as respective representatives of a plurality of normal objects of the user data in the storage server, and migrate the respective ACLs of the plurality of normal objects to the storage server , as the respective ACLs of the plurality of index objects to minimize the probability that any change of any one of the plurality of normal objects affects the correctness of the migration of the user data; and in the In a data phase following the indexing phase of the plurality of phases, the index object-based Easy Storage Services migration management module triggers one or more migration agents to migrate the remote Easy Storage Services compatible server the respective object data of a set of normal objects in the plurality of normal objects to the storage device layer of the storage server as respective replacements of a set of index objects in the plurality of index objects to complete the user The migration of data.

In addition to the above method, the present invention also discloses a storage server. The storage server may include a host device and a storage device layer, wherein the host device is used to control the operation of the storage server. For example, the host device A processing circuit may be included for controlling the host device for easy storage service migration management based on index objects in the storage server. Additionally, the storage device layer may include at least one storage device for storing information for the storage server, eg, a user in response to a user migrating the storage server from a Remote Easy Storage Service compliant server A request of data to the storage server, a simple storage based on indexing objects in a plurality of program modules running on the processing circuit during an indexing phase of a migration of the user data The service migration management module creates and stores a plurality of index objects to the storage device layer as respective representatives of the plurality of normal objects of the user data in the storage server, and the respective ones of the plurality of normal objects Migrate the ACLs of the multiple index objects to the storage server as the respective ACLs of the plurality of index objects, so as to minimize any change of any one of the multiple regular objects from affecting the user data the probability of correctness of the migration; and in a data phase following the index phase of the plurality of phases, the index object-based simple storage service migration management module triggers one or more migration agents to The remote simple storage service compatible server migrates the respective object data of a group of normal objects of the plurality of normal objects to the storage device layer of the storage server as a set of indexes in the plurality of index objects respective replacements of objects to complete the migration of the user data.

The method and related apparatus of the present invention can use index objects to perform seamless migration of simple storage services. After an indexing stage/phase (eg, a short period of time), the method and related apparatus of the present invention Seamless migration can be done with less overhead in a data phase/epoch, where the overhead of a data stage/epoch is typically less than that of an indexing stage/epoch.

Embodiments of the present invention disclose a method and device for performing seamless migration of simple storage services using index objects. For example, the method and related devices can use a novel seamless migration framework for simple storage services for access control, and In particular, seamless migration of simple storage services for private clouds, such as seamless migration of simple storage services from any public/private cloud to a private cloud (such as the destination cloud for data migration), allows the use of Users can access data normally and smoothly during the data migration process. For better understanding, a first cloud, such as Easy Storage Service compliant storage (eg it may be almost full and access may be slow), and a second cloud such as Bigtera storage implemented according to the method , which can be used as examples of the public/private cloud and the private cloud (eg, the target cloud), respectively, but the present invention is not limited thereto. Furthermore, when the migration starts at the first point in time T1, the method and related apparatus may precede an indexing phase, such as an INDEX phase (eg, an indexing period, such as an INDEX period), starting from the first point in time T1, at the first An index object (such as a dummy index object) is created in the second cloud, and can be in a data stage, such as a DATA stage ( For example, a data epoch, such as a DATA epoch, migrates (eg, transfers) object data from the first cloud to the second cloud. In some embodiments, the simple storage service seamless migration framework may utilize one or more migration agents to perform object migration, and the device may include at least a portion (eg, a portion or all) of the storage server, For example, the apparatus may include a portion of a storage server, such as a host device in the storage server or associated control circuitry located within the host device (eg, processing circuitry running on one or more program modules corresponding to the method) and related circuitry), in another example, the device may include an entire storage server, according to some embodiments for storage servers, the INDEX period may represent a short period of time (eg, hours or days), while the DATA A period may represent a long period of time (eg, several weeks).

FIG. 1 is a schematic diagram of a storage server 10 according to an embodiment of the present invention. The storage server 10 includes a host device 50 and includes at least one storage device, such as one or more storage devices (eg, a plurality of storage devices 90 ). ), a plurality of storage devices 90 are coupled to the main device 50 . According to this embodiment, the main device 50 can be used to control the operation of the storage server 10 , and the plurality of storage devices 90 can be used to store information for the storage server 10 . As shown in FIG. 1, host device 50 may include processing circuitry 52 (eg, at least one processor/processor core and associated circuitry such as random access memory (RAM)) for controlling the operation of host device 50 , bus, etc.), used to couple to a plurality of storage devices 90 and to a storage or memory device (eg, one or more conventional hard disk drives (HDDs) and/or coupled to the host device 50 at least one storage interface circuit 54 of one or more solid state drives (SSD), and a network interface circuit 58 for coupling the host device 50 to at least one network. A storage or memory device may include at least one storage device (eg, one or more storage devices), which may be collectively referred to as storage device 56 , for example, storage device 56 may include a set of storage devices in which one of the set of storage devices stores The device can be used as the system disk of the main device 50, and other storage devices of the group of storage devices can be used to store user data for the main device 50, but the invention is not limited to this. In another example, the storage device 56 may contain a storage device, and this storage device may be used as the system disk of the master device 50 .

According to this embodiment, the processing circuit 52 of the running program module 52P (especially the index object-based simple storage service migration management module 53 corresponding to the method) can control the operation of the main device 50 according to the method, for example, control The host device 50 performs simple storage service migration management based on index objects in the storage server 10, and the storage interface circuit 54 may conform to one or more standards (eg, one or more Serial Advanced Technology Attachments, Serial ATA or SATA) standard, Peripheral Component Interconnect (PCI) standard, Peripheral Component Interconnect Express (PCIe) standard, Non-Volatile Memory Express (NVMe) ) standard, non-volatile memory storage device external access (NVMe-over-Fabrics, NVMeoF) standard, Small Computer System Interface (Small Computer System Interface, SCSI) standard, Universal Flash Storage (Universal Flash Storage, UFS) standard ) standard, etc.), and can communicate according to one or more standards to allow the processing circuit 52 running the program module 52P to access the storage device 56 and the plurality of storage devices 90 through the storage interface circuit 54, in addition, the network The interface circuit 58 can be used to provide wired or wireless network connection, and one or more client devices corresponding to one or more users can be connected to the storage server 10 (eg, the storage server 10 therein) through the wired or wireless network device 56 and a plurality of storage devices 90) to access (eg, read or write) user data. For better understanding, the host device 50 shown in the left half of FIG. 1 and the related circuits/modules/devices in the architecture (eg, the processing circuit 52 and the storage interface circuit 54 of the running program module 52P) can be respectively As an example of the above-described host device and associated circuits/modules/devices therein (eg, processing circuits and associated circuits corresponding to the method and running one or more program modules).

In the architecture shown in FIG. 1, the storage server 10 may be shown as including the main device 50 and a plurality of storage devices 90 coupled to the main device 50, but the present invention is not limited thereto, for example, the main device 50 A casing/casing (eg, a computer casing, which may be composed of metal and/or one or more other materials) and at least a part (eg, a part or all) of the plurality of storage devices 90 , for example, the storage server 10 may further include a plurality of storage devices coupled to the host device 50 and At least one switch circuit (eg, one or more switch circuits) between at least a portion (eg, a portion or all) of the device 90 for performing the switching between the host device 50 and at least a portion of the plurality of storage devices 90 described above signal switching between.

According to some embodiments, the processing circuit 52 or the storage interface circuit 54 of the running program module 52P may configure at least a portion (eg, a portion or all) of the plurality of storage devices 90 to form a storage pool structure, However, the present invention is not limited thereto. According to some embodiments, the processing circuit 52 or the storage interface circuit 54 of the running program module 52P can set at least a part (eg, a part or all) of the plurality of storage devices 90 to form the storage server 10 Fault-tolerant disk array (Redundant Array of Independent Disks, RAID) (such as All Flash Array (All Flash Array, AFA)).

FIG. 2 is a schematic diagram of some implementation details of the storage device in the storage server 10 shown in FIG. 1 according to an embodiment of the present invention, wherein the storage device 100 can be used as the storage device of any one of the plurality of storage devices 90 . For example, in particular, each storage device of the plurality of storage devices 90 may be implemented according to the architecture of the storage device 100 , but the present invention is not limited thereto. The storage device 100 can be used to provide storage space to the host device 50 . Under the control of the host device 50 , one or more client devices can access (eg, read or write) user data in the storage space. The host device 50 Examples of the storage device 100 may include, but are not limited to, personal computers (such as desktop computers or notebook computers), and examples of the storage device 100 may include, but are not limited to, solid-state drives and various types of embedded memory devices (such as Flash memory storage or embedded multimedia memory card (embedded multi media card, EMMC) standard, etc.). According to this embodiment, the storage device 100 may include a controller (eg, the memory controller 110 ), and further include a non-volatile memory 120 , wherein the controller is used to control the operation of the storage device 100 and access the non-volatile memory 120 . The volatile memory 120 and the non-volatile memory 120 are used to store information. The non-volatile memory 120 may include at least one non-volatile memory element (eg, one or more non-volatile memory elements), Such as a plurality of non-volatile memory elements 120-1, 120-2, ..., and 122-N, where "N" may represent a positive integer greater than one, for example, the non-volatile memory 120 may be a flash memory The body, and the plurality of non-volatile memory elements 120-1, 120-2, . . . , and 122-N may be a plurality of flash memory chips or a plurality of flash memory dies, However, the present invention is not limited to this.

According to the present embodiment, the memory controller 110 can be used to control the access of the non-volatile memory 120 , so as to allow the host device 50 to access the non-volatile memory 120 through the memory controller 110 to store the server 10 As shown in FIG. 2, the memory controller 110 may include a processing circuit (eg, a microprocessor 112 ), a storage unit (eg, a read-only memory (ROM) 112M) , the control logic circuit 114, the random access memory 116, and the transmission interface circuit 118, wherein the above-mentioned elements can be coupled to each other by a bus bar. The random access memory 116 is implemented by a static random access memory (static RAM, SRAM), but the invention is not limited to this, the random access memory 116 can be used to provide internal storage space for the memory controller 110. For example, the random access memory 116 can be used as a buffer memory for buffering data, in addition, the ROM 112M of this embodiment is used to store the code 112C, and the microprocessor 112 is used to execute the The program code 112C controls the operation of the memory controller 110 to control the access of the non-volatile memory 120 to allow the host device 50 to access the non-volatile memory 120 through the memory controller 110 . Note that, in some examples, the code 112C may be stored in random access memory 116 or any type of memory. Additionally, control logic 114 may be used to control non-volatile memory 120, and may include A data protection circuit (not shown) for data protection and/or error correction, but the invention is not limited thereto, the transmission interface circuit 118 may conform to a specific communication standard (eg, Serial Advanced Technology Attachment standard, External Component Interconnection standard) , Fast External Component Interconnect Standard, Non-Volatile Memory Storage Device Standard, Non-Volatile Memory Storage Device External Access Standard, Small Computer System Interface Standard, Universal Flash Memory Storage Standard, etc.), and can be based on the A specific communication standard is used for communication, for example, for the storage device 100 to communicate with the host device 50 , wherein the storage interface circuit 54 can conform to the specific communication standard for the host device 50 to communicate with the storage device 100 .

According to some embodiments, the host device 50 may transmit a plurality of host commands and a plurality of corresponding logical addresses to the memory controller 110 to access the storage device 100, and the memory controller 110 receives the plurality of host commands and The plurality of logical addresses, and the plurality of host commands are converted into a plurality of memory operation commands (for brevity, may be referred to as a plurality of operation commands), and the plurality of operation commands are further used to control the non-volatile memory. The volatile memory 120 is used for reading, writing/programming, etc., on storage units (eg, data pages) within the non-volatile memory 120 having a plurality of physical addresses, wherein the plurality of physical addresses may be associated with the A plurality of logical addresses are associated, when the memory controller 110 is in any one of the non-volatile memory elements 122-n ( "n" may represent any integer in the range [1, N]), when an erase operation is performed on the non-volatile memory device 122-n, at least one block of the plurality of blocks can be erased , where each of the blocks may contain multiple pages (eg, data pages), and an access operation (eg, read or write) may be performed on one or more pages.

According to some embodiments, storage server 10 (eg, host device 50 ) may operate according to a Ceph solution, such that storage server 10 is defined in software as part of a distributed storage system, preferably a plurality of storage systems ( For example, multiple storage servers {10}, such as storage server 10, can be used to form a resource pool of a distributed storage system to support features such as fault tolerance, automatic failover control etc.) various types of access, such as block device type (for example in a distributed storage system to access an emulated block device), file system type (for example in a distributed storage system used to access a file system) and object type (eg, used to access an object in an object namespace in a distributed storage system), but the invention is not limited thereto. For example, a file can be stored as an object in an object namespace in a distributed storage system, and for example, a block of data corresponding to an address can be stored in a distributed storage system as an object namespace In order to implement features of a distributed storage system, the storage server 10 may be used to store additional information (eg, metadata and journals) in addition to data (eg, user data).

FIG. 3 is a migration control scheme based on an index object of a method for using an index object to perform seamless migration of a simple storage service according to an embodiment of the present invention, wherein the method can be applied to the storage server 10 . The storage server 10 can play the role of the second cloud and can be regarded as the destination of data migration, while the remote simple storage service compatible server can play the role of the first cloud and can be regarded as the source of data migration (source), for example, before the first time point T1, the user can access (eg read or write) data in the remote easy storage service compatible server, and find that he/she is in the remote easy storage The personal storage capacity in the service compatible server is almost exhausted and may be plagued by poor performance of the remote simple storage service compatible server.

In response to a request to migrate the user data of the user of the storage server 10 to the storage server 10 from the remote simple storage service compatible server, the indexing phase (eg, the INDEX phase) in the multiple phases of user data migration ) (for example, according to the user's request, the user data is migrated from the remote simple storage service compatible server to the storage server 10), the index object-based simple storage in the program module 52P running on the processing circuit 52 The storage service migration management module 53 can create and store a plurality of index objects (eg, virtual objects without object data) to the storage device layer of the storage server 10 to serve as user data in the storage server 10 The respective representatives of the plurality of normal objects (normal objects), and the respective access control lists (Access Control List, ACL) of the plurality of normal objects are migrated to the storage server 10 to serve as the respective storage of the plurality of index objects. Take the control list as if the plurality of index objects are a plurality of normal objects, so as to minimize the probability that any change of any normal object in the plurality of normal objects affects the correctness of the migration of the user data, wherein the storage device A layer includes at least one storage device (eg, one or more storage devices), such as a plurality of storage devices 90 , coupled to host device 50 . By migrating the respective ACLs of the plurality of normal objects to the storage server 10 as the respective ACLs of the plurality of index objects, the simple storage service migration management module 53 based on the index objects can utilize many index objects to emulate multiple normal objects. For example, the destination (eg, the storage server 10 ) can start to migrate at the first time point T1. After the first time point T1, when the user sends any other request When sent to a destination (eg, storage server 10 ) to attempt any changes to any of the normal objects, the storage server is based on whether one or more predetermined conditions are met (eg, whether any other requirement meets one or more predetermined rules) The server 10 can selectively proxy (eg, forward) any other request to the source (eg, a remote Easy Storage Service compatible server) and will respond with a response to any other request ) proxied from the source to the user.

During the data phase (eg, the DATA phase) that follows the index phase in the various phases, the index object-based simple storage service migration management module 53 may trigger one or more other program modules (eg, one or more migration agent programs). ) to migrate the respective object data of a group of normal objects among the plurality of normal objects to the storage device layer of the storage server 10 from the remote simple storage service compatible server, so as to be used as a group of index objects among the plurality of index objects to complete the migration of user data, for example, the destination (eg, storage server 10 ) can start the data phase at the second time point T2 and complete the migration at time point T3, assuming that the data phase In the meantime, the user transmits any other request to the destination (eg, the storage server 10 ) in an attempt to make any changes to any of the normal objects, similarly, based on whether one or more predetermined conditions (eg, any other request) are met conforms to one or more predetermined rules), the storage server 10 may selectively proxy (eg forward) any other request to the source (eg, a remote simple storage service compatible server) and will correspond to any one The response to other requests is forwarded (eg, forwarded) from the source to the user, for example, when any other request is a Get-object request (eg, "GetObject"), the storage server 10 may proxy (eg, forward) the request between the user and the source, especially if some of its respective object data has not been migrated to the destination (eg, during migration, when the storage server 10 When an object acquisition request is received, the respective object data of the index objects have not been copied to the destination and need to be copied to the destination) of the indexing agent (eg forwarding) the request. For better understanding, when any other request does not exist (eg, the user does not transmit any other request during the time points T1 and T3), the group of normal objects may include all of the plurality of normal objects, and the group An index object can contain all multiple index objects.

One or more migration agents may be implemented by one or more of the following program modules as long as they do not interfere with the practice of the present invention: (1) one or more program modules in the program modules 52P running on the processing circuit 52; (2) In the case of a multi-node architecture with a plurality of processing circuits {52} in the main device 50, among the plurality of processing circuits {52} operating on one or more other processing circuits one or more program modules, wherein the plurality of processing circuits {52} include the processing circuit 52; and (3) In the case of a multi-host architecture with multiple master devices {50} in the storage server 10, the operation of one or more other master devices in the multiple master devices {50} is One or more program modules on one or more corresponding processing circuits, wherein the plurality of master devices {50} include the master device 50; However, the present invention is not limited to this.

According to some embodiments, the method and related apparatus can be used to utilize Ceph's software library (eg "librados" software library) to perform certain basic operations for the Ceph architecture, wherein the Ceph software library can be used to provide client applications Provides direct access to an object-based reliable autonomic distributed object store (RADOS) storage system, and is a feature of certain Cephs (such as a reliable autonomic distributed object store (RADOS block device, RADOS) RBD), Reliable Automated Distributed Object Storage Gateway (RADOS gateway, RGW), Ceph File System, etc.) provide a foundation, and Reliable Automated Distributed Object Storage can be part of the Ceph Distributed Storage System and can be regarded as A utility for interacting with a Ceph object storage cluster, but the invention is not limited thereto.

FIG. 4 is a schematic diagram of some related operations of the index object-based migration control scheme shown in FIG. 3 according to an embodiment of the present invention. "Storage Service Compatible Storage" and "Bigtera Storage") can be used as examples of the Remote Simple Storage Service Compatible Server and the Storage Server 10, respectively. For the sake of example, labeled as "S3 client") respectively access the remote easy storage service compatible server and the storage server 10, for example, each remote easy storage service compatible server and the storage server 10 can Provides and is compatible with simple storage services, especially object storage, to handle large amounts of unstructured data, and can be implemented through a representational state transfer (REST) based application programming interface ( application program interface, API) (such as a web service application program interface that adheres to the constraints of the presentation-level state transition architecture, such as a presentation-level state transition application programming interface (RESTful API)) and by utilizing the hypertext transfer protocol ( hypertext transfer protocol (HTTP)/hypertext transfer protocol secure (HTTPS) to be accessed. For certain Simple Storage Service features, each Remote Simple Storage Service compatible server and

The storage server 10 may be used to store one or more objects in a bucket (eg, a container of objects), and any of the one or more objects may contain a file and optionally Contains any metadata describing the file, where the object prefix of any object may contain one or more symbols "/" to support the folder concept to simulate a folder, and , each remote simple storage service compatible server and storage server 10 may have at least one Ceph object gateway (labeled "gateway" for brevity), such as built on librados to provide A gateway application representing layer state transition provides at least one object storage interface to the Ceph storage cluster, wherein the at least one Ceph object gateway supports a remote simple storage service compatible server and the storage server 10 is a simple storage Service compatible and Swift compatible. As shown in the bottom half of Figure 4, during the migration of user data (marked as "data" in the open arrow for brevity), the user may send any other request (eg, a GET request) (by From using the simple storage service client) to the storage server 10 to request part of the user data, the storage server 10 can proxy (eg forward) any other request (eg, GET request) to the remote simple storage service partner. Container server, and proxy (eg forwarding) corresponding to any other request (such as part of the data from the remote ESS compatible server to the user (eg the user's ESS client), for brevity See, on some dashed arrows labeled "Data") this response. For the sake of brevity, the similar content of this embodiment will not be repeated in detail here.

According to some embodiments, in an INDEX stage (eg, INDEX period), by default, each index object of the plurality of index objects may be created to have a source (src) object (eg, in the first Certain attributes of a corresponding object in the cloud, such as a normal object, where the index object acts as a representative of the normal object, and the attributes may include the last modification time (last modified time) of the source object -modified-time, mtime) (e.g. source modification time (src-mtime), such as bigtera-scr-mtime), size of source object (e.g. source size (src-size), such as bigtera-src-size), and source object The entity tag (Etag) (eg a file hash for integrity checking, such as MD5 sum), such as the source entity tag (src-etagt), such as bigtera-src-etag).

FIG. 5 is a schematic diagram of some objects and some related features of the index object-based migration control scheme shown in FIG. 3 according to an embodiment of the present invention. During the migration of user data, the source end (such as the remote simple A storage service compliant server) may contain one or more buckets (eg, one or more containers of objects), such as a remote bucket, and the destination (eg, storage server 10) may contain a or receptacles (eg, one or more containers of objects), such as the one shown above the distal receptacle, wherein the request and the response, respectively, represent a The request received by the simple storage service client) and the response returned by the storage server 10 to the user (such as the user's simple storage service client), for example, the remote storage may contain a plurality of normal Objects, for better understanding, are shown as small circles in the distal reservoir. In addition, the repository of the destination may contain at least a portion (eg, a portion or all) of index items, and in particular one or more normal items, depending on the process of the migration (eg, the percentage of the migration complete, such as 1%, 2%, etc.) and/or the presence of any other requirement.

Before migrating the object data of any one of the index objects of at least a part of the plurality of index objects, the storage server 10 may set the destination store to have the same name as the remote store name (eg, the store name of the remote store) ) the same store name and having the same user credentials as a remote user credential (e.g. user credentials of a remote store) and especially having fake usage information, such as fake usage (such as size, number of objects, etc.) to serve as temporary user information for migration phases (eg, indexing and data phases). Initially, before migrating the object data of any index object, the storage server 10 may first create a plurality of index objects with temporary attributes such as dummy index objects (eg, without object data) in the destination storage. virtual object), when a normal object among a plurality of normal objects has been migrated to the destination or an operation on the normal object has been performed on the destination (for example, the object data of the normal object has been migrated, or the user has overridden the normal object), the normal object may exist in the storage (as shown in Figure 5), in addition, for each index object of a plurality of index objects, the temporary attributes may contain: (1) The last modification time of the source object (such as the last modification time of the source object), such as bigtera-src-mtime; (2) The size of the source object (such as the size of the source object), such as bigtera-src-size; (3) Entity tags of source objects (such as entity tags of source objects), such as bigtera-src-etag;

Wherein any of these temporary attributes (eg each attribute) can be implemented as an extended attribute (x-attribute/XATTR/xattr), but the invention is not limited to this, once the object data of the source object is migrated to the destination or The user overwrites the object data at the destination, and the destination (for example, the storage server 10 ) can remove the temporary attributes listed above. For the sake of brevity, similar contents of this embodiment are not repeated here. Detailed Description.

According to some embodiments, taking the normal object shown in the upper right of FIG. 5 as an example of the source object, when the object data of the source object is migrated to the destination or the user overwrites the object data at the destination, the storage server The device 10 may invalidate an index object that is a representative of a normal object (eg, by removing its temporary attributes) and/or replace the index object with a normal object in the store at the destination, but the invention is not limited to this.

Generally speaking, changing the ACL of an object does not change the modification time of the object, and the data migration method of the related art does not seem to take this into account when proposed, so it cannot properly handle deleted objects, read-only objects, etc. . The method and associated apparatus of the present invention may suitably operate under any of a variety of conditions, further implementation details of which are described in the examples below.

FIG. 6 is a schematic diagram of a migration agent control scheme of a method for using index objects to perform seamless migration of simple storage services according to an embodiment of the present invention, in response to migrating user data from a remote simple storage service compatible server to storage Requirement of server 10 that storage server 10 may migrate management module 53 using a simple storage service based on indexing objects during a set-up phase (eg, a setup period) before the indexing phase of the multiple phases To perform preprocessing of user data migration, for example, the preprocessing may include the operations of steps S01 to S05, but the present invention is not limited thereto.

In step S01, the simple storage service migration management module 53 based on the index object, for example, can be a group of simple storage service users (for example, a single user, such as the above-mentioned user, or includes a group of simple storage service users immediately after the request is triggered. Multiple users of the above-mentioned users) start simple storage service migration.

In step S02, the index object-based simple storage service migration management module 53 can migrate respective user credentials and respective settings (eg, quota) of the group of simple storage service users.

In step S03, the index object-based simple storage service migration management module 53 can create one or more storages in the destination for each user of the group of users, and set relevant migration information, for example, the migration The information may include proxy parameters (eg, remote user credentials to proxy (eg, forward)) and other proxy parameters to direct the destination (eg, storage server 10 ) to proxy (eg, forward) any other request to A source (eg, a remote Easy Storage Service compatible server), and forwards a response corresponding to any other request from the source proxy (eg, forwarding) to the user.

In step S04, for each storage of the one or more storages, the simple storage service migration management module 53 based on the index object can migrate the storage access control list of one of the source ends corresponding to the remote storage (bucket ACL), bucket metadata, and bucket quota as the bucket access control list, bucket metadata, and bucket quota for this bucket of the one or more buckets body quota.

In step S05, the simple storage service migration management module 53 based on the index object can switch at least one endpoint to access user data from at least one cluster of the remote simple storage service compatible servers to the storage server 10 of at least one cluster.

In step S06, after the setup phase, the index object-based simple storage service migration management module 53 may create an index storage job (eg, an INDEX job) for each of the one or more storages, To perform the migration of the corresponding remote storage of the source end to the storage of the one or more storages, for example, the simple storage service migration management module 53 based on the indexing object can set the storage to the indexing stage (eg INDEX stage).

For better understanding, any migration agent (eg, each agent) of one or more migration agents can be regarded as a program module that actually performs object migration, such as a simple storage service migration management module based on indexed objects Group 53 may create multiple job types (eg, multiple index jobs corresponding to the index phase (eg, multiple INDEX jobs) and multiple data jobs (eg, multiple DATA jobs) corresponding to the data phase), and especially Refers to sending and/or assigning multiple job types to one or more migration agents for data phase and index phase operations, respectively. Assuming that the one or more agents include multiple agents, the multiple agents may be implemented to run on multiple cluster nodes, respectively, with an agent running on each of the multiple cluster nodes, Therefore, the index object-based simple storage service migration management module 53 can simultaneously transfer user data (eg, the respective object data of the object) by means of one or more migration agents (eg, multiple migration agents) in parallel Migrate from source to destination.

In step S07, the simple storage service migration management module 53 based on the index object can perform the INDEX work by one of the one or more migration agents, so that each object at the source (eg, a plurality of normal object) does the following: (1) Create a virtual object with temporal properties (eg source object last modified time, such as bigtera-src-mtime; source object size, such as bigtera-src-size; and source object entity tag, such as bigtera-src-etag) (for example, an object without object data) as one of the index objects among multiple index objects, where temporary attributes can be false attributes of each object at the source (such as false last modified time, false entity tags such as fake file hash and fake size), rather than the real attributes of each object at the source, and thus can be considered special attributes; (2) Migrate the object access control list of the object from the source end to the destination;

All index objects of the plurality of index objects are created after all operations in step S07 are completed.

When an indexing job (eg, an INDEX job) is completed, the indexed object-based simple storage service migration management module 53 may create a data job (eg, a DATA job) for each of the one or more storages to perform Migration of the corresponding distal reservoir of the source end to the one of the one or more reservoirs.

In step S08, for each of the one or more storages, the index object-based simple storage service migration management module 53 may execute DATA by an agent of one or more migration agents work for each object at the source (e.g., in the absence of any other requirement, each of a plurality of normal objects; or in the presence of any other requirement, each normal of the set of normal objects object, where any of the other requirements may cause an object to be overridden in the destination as a normal object in the repository) to do the following: (1) Migrate the object data of the object from the source to the destination;

where the migration agent can migrate the latest version of the storage quota (eg, in the absence of any other requirement, the original storage quota of the corresponding remote storage; or in the presence of any other requirement) , the storage quota updated in response to any changes made by the user) as the storage quota for the storage at the end of the job, for example, the index object-based simple storage service migration management module 53 can use the storage Set to data stage (eg DATA stage).

Among all the reservoirs corresponding to the set of Easy Storage Service users, while one of the reservoirs is being processed in the indexing phase (eg, the INDEX phase), the other one of the reservoirs is available at is being processed in a data phase (eg DATA phase). Assuming that the one or more bins contain multiple bins, when one bin of the multiple bins is being processed in an indexing stage (eg, the INDEX stage), another bin of the multiple bins is available in the data is being processed in a stage (for example, a DATA stage), and for the sake of brevity, the similar content of this embodiment will not be repeated in detail here.

According to some embodiments, the indexing phase (eg, the INDEX phase) may be much shorter than the data phase (eg, the DATA phase). During the indexing phase, the simple storage service migration management module 53 based on the indexed object may The migration agent performs INDEX work to first list remote objects (such as multiple normal objects at the source end) for creating virtual objects and migrating object ACLs. Since any other requirements may exist during the indexing phase (eg, the INDEX phase), different situations need to be considered, such as a first situation where the index object has not been created, and a second situation where the storage server 10 is creating an index object, etc. etc., for example, processing for PUT/POST requests should take place at a local site (eg, a destination such as storage server 10), and storage server 10 can perform this processing for PUT/POST requests and Also proxy the PUT/POST request to a remote site (e.g. origin, such as a Remote Simple Storage Service compatible server) for storage usage and List-Objects requests ( e.g. ListObjects request) results are correct. In the event that any other requirement exists in the indexing phase (eg, the INDEX phase) where the user can overwrite one or more objects, if an indexing object is overwritten by the user, the object is no longer an object index object, so the migration agent will not migrate object data for that object in the data phase (such as the DATA phase), in addition, before the data phase (such as the DATA phase), all index objects may have been created (including their object access control list). In the data phase (eg, the DATA phase), the indexed object-based simple storage service migration management module 53 may perform DATA work by the migration agent in step S08 to list local objects (eg, objects in local locations, such as index object), and migrates the object data for each local object. Generally speaking, most operations can be performed normally, except for GET/HEAD object requirements, in order to solve the racing hazard, the storage server 10 can use atomic (atomic) operations to create index objects and by the index objects To migrate data (such as the object data of multiple normal objects at the source), for example, when the migration agent is creating an index object or migrating data about an index object, if a target object of the PUT operation already exists or does not have the above temporary attribute, the PUT operation is canceled, which means that the user has overwritten the object. For proxying (eg, forwarding) index objects in a reliable automated distributed object storage gateway, in the presence of any other requirements, when a client device (eg, a simple storage service client) operates on an index object or While the store for the destination is in the INDEX phase, it may be required that the destination (eg, storage server 10 ) can proxy any other request to a remote location (eg, a source, such as a remote simple storage service) server), and transmits the proxy response (for example, the response corresponding to any other request) to the client device. For the sake of brevity, the similar content of these embodiments will not be described in detail here.

FIG. 7 is a schematic diagram of an example of the Ceph object storage architecture with the index object-based simple storage service migration management module 53 shown in FIG. 1 , and the program module 52P running on the processing circuit 52 shown in FIG. 1 Can include a Ceph Object Gateway module (labeled "Ceph Object Gateway" in Figure 7 for brevity), where the Ceph Object Gateway module includes sub-modules, such as a reliable Automated decentralized object storage gateway module (denoted "RADOS gateway" for brevity and better understanding) and librados software library (denoted "librados" for brevity), and index-based Object Simple Storage Service Migration Management Module 53 is implemented in the Reliable Automated Distributed Object Storage Gateway module. According to this embodiment, client devices (labeled as "clients") can use the Hypertext Transfer Protocol (eg Simple Storage Service or Swift Application Programming Interface) (labeled as "HTTP (S3 or Swift API)" for brevity) to access storage server 10, which can use multi-node architecture, multi-host architecture and the like, but the present invention is not limited thereto.

As shown in FIG. 7, a reliable automated distributed object storage (eg, an object-based storage system of a reliable automated distributed object storage) of the storage server 10 may include at least one public network (eg, one or multiple public networks) to couple to multiple object storage device (OSD) nodes (labeled "OSD nodes" for brevity) of the host device 50 shown in FIG. 1, and also include Other parts of the Ceph object storage architecture, such as at least one cluster network (eg, one or more cluster networks) coupled to the plurality of object storage device nodes, an odd number of monitors constituting a quorum monitor and so on. The at least one public network can be used to achieve a first predetermined speed such as gigabits per second (designated "1G" for brevity), and the at least one cluster network can be used to achieve a second predetermined speed such as Ten gigabits per second (denoted "10G" for brevity), where the second predetermined speed is as many bits per second as possible, the plurality of object storage device nodes may include journals for ) (e.g. various types of journals, such as object storage device journaling, etc.), some solid state drives, and more traditional hard drives, and especially include non-fault-tolerant disk array drive architectures such as cluster disks/drives (just a bunch of disks/drives, JBOD)), for the sake of brevity, the similar content of these embodiments will not be repeated in detail here.

For the application programming interface/operation of the state transition of the simple storage service presentation layer, the examples of the object-related operations of the simple storage service action may include but are not limited to: (1) Object listing request (such as ListObjects request); (2) Object header (Head-Object) requirements (for example, the HeadObject request used to obtain object properties); (3) Object acquisition request (such as GetObject request); (4) The object obtains the access control list (Get-Object-ACL) request (for example, GetObjectaACL request); (5) Object placement (Put-Object) requirements (such as PutObject request); (6) Create a multi-component upload (Create-Multipart-Upload) request (such as CreateMultipartUpload request); (7) Complete multi-component upload (Complete-Multipart-Upload) requirements (for example, CompleteMultipartUpload request); (8) Object placement access control list (Put-Object-ACL) request (for example, PutObjectACL request); (9) Object deletion (Delete-Object) request (such as DeleteObject request); Some of these subsequent embodiments may focus on the object-related operations listed above.

FIG. 8 is a schematic diagram of an object listing control scheme of a method for using index objects to perform seamless migration of simple storage services according to an embodiment of the present invention. As shown in FIG. 8, it is used for processing during user data migration. A workflow of one or more object listing requirements may include steps S10-S14, wherein S14 may include some sub-steps such as steps S14A-S14C.

In step S10, the storage server 10 (eg, the simple storage service migration management module 53 based on indexed objects) may receive an object listing request among one or more object listing requests from the user.

In step S11 , in response to the object listing request, the simple storage service migration management module 53 based on the index object may check for the object listing request corresponding to the object listing in at least one of the storages in the storage device layer of the storage server 10 Whether a reservoir (eg, a destination's reservoir) is being processed in an indexing phase (eg, an INDEX phase), where the at least one reservoir may represent at least one container (eg, one or more containers) of the item, and if so , go to step S12 ; if not, go to step S14 (eg, step S14A), for example, the object lists the object that requires the storage body to be listed.

In step S12, since the storage is being processed in the indexing phase (eg, the INDEX phase), the simple storage service migration management module 53 based on the indexed object can control the storage server 10 to list the object as a requesting agent (eg. forwarding) to the source (such as a remote Simple Storage Service compatible server).

In step S13, the simple storage service migration management module 53 based on the index object can control the storage server 10 to send a response agent corresponding to the object listing request from the source (eg, a remote simple storage service compatible server) (eg forwarding) to the user.

In step S14 , since the storage is not being processed in the indexing phase (eg, the INDEX phase), the simple storage service migration management module 53 based on the indexed object can control the storage server 10 to perform processing according to the object listing request At least one normal action, no need to list the object and request an agent (such as forwarding) to the source (such as a remote simple storage service compatible server). For example, the simple storage service migration management module 53 based on the index object can control the storage The server 10 selectively lists a local object listing result (eg, a local object listing result at the local location, such as the result of executing the object listing request at the local location) based on whether a related ACL check is passed or not. Post back to the user.

In step S14A, the index object-based simple storage service migration management module 53 can check whether the ACL check is passed, and if so, go to step S14B; if not, go to step S14C, for example, when the ACL When indicating that the user has the permission to apply the object listing request to the storage (for example, the permission to execute the obtaining result of the object listing request), the simple storage service migration management module 53 based on the index object can determine the ACL The check is passed, for another example, when the ACL indicates that the user does not have the permission to apply the object listing request to the store (such as the permission to execute the fetch result of the object listing request), the simple storage service based on the index object The migration management module 53 can determine that the ACL check fails.

In step S14B, the index object-based simple storage service migration management module 53 can control the storage server 10 to return the local object listing result to the user.

In step S14C, the simple storage service migration management module 53 based on the index object can control

The storage server 10 prevents local object listing results from being passed back to the user.

For better understanding, the method may be described using the workflow shown in FIG. 8, but the present invention is not limited thereto. According to some embodiments, one or more steps in the workflow shown in FIG. 8 may be is added, deleted, or changed, for example, at least a part (eg, part or all) of the workflow may be performed multiple times, and in particular any step that includes a check operation may be performed multiple times to generate multiple Check the results to trigger the execution of some subsequent steps, respectively.

FIG. 9 is a schematic diagram of an object header control scheme of a method for using index objects to perform seamless migration of simple storage services according to an embodiment of the present invention. As shown in FIG. 9 , it is used to process an A workflow required by one or more object headers may include steps S20-S28, S29A, and S29B, wherein step S25 may include some sub-steps (eg, S25A-S25C), and step S27 may include some sub-steps (eg, S27A-S27C) .

In step S20 , the storage server (eg, the simple storage service migration management module 53 based on indexed objects) may receive an object header request from one or more object header requests from the user, for example, the object header request requires the acquisition purpose The object properties of an object in the storage of the ground, and the object can be regarded as a target object required by the object header, and the object properties can include: (1) The last modification time of the object, such as the last modification time of the object; (2) Object entity tag, such as the object's entity tag (eg file hash); (3) the size of the item, such as the size of the item; Where these attributes can be retrieved by an HTTP HEAD request.

In step S21 , in response to the object header request, the index object-based simple storage service migration management module 53 may check the at least one storage in the storage device layer of the storage server 10 that corresponds to the object header request. Whether a storage (eg, the storage of the destination) is being processed in the indexing stage (eg, the INDEX stage), if so, go to step S22; if not; go to step S26.

In step S22, since the storage is being processed in the indexing phase (eg, the INDEX phase), the simple storage service migration management module 53 based on the indexed object can check whether the object (eg, the target object required by the object header) is If yes, go to step S23; if not, go to step S28.

In step S23, in response to the existence of the object, the simple storage service migration management module 53 based on the index object can check whether the object (such as the target object required by the object header) is still an index object, and if so, go to step S23 S24; if not, go to step S27 (eg step S27A).

In step S24, since the object is still an index object, the simple storage service migration management module 53 based on the index object can check whether the object ACL of the object has been migrated;

, go to step S25 (eg step S25A); if no, go to step S29A, for example, there is usually a short time delay between creating a virtual object (such as the above virtual object) and migrating the object ACL Therefore, the simple storage service migration management module 53 based on the index object can perform the check operation of step S24 in the workflow to comply with the access control policy.

In step S25, the index object-based simple storage service migration management module 53 can control the storage server 10 to perform at least one special action according to the object header request, for example, the index object-based simple storage service migration management module 53 The storage server 10 can be controlled to selectively record in the index object (such as the index object mentioned in step S23 ) a special attribute (such as a source modification time, such as The attributes in bigtera-src-mtime, source entity tags, such as bigtera-src-etag, and source size, such as bigtera-src-size) are passed back to the consumer.

In step S25A, the index object-based simple storage service migration management module 53 can check whether the access control list check is passed, if yes, go to step S25B; if not, go to step S25C, for example, when the access control When the list indicates that the user has the permission to apply the object header request to the object (eg, the permission to obtain the result of executing the object header request), the index object-based simple storage service migration management module 53 can determine the ACL check By, for example, when the ACL indicates that the user does not have the permission to apply the object header request to the object (eg, the permission to obtain the result of executing the object header request), the simple storage service migration management based on the index object The module 53 can determine that the access control list fails.

In step S25B, the index object-based simple storage service migration management module 53 can control the storage server 10 to return the attributes recorded in the special attributes of the index object (eg, the index object mentioned in step S23 ) back to to the user.

In step S25C, the simple storage service migration management module 53 based on the index object can control the storage server 10 to prevent the properties to be recorded in the special properties of the index object (eg, the index object mentioned in step S23) Post back to the user.

In step S26, since the storage is not being processed in the indexing phase (eg, the INDEX phase), the simple storage service migration management module 53 based on the indexed object may check the object (eg, the target object required by the object header) Whether it is still an index object, if yes, go to step S25 (for example, step S25A); if not, go to step S27 (for example, step S27A). stage), all object ACLs have been migrated, so there is no need to obtain remote object ACLs again.

In step S27, since the object (eg, the target object requested by the object header) is no longer an index object (eg, the object becomes a normal object), the simple storage service migration management module 53 based on the index object can control the The storage server 10 performs at least one normal action according to the object header request. For example, the simple storage service migration management module 53 based on the index object can control the storage server 10 to check whether the selection is passed according to a related access control list. The object's local metadata (such as the object's local location's metadata) is automatically returned to the user.

In step S27A, the index object-based simple storage service migration management module 53 can check whether the ACL check is passed, and if so, go to step S27B; if not, go to step S27C, for example, the operation of step S27A can be similar operation in step S25A.

In step S27B, the simple storage service migration management module 53 based on the index object can control the storage server 10 to return the local metadata of the object to the user.

In step S27C, the index object-based simple storage service migration management module 53 can control the storage server 10 to prevent the local metadata of the object from being returned to the user.

In step S28 , the index object-based simple storage service migration management module 53 can control the storage server 10 to proxy (eg forward) the request (eg, object header request) to the source/remote location (eg, remote simple Storage Services Compliant Server) to enable the remote location to check the object ACL.

In step S29, the simple storage service migration management module 53 based on the index object can control the storage server 10 to proxy (eg forward) the response corresponding to the object header request to the user.

In step S29E, the simple storage service migration management module 53 based on the index object can control the storage server 10 to return an error notification (eg '503 Service Unavailable'), for example, because the migration agent will Migrate the ACL, which tells the client to try again later.

For better understanding, the method may be described using the workflow shown in FIG. 9, but the invention is not limited thereto. According to some embodiments, one or more steps in the workflow shown in FIG. 9 may be is added, deleted, or changed, for example, at least a part (eg, part or all) of the workflow may be performed multiple times, and in particular any step that includes a check operation may be performed multiple times to generate multiple Check the results to trigger the execution of some subsequent steps, respectively.

FIG. 10 is a schematic diagram of an object acquisition control scheme of a method for using index objects to perform seamless migration of simple storage services according to an embodiment of the present invention. As shown in FIG. 10, it is used to process an object during user data migration. A workflow for acquiring requests for or multiple objects may include steps S30-S36, wherein step S36 may include some sub-steps (eg, S36A-S36C).

In step S30 , the storage server 10 (eg, the simple storage service migration management module 53 based on indexed objects) may receive an object acquisition request from the one or more object acquisition requests from the user, for example, the object acquisition request request Obtain object data with attributes, such as the object data of an object in the storage of the destination and the object attributes of the object, and this object can be regarded as a target object of the object acquisition request.

In step S31 , in response to the object acquisition request, the simple storage service migration management module 53 based on the index object may check a storage corresponding to the object acquisition request in the at least one storage in the storage device layer of the storage server. Whether (eg, the destination's storage) is being processed in the indexing stage (eg, the INDEX stage), if so, go to step S32; if not, go to step S33.

In step S32, since the storage is being processed in the indexing phase (eg, the INDEX phase), the simple storage service migration management module 53 based on the indexed object can check whether the object (eg, the target object of the object acquisition request) is If yes, go to step S33; if not, go to step S34. For part of the workflow between steps S32 and S34, examples of the reasons why the object does not exist may include: the index object has not been created; and the index object has been deleted by the user.

In step S33, the index object-based simple storage service migration management module 53 can check whether the object (for example, the target object of the object acquisition request) is still an index object, if so, go to step S34; if not, then Proceed to step S36 (eg, step S36A).

In step S34, the index object-based simple storage service migration management module 53 can control the storage server 10 to proxy (eg forward) the request (eg, the object acquisition request) to the source/remote location (eg, remote Simple Storage Service compatible server).

In step S35, the simple storage service migration management module 53 based on the index object can control the storage server 10 to proxy (eg forward) the response corresponding to the object acquisition request to the user.

In step S36, since the object (eg, the target object of the object acquisition request) is no longer an index object (eg, the object becomes a normal object), the simple storage service migration management module 53 based on the index object can control the storage The server 10 performs at least one normal action according to the object acquisition request. For example, the simple storage service migration management module 53 based on the index object can control the storage server 10 to selectively check whether the access control list is passed or not. Local data for the object (such as the data for the object at the local location) is returned to the user.

In step S36A, the index object-based simple storage service migration management module 53 can check whether the access control list check is passed, if yes, go to step S36B; if not, go to step S36C, for example, when the access control When the list indicates that the user has the permission to apply the object acquisition request to the object (eg, the permission to obtain the result of executing the object acquisition request), the simple storage service migration management module 53 based on the index object can determine the access control list The check is passed, for example, when the ACL indicates that the user does not have the right to apply the object acquisition request to the object (such as the right to obtain the result of executing the object acquisition request), the simple storage service based on the index object The migration management module 53 can determine that the access control list fails.

In step S36B, the simple storage service migration management module 53 based on the index object can control the storage server 10 to return the local data of the object to the user.

In step S36C, the simple storage service migration management module 53 based on the index object can control the storage server 10 to prevent the local data of the object from being returned to the user.

For better understanding, the method may be illustrated using the workflow shown in FIG. 10, but the invention is not limited thereto. According to some embodiments, one or more steps in the workflow shown in FIG. 10 may be is added, deleted, or changed, for example, at least a part (eg, part or all) of the workflow may be performed multiple times, and in particular any step that includes a check operation may be performed multiple times to generate multiple Check the results to trigger the execution of some subsequent steps, respectively.

FIG. 11 is a schematic diagram of an object acquisition ACL control scheme of a method for using index objects to perform seamless migration of simple storage services according to an embodiment of the present invention. As shown in FIG. 11, during user data migration A workflow for processing one or more objects to obtain an ACL request may include steps S40-S48, wherein step S45 may include some sub-steps (eg, S45A-S45C).

In step S40 , the storage server 10 (eg, the simple storage service migration management module 53 based on the index object) may obtain an ACL from an object in one or more object obtain ACL requests received from the user request, for example, the object get access control list request request to get the object access control list (such as the access control list of a bank in the destination's store), and this object can be regarded as the object get access control list request of a target object.

In step S41 , in response to the request for obtaining the ACL for the object, the simple storage service migration management module 53 based on the index object may check the at least one storage in the storage device layer of the storage server 10 corresponding to the object obtained Whether a bank requested by the access control list (eg, the storage of the destination) is being processed in the index stage (eg, the INDEX stage), if so, go to step S42; if not, go to step S45 (eg, step S45A) .

In step S42, since the storage is being processed in the indexing phase (eg, the INDEX phase), the simple storage service migration management module 53 based on the indexed object may check the object (eg, the object required to obtain the access control list). whether the target object) exists, if yes, go to step S43; if not, go to step S46.

In step S43, the simple storage service migration management module 53 based on the index object can check whether the object (for example, the target object required by the object to obtain the access control list) is still an index object, and if so, go to step S44; If not, go to step S45 (eg step S45A).

In step S44, since the object is still an index object, the simple storage service migration management module 53 based on the index object can check whether the object ACL of the object has been migrated, and if so, go to step S45 (for example, step S45A ); if not, go to step S48.

In step S45, the simple storage service migration management module 53 based on the index object can control the storage server 10 to perform at least one normal action according to the request for obtaining the ACL for the object, for example, the simple storage service migration based on the index object The management module 53 can control the storage server 10 to selectively return the local ACL of the object (eg, the ACL of the object at the local location) to the relevant ACL check whether it passes or not. user.

In step S45A, the index object-based simple storage service migration management module 53 can check whether the ACL check is passed, if yes, go to step S45B; if not, go to step S45C, for example, when the ACL check is passed Indicates that the user has the permission to apply the ACL request for the object to the object (for example, the permission to execute the result of the ACL request for the object), the index object-based simple storage service migration management module 53 It can be determined that the ACL check is passed, and for example, when the ACL indicates that the user does not have the authority to apply the ACL request for the object to the object (for example, get the ACL request to execute the object) When the access control list check is not passed, the simple storage service migration management module 53 based on the index object can determine that the ACL check is not passed.

In step S45B, the simple storage service migration management module 53 based on the index object can control the storage server 10 to return the local access control list of the object to the user.

In step S45C, the simple storage service migration management module 53 based on the index object can control the storage server 10 to prevent the local access control list of the object from being returned to the user.

In step S46, the simple storage service migration management module 53 based on the index object can control the storage server 10 to proxy (eg forward) the request (eg, the request for the object to obtain an ACL) to the source/remote location (eg Remote Simple Storage Service compatible server).

In step S47, the simple storage service migration management module 53 based on the index object can control the storage server 10 to proxy (eg forward) the response corresponding to the object's request for obtaining the ACL to the user.

In step S48, the simple storage service migration management module 53 based on the index object can control the storage server 10 to return an error notification (eg '503 Service Unavailable').

For better understanding, the method may be described using the workflow shown in FIG. 11, but the invention is not limited thereto. According to some embodiments, one or more steps in the workflow shown in FIG. 11 may be is added, deleted, or changed, for example, at least a part (eg, part or all) of the workflow may be performed multiple times, and in particular any step that includes a check operation may be performed multiple times to generate multiple Check the results to trigger the execution of some subsequent steps, respectively.

FIG. 12 is a schematic diagram of an object placement control scheme of a method for using index objects to perform seamless migration of simple storage services according to an embodiment of the present invention. As shown in FIG. 12, it is used to process an object during user data migration. A workflow for writing related requirements of one or more object data (such as one or more object placement requirements, one or more create multi-component upload requirements, and one or more complete multi-component upload requirements) may include steps S50-S58 , wherein step S56 may include some sub-steps (eg S56A~S56C).

In step S50 , the storage server 10 (eg, the simple storage service migration management module 53 based on indexed objects) may receive from the user a request for writing data in one or more objects (eg, one or more placement requests, a One or more Create Multi-Component Upload Requests, and one or more Complete Multi-Component Upload Requests) related requests (e.g. an Object Placement/Create Multi-Component Upload/Complete Multi-Component Upload requests, such as an Object Placement request, a creation of a multi-part upload request, or a completion of a multi-part upload request), for example, the object data write related request requires an object to be written to the destination storage, and this object can be regarded as the object data Write a target object of the relevant request.

In step S51, in response to the relevant requirements for writing the object data, the simple storage service migration management module 53 based on the index object can check the at least one storage in the storage device layer of the storage server 10 corresponding to the object data writing Whether a bank (eg, the destination's bank) of the input related requirements is being processed in the indexing stage (eg, the INDEX stage), if so, go to step S52; if not, go to step S56 (eg, step S56A).

In step S52, the simple storage service migration management module 53 based on the index object can control the storage server 10 to proxy (eg forward) the request (eg, the request for writing the object data) to the source end (eg, the remote simple Storage Services Compliant Server) to enable the remote Easy Storage Services Compliant server to check the object's ACL.

In step S53, the index object-based simple storage service migration management module 53 can control whether the request (for example, the object data writing related request) is successful without any error, if so (for example, a response received from the source, such as Corresponding to the response to the relevant request for object data writing, indicating that no error has occurred), go to step S54; if no (for example, a response received from the source, such as a response corresponding to the relevant request for object data writing, indicating one or multiple errors and one error response), then go to step S57.

In step S54, the simple storage service migration management module 53 based on the index object can check whether the object (for example, the target object for which the object data is required to be written) exists, if yes, go to step S55; if not, go to step S55 Step S56 (eg, step S56A).

In step S55, the simple storage service migration management module 53 based on the index object can check whether the object ACL of the object has been migrated, and if so, go to step S56 (eg, step S56A); if not, go to step S58 .

In step S56, the index object-based simple storage service migration management module 53 can control the storage server 10 to perform at least one normal action according to the object data writing requirements, for example, the index object-based simple storage service migration management The module 53 can selectively write data (eg, the object data carried by the object data write related request) as the object data of the object by checking whether it passes or not according to a related access control list.

In step S56A, the simple storage service migration management module 53 based on the index object can check whether the ACL check is passed, if yes, go to step S56B; if not, go to step S56C, for example, when the ACL check is When instructing the user to have the permission to write the object data into the relevant request and apply it to the object (the permission to write the object data into the object data carried by the relevant request), the simple storage service migration management model based on the index object Group 53 can determine that the ACL check is passed, and for example, when the ACL indicates that the user does not have the permission to apply the object data to the relevant request (the object data is written in the relevant request) When the access control list check is not passed, the simple storage service migration management module 53 based on the index object can determine that the ACL check is not passed.

In step S56B, the simple storage service migration management module 53 based on the index object can control the storage server 10 to write the data (for example, the object data carried in the relevant request for writing the object data) as the object's Object data.

In step S56C, the simple storage service migration management module 53 based on the index object can control the storage server 10 to prevent the data (such as the object data carried by the object data writing in the relevant request) from being written as the object's The object data.

In step S57 , the index object-based simple storage service migration management module 53 can control the storage server 10 to proxy (eg forward) the error response corresponding to the object data writing related request to the user.

In step S58, the simple storage service migration management module 53 based on the index object can control the storage server 10 to migrate the object ACL of the object from the source end to the destination.

For better understanding, the method may be described using the workflow shown in FIG. 12, but the invention is not limited thereto. According to some embodiments, one or more steps in the workflow shown in FIG. 12 may be is added, deleted, or changed, for example, at least a part (eg, part or all) of the workflow may be performed multiple times, and in particular any step that includes a check operation may be performed multiple times to generate multiple Check the results to trigger the execution of some subsequent steps, respectively.

FIG. 13 is a schematic diagram of an object placement ACL control scheme of a method for using index objects to perform seamless migration of simple storage services according to an embodiment of the present invention. As shown in FIG. 13, during user data migration A workflow for processing one or more object placement ACL requests may include steps S60-S65, wherein step S64 may include some sub-steps (eg, S64A-S64C).

In step S60 , the storage server 10 (eg, the simple storage service migration management module 53 based on indexed objects) may receive an object placement ACL request from the user of one or more object placement ACL requests, For example, the object placement ACL request requires writing or updating an object ACL (such as an object's ACL in the destination's storage), and this object can be considered as the object placement ACL A target object requested.

In step S61, in response to the object placement ACL request, the simple storage service migration management module 53 based on the index object can check the at least one storage in the storage device layer of the storage server 10 corresponding to the object Whether a bank (eg, the bank of the destination) requested by the put ACL is being processed in the index phase (eg, the INDEX phase), if so, go to step S62, if not, go to step S64 (eg, step S64A) ).

In step S62, the index object-based simple storage service migration management module 53 can control the storage server 10 to proxy (eg forward) the request (eg, the object placement ACL request) to the source (eg, the remote simple storage service compatible server).

In step S63, the index object-based simple storage service migration management module 53 can check whether the request (eg, the object placement ACL request) is successful without any error, and if so (eg, a response received from the source, such as Corresponding to the response of the object placement ACL request, indicating that no error occurs), go to step S64 (eg, step S64A); if not (eg, a response received from the source, such as corresponding to the object placement ACL request) The response is an error response indicating one or more errors), then go to step S65.

In step S64, the index object-based simple storage service migration management module 53 can control the storage server 10 to perform at least one normal action according to the object placement ACL request, for example, index object-based simple storage service migration The management module 53 can control the storage server 10 to selectively update the object storage with the new/latest ACL carried by the object placement ACL request according to whether an associated ACL check is passed or not. Fetch a control list (eg, the access control list for the target object).

In step S64A, the index object-based simple storage service migration management module 53 can check whether the ACL check is passed, and if so, go to step S64B; if not, go to step S64C, for example, when the access control list When indicating that the user has the permission to apply the object's ACL request to the object (for example, the permission to write or update the object's ACL), the simple storage service migration management module 53 based on the index object can determine The ACL check is passed, for example, when the ACL indicates that the user does not have the permission to place the ACL request on the object (such as the permission to write or update the ACL of the object) ), the simple storage service migration management module 53 based on the index object can determine that the ACL check fails.

In step S64B, the index object-based simple storage service migration management module 53 can control the storage server 10 to update the object access control with the new access control list carried by the object placement access control list request A list (eg, the ACL for the target object).

In step S64C, the index object-based simple storage service migration management module 53 can control the storage server 10 to proxy (eg forward) the error response corresponding to the object placement ACL request to the user.

For better understanding, the method may be described using the workflow shown in FIG. 13, but the invention is not limited thereto. According to some embodiments, one or more steps in the workflow shown in FIG. 13 may be is added, deleted, or changed, for example, at least a part (eg, part or all) of the workflow may be performed multiple times, and in particular any step that includes a check operation may be performed multiple times to generate multiple Check the results to trigger the execution of some subsequent steps, respectively.

FIG. 14 is a schematic diagram of an object deletion control scheme of a method for using index objects to perform seamless migration of simple storage services according to an embodiment of the present invention. As shown in FIG. 14, it is used to process an object during user data migration. A workflow for deleting requests for or multiple objects may include steps S70-S75, wherein step S74 may include some sub-steps (eg, S74A-S74D).

In step S70, the storage server 10 (eg, the simple storage service migration management module 53 based on indexed objects) may receive an object deletion request in one or more object deletion requests from the user, for example, the object deletion request request Deletes an object in the storage of the destination, and this object can be regarded as a target object of the object deletion request.

In step S71 , in response to the object deletion request, the simple storage service migration management module 53 based on the index object may check for an object corresponding to the object deletion request in the at least one storage in the storage device layer of the storage server 10 . Whether the storage (eg, the storage of the destination) is being processed in the index phase (eg, the INDEX phase), if so, go to step S72, if not, proceed to step S74 (eg, step S74A).

In step S72, the simple storage service migration management module 53 based on the index object can control the storage server 10 to proxy (for example, forward) the request (for example, an object deletion request) to the source end (for example, a remote easy storage service compatible server).

In step S73, the index object-based simple storage service migration management module 53 can check whether the request (eg, the object deletion request) is successful without any error, and if so (eg, a response received from the source, such as corresponding to the object) The response to the deletion request, indicating that no errors have occurred), go to step S74 (eg, step S74A); if no (eg, the response received from the source, such as the response corresponding to the object deletion request, indicates one or more errors) an error response), then go to step S75.

In step S74, the index object-based simple storage service migration management module 53 can control the storage server 10 to perform at least one normal action according to the object deletion request, for example, the index object-based simple storage service migration management module 53 The storage server 10 can be controlled to selectively delete the object (eg, the target object) based on at least a related ACL check whether it passes or not. Also, for example, the simple storage service migration management module 53 based on the index object can control the storage The server 10 selectively deletes the object (eg, the target object) according to at least whether the object exists. Preferably, the simple storage service migration management module 53 based on the index object can control the storage server 10 to A condition is satisfied to selectively delete the object (eg, the target object), wherein the first condition indicates that the ACL check is passed, and the second condition indicates that the object exists.

In step S74A, the index object-based simple storage service migration management module 53 can check whether the ACL check is passed, if yes, go to step S74B; if not, go to step S74D, for example, when the ACL check is passed When indicating that the user has the authority to apply the object deletion request to the object (for example, the authority to delete the object), the simple storage service migration management module 53 based on the index object can determine that the ACL check is passed, and for example, When the access control list indicates that the user does not have the right to apply the object deletion request to the object (eg, the right to delete the object), the index object-based simple storage service migration management module 53 can determine the access control list Check failed.

In step S74B, the simple storage service migration management module 53 based on the indexed object can check whether the object (for example, the target object of the object deletion request) exists, if yes, go to step S74C; if not, go to step S74D.

In step S74C, the simple storage service migration management module 53 based on the index object can control the storage server 10 to delete the object (eg, the target object).

In step S74D, the indexed object-based simple storage service migration management module 53 can control the storage server 10 to prevent the object (eg, the target object) from being deleted.

In step S75, the index object-based simple storage service migration management module 53 can control the storage server 10 to proxy (eg forward) the error response corresponding to the object deletion request to the user.

For better understanding, the method may be described using the workflow shown in FIG. 14, but the invention is not limited thereto. According to some embodiments, one or more steps in the workflow shown in FIG. 14 may be is added, deleted, or changed, for example, at least a part (eg, part or all) of the workflow may be performed multiple times, and especially any step that includes a check operation may be performed multiple times, to generate multiple Check the results to trigger the execution of some subsequent steps, respectively. In addition, the combination of steps S74A and S74B can be used to check whether the first condition and the second condition are both satisfied, and the conditions checked by steps S74A and S74B respectively can be exchanged, for example, a simple storage service migration management model based on index objects Group 53 may check whether the second condition is satisfied in step S74A, and check whether the first condition is satisfied in step S74B, if both the first condition and the second condition are satisfied (eg, the access If the control list is checked and the object exists), go to step S74C; otherwise, go to step S74D. Table I indexing stage data stage GET object Proxy if the object does not exist or has a "source modified time" Proxy if object has "source modified time" GET object ACL; HEAD object Proxy if object does not exist allow PUT object access control list proxy, and write to local allow DELETE object proxy, and deletes the local object (if it exists) allow list objects acting allow upload object proxy, and write to local allow

Table 1 depicts some examples of simple behaviors of reliable automated distributed object storage gateway actions for storage server 10 in an indexing phase (eg, INDEX phase) and a data phase (eg, DATA phase), where the requirements, such as GET Object, GET Object ACL, HEAD Object, PUT Object ACL, DELETE Object, List Object, and Upload Object (listed in the left half of Table 1), representing one or more objects, respectively get request, one or more object get ACL request, one or more object header request, one or more object put ACL request, one or more object delete request, one or more object list Requests, and one or more requests related to object data writing (eg, one or more requests to place objects, one or more requests to create a multi-component upload, and one or more requests to complete a multi-component upload). Under the control of the simple storage service migration management module 53 based on the index object, the storage server 10 can operate correctly and efficiently. For better understanding, some operations described in the above embodiments can be briefly summarized in the table One of the " Index Phase " and " Data Phase " fields.

For example, during the indexing phase, when any other request represents an object acquisition request, if the target object does not exist or has a last modification time (eg, source modification time), the storage server 10 may proxy this request to the source terminal; when any other request represents the object to obtain either the ACL request and the object header request, if the target object does not exist, the storage server 10 can proxy this request to the source end; when any other request When placing an ACL request on behalf of an object, the storage server 10 can proxy the request to the source, which writes the object ACL carried by the request to a local location (eg, a destination, such as a storage server) 10); when any other request represents an object deletion request, the storage server 10 can proxy this request to the source, and if the target object exists, delete the local object (such as the target object); when any other request represents When an object listing request is made, the storage server 10 can proxy this request to the source; and when any other request is written on behalf of the object data to a related request (such as an object placement request, creating a multi-part upload request, or completing a multi-part upload request) ), the storage server 10 can proxy the request to the source, and write the object data carried by the request to a local location (eg, the destination, such as the storage server 10 ). In the data phase, when any other request represents an object acquisition request, if the target object has a last modification time (eg, source modification time), the storage server 10 may proxy the request; and when any other request represents a request in all When any of the remaining requests is made, the storage server 10 may allow the related processing of the request without proxying the request to the source.

Because the indexing phase only takes a short time, and because the simple storage service migration management module 53 based on the index object can properly control the storage server 10 during the user migration by means of various control schemes of the method, the storage The server 10 can simply perform a simple storage service seamless migration to a private cloud (eg, a simple storage service seamless migration from any public/private cloud to a private cloud) to allow users to normally perform data migration during data migration. and smooth access to data, the storage server 10, under the control of the index object-based simple storage service migration management module 53, can correctly migrate user data while properly handling any other requests (if any), so No erroneous data will be generated (eg due to incorrect migration management of the relevant technology, usually when the user changes (eg writes, overwrites, deletes, etc.) something during the migration Discovery), thereby making the migration of users seamless, and for the sake of brevity, the similar content of this embodiment will not be repeated in detail here. The above descriptions are only preferred embodiments of the present invention, and all equivalent changes and modifications made according to the scope of the patent application of the present invention shall fall within the scope of the present invention.

50: Main unit 52: Processing circuit 52P: Program Module 53: Simple storage service migration management module based on index objects 54: Storage interface circuit 56: Storage Device 58: Network interface circuit 90: Multiple storage devices 100: storage device 110: Memory Controller 112: Microprocessor 112C: Code 112M: read-only memory 114: Control logic circuit 116: Random Access Memory 118: Transmission interface circuit 120: non-volatile memory 122-1~122-N: Non-volatile memory components T1~T3: Time point S01~S08, S10~S14, S20~S29, S30~S36, S40~S48, S50~58, S60~S65, S70~S75, S14A, S14B, S14C, S25A, S25B, S25C, S27A,S S27B, S27C , S29E, S36A, S36B, S36C, S45A, S45B, S45C, S56A, S56B, S56C, S64A, S64B, S64C, S74A, S74B, S74C: Steps

FIG. 1 is a schematic diagram of a storage server according to an embodiment of the present invention. FIG. 2 is a schematic diagram of some implementation details of the storage device in the storage server shown in FIG. 1 according to an embodiment of the present invention. FIG. 3 is an index object-based migration control scheme of a method for using index objects to perform seamless migration of simple storage services according to an embodiment of the present invention. FIG. 4 is a schematic diagram of some related operations of the index object-based migration control scheme shown in FIG. 3 according to an embodiment of the present invention. FIG. 5 is a schematic diagram of some objects and some related features of the index object-based migration control scheme shown in FIG. 3 according to an embodiment of the present invention. FIG. 6 is a schematic diagram of a migration agent control scheme of a method for performing seamless migration of simple storage services by using index objects according to an embodiment of the present invention. FIG. 7 is a schematic diagram of an example of a Ceph object storage architecture with an index object-based simple storage service migration management module shown in FIG. 1 . FIG. 8 is a schematic diagram of an object listing control scheme of a method for using index objects to perform seamless migration of simple storage services according to an embodiment of the present invention. FIG. 9 is a schematic diagram of an object header control scheme of a method for using index objects to perform seamless migration of simple storage services according to an embodiment of the present invention. FIG. 10 is a schematic diagram of an object acquisition control scheme of a method for using index objects to perform seamless migration of simple storage services according to an embodiment of the present invention. FIG. 11 is a schematic diagram of an object acquisition ACL control scheme of a method for using index objects to perform seamless migration of simple storage services according to an embodiment of the present invention. FIG. 12 is a schematic diagram of an object placement control scheme of a method for using index objects to perform seamless migration of simple storage services according to an embodiment of the present invention. FIG. 13 is a schematic diagram of an object placement ACL control scheme of a method for using index objects to perform seamless migration of simple storage services according to an embodiment of the present invention. FIG. 14 is a schematic diagram of an object deletion control scheme of a method for using index objects to perform seamless migration of simple storage services according to an embodiment of the present invention.

T1~T3: Time point

Claims (18)

A method for seamless migration of simple storage services using index objects, the method is applied to a storage server, and the method includes: In response to a request to migrate a user data of a user of the storage server to the storage server from a remote simple storage service compatible server, in one of multiple stages of a migration of the user data During the indexing phase, a simple storage service migration management module based on indexing objects among a plurality of program modules running on a host device in the storage server is used to create and store a plurality of indexing objects to the storage server a storage device layer in the storage server to serve as the respective representatives of the plurality of normal objects of the user data in the storage server, and to migrate the respective ACLs of the plurality of normal objects to the storage server as the the respective ACLs of the plurality of index objects to minimize the probability that any change of any one of the plurality of normal objects affects the correctness of the migration of the user data, wherein the storage server includes There is the host device and the storage device layer, the storage device layer includes at least one storage device coupled to the host device, the host device is used to control the operation of the storage server, and the at least one storage device is used for to store information for the storage server; and In a data phase following the indexing phase of the plurality of phases, utilizing the index object-based easy storage service migration management module to trigger one or more migration agents to be compatible from the remote easy storage service The server migrates the respective object data of a set of normal objects in the plurality of normal objects to the storage device layer of the storage server as respective replacements of a set of index objects in the plurality of index objects to Complete the migration of the user data. The method described in Item 1 of the scope of the patent application further includes: In response to the request to migrate the user data from the remote simple storage service compliant server to the storage server, in a setup phase before the indexing phase of the plurality of phases, the index object-based simple The storage service migration management module performs preprocessing of the migration of the user data. 2. The method of claim 2, wherein the preprocessing comprises switching at least one endpoint for accessing the user data from at least one cluster of remote easy storage service compatible servers to the storage server at least one cluster of devices. The method of claim 1, wherein the respective ACLs of the plurality of normal objects are migrated to the storage server as the respective ACLs of the plurality of index objects, as if The plurality of index objects are the plurality of normal objects, so as to minimize the probability that any change of any one of the plurality of normal objects affects the accuracy of the migration of the user data. The method of claim 1, wherein a workflow for processing one or more object list requests during the migration of the user data comprises: receiving from the user an object list request in the one or more object list requests; In response to the object list request, checking whether a store corresponding to the object list request in at least one store in the storage device layer is being processed in the indexing phase, wherein the at least one store represents at least one of the objects a container; and Since the store is being processed in the indexing phase, the storage server is controlled to proxy the object list request to the remote simple storage service compatible server. The method of claim 5, wherein the workflow for processing the one or more object list requests during the migration of the user data further comprises: After controlling the storage server to proxy the object list request to the remote simple storage service compatible server, controlling the storage server to send a response corresponding to the object list from the remote simple storage service server The container server proxies to this user. The method of claim 5, wherein the step of checking whether the storage is being processed in the indexing phase is performed multiple times to generate a first check result and a second check result, respectively, wherein the first check result is a check result indicates that the store is being processed during the indexing phase, and the second check result indicates that the store is not being processed during the indexing phase; controlling the storage server to proxy the object list request to the The steps of the remote simple storage service compatible server are performed in response to the first check result; and the workflow for processing the one or more object list requests during the migration of the user data further includes Have: In response to the second check result, the storage server is controlled to perform at least one normal action according to the object list request without proxying the object list request to the remote simple storage service compatible server. The method of claim 7, wherein the at least one normal action includes: Controlling the storage server to selectively return a local object list result to the user according to whether an associated ACL check passes, wherein if the associated ACL check passes, the storage The server returns the local object list result to the user, otherwise, the storage server prevents the local object list result from being returned to the user. The method of claim 1, wherein a workflow for processing one or more object header requests during the migration of the user data comprises: receiving an object header request from the one or more object header requests from the user; In response to the object header request, checking whether a reservoir corresponding to the object header request is being processed in the indexing phase of at least one reservoir in the storage device layer, wherein the at least one reservoir represents at least one of the object headers. a container; checking for the existence of a target object requested by the object header as the store is being processed during the indexing phase; checking whether the target object is still an index object due to the existence of the target object; and Since the target object is still the index object, check whether an object access control list of the target object has been migrated to the storage server, so as to selectively perform at least one predetermined action according to the object header request without requiring the Object header request proxy to this Remote Simple Storage Service compatible server. The method of claim 9, wherein the at least one predetermined action includes: controlling the storage server to selectively return to the user a plurality of attributes recorded in a plurality of temporary attributes of the indexed object according to an associated access control list check, wherein if the associated access control list is passed If the ACL check is passed, the storage server returns the plurality of attributes recorded in the plurality of temporary attributes of the index object to the user; otherwise, the storage server blocks the The plurality of attributes recorded in the plurality of temporary attributes are returned to the user. The method of claim 1, wherein a workflow for processing one or more object acquisition requests during the migration of the user data comprises: receiving an object acquisition request from the one or more object acquisition requests from the user; In response to the object acquisition request, checking whether a bank corresponding to the object acquisition request is being processed in the indexing phase in at least one of the storage device layers, wherein the at least one bank represents at least one of the object's a container; checking for the existence of a target object of the object acquisition request as the store is being processed during the indexing phase; checking whether the target object is still an index object due to the existence of the target object; and Since the target object is still the index object, the storage server is controlled to perform at least one normal action according to the object acquisition request, without proxying the object acquisition request to the remote simple storage service compatible server. The method of claim 11, wherein the at least one normal action includes: controlling the storage server to selectively return local data of the target object to the user according to whether an associated ACL check passes, wherein if the associated ACL check passes, the The storage server returns the local data to the user, otherwise the storage server prevents the local data from being passed back to the user, where the local data represents the object data of the target object in the storage server . The method of claim 1, wherein a workflow for processing one or more object access control list requests during the migration of the user data comprises: receiving an object get ACL request from the user in the one or more object get ACL requests; In response to the object get ACL request, checking whether a memory corresponding to the object get ACL request is being processed in the indexing phase in at least one of the banks in the storage layer, wherein the at least one a receptacle represents at least one container for the article; checking for the existence of a target object requested by the object to obtain an ACL because the store is being processed during the indexing phase; checking whether the target object is still an index object due to the existence of the target object; and Since the target object is still the index object, checking whether an object access control list of the target object has been migrated to the storage server, so as to selectively perform at least one normal action according to the request for obtaining the access control list of the object, There is no need for the object to obtain an access control list request proxy to the remote simple storage service compliant server. The method of claim 13, wherein the at least one normal action includes: Controlling the storage server to selectively return a local ACL for the target object to the user based on an associated ACL check, wherein if the associated ACL If the check is passed, the storage server returns the local access control list to the user; otherwise, the storage server prevents the local access control list from being returned to the user, wherein the local access control list represents An access control list for the target object in the storage server. The method of claim 1, wherein the storage server is implemented as a private cloud, and the remote simple storage service compatible server is implemented as a public cloud or another private cloud. A main device comprising: a processing circuit for controlling the host device to perform simple storage service migration management based on index objects in a storage server, wherein the storage server includes the host device and a storage device layer, the storage device layer includes a at least one storage device coupled to the host device, the host device is used to control the operation of the storage server, and the at least one storage device is used to store information for the storage server, wherein: In response to a request to migrate a user data of a user of the storage server to the storage server from a remote simple storage service compatible server, in one of multiple stages of a migration of the user data In the indexing phase, a simple storage service migration management module based on indexing objects among a plurality of program modules running on the processing circuit creates and stores a plurality of indexing objects to the storage device layer as the storage server the respective representatives of the plurality of normal objects in the user data, and migrate the respective access control lists of the plurality of normal objects to the storage server as the respective access controls of the plurality of index objects a list to minimize the chance that any change in any one of the plurality of normal objects affects the correctness of the migration of the user data; and In a data phase subsequent to the indexing phase of the plurality of phases, the index object-based easy storage service migration management module triggers one or more migration agents from the remote easy storage service compatible server Migrating the respective object data of a set of normal objects in the plurality of normal objects to the storage device layer of the storage server as respective replacements of a set of index objects in the plurality of index objects to complete the This migration of user data. The main device as described in item 16 of the claimed scope further includes: A casing is used to install a plurality of components of the main device and the at least one storage device, wherein the plurality of components of the main device include the processing circuit. A storage server, including: A main device for controlling the operation of the storage server, the main device includes: a processing circuit for controlling the host device for easy storage service migration management based on index objects in the storage server; and A storage device layer, including: at least one storage device, coupled to the host device, for storing information for the storage server; in: In response to a request to migrate a user data of a user of the storage server to the storage server from a remote simple storage service compatible server, in one of multiple stages of a migration of the user data In the indexing phase, a simple storage service migration management module based on indexing objects among a plurality of program modules running on the processing circuit creates and stores a plurality of indexing objects to the storage device layer as the storage server the respective representatives of the plurality of normal objects in the user data, and migrate the respective access control lists of the plurality of normal objects to the storage server as the respective access controls of the plurality of index objects a list to minimize the chance that any change in any one of the plurality of normal objects affects the correctness of the migration of the user data; and In a data phase subsequent to the indexing phase of the plurality of phases, the index object-based easy storage service migration management module triggers one or more migration agents from the remote easy storage service compatible server Migrating the respective object data of a set of normal objects in the plurality of normal objects to the storage device layer of the storage server as respective replacements of a set of index objects in the plurality of index objects to complete the This migration of user data.

Images