CN113568567A - Method, host device and storage server for seamless migration of simple storage service by using index object - Google Patents
Method, host device and storage server for seamless migration of simple storage service by using index object Download PDFInfo
- Publication number
- CN113568567A CN113568567A CN202110360172.XA CN202110360172A CN113568567A CN 113568567 A CN113568567 A CN 113568567A CN 202110360172 A CN202110360172 A CN 202110360172A CN 113568567 A CN113568567 A CN 113568567A
- Authority
- CN
- China
- Prior art keywords
- index
- storage server
- storage
- migration
- access control
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 230000005012 migration Effects 0.000 title claims abstract description 259
- 238000013508 migration Methods 0.000 title claims abstract description 259
- 238000000034 method Methods 0.000 title claims abstract description 75
- 230000004044 response Effects 0.000 claims abstract description 77
- 238000012545 processing Methods 0.000 claims description 41
- 230000009471 action Effects 0.000 claims description 15
- 230000008859 change Effects 0.000 claims description 11
- 238000007781 pre-processing Methods 0.000 claims description 4
- 239000003795 chemical substances by application Substances 0.000 description 37
- 238000010586 diagram Methods 0.000 description 26
- 238000012217 deletion Methods 0.000 description 16
- 230000037430 deletion Effects 0.000 description 16
- 230000004048 modification Effects 0.000 description 13
- 238000012986 modification Methods 0.000 description 13
- 102220539807 Centromere protein U_S74A_mutation Human genes 0.000 description 7
- 238000012546 transfer Methods 0.000 description 7
- 102220474389 Retinoic acid receptor RXR-alpha_S27A_mutation Human genes 0.000 description 5
- 102220502165 TP53-binding protein 1_S25A_mutation Human genes 0.000 description 5
- 102220604668 Transcription factor Sp1_S56A_mutation Human genes 0.000 description 5
- 102220198042 rs121913407 Human genes 0.000 description 5
- 102220539817 Centromere protein U_S74D_mutation Human genes 0.000 description 4
- 102220475064 HSPB1-associated protein 1_S64A_mutation Human genes 0.000 description 4
- 241000932075 Priacanthus hamrur Species 0.000 description 4
- 102220275633 rs200942544 Human genes 0.000 description 4
- 102220471474 Protein MON2 homolog_S36A_mutation Human genes 0.000 description 3
- 102220471486 Protein MON2 homolog_S36C_mutation Human genes 0.000 description 3
- 102220501418 Putative uncharacterized protein LINC00574_S27C_mutation Human genes 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 3
- 238000004891 communication Methods 0.000 description 3
- 230000005055 memory storage Effects 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 102200082816 rs34868397 Human genes 0.000 description 3
- 102220101274 rs777100532 Human genes 0.000 description 3
- 102220342298 rs777367316 Human genes 0.000 description 3
- 239000007787 solid Substances 0.000 description 3
- 102220502164 TP53-binding protein 1_S29A_mutation Human genes 0.000 description 2
- 230000008878 coupling Effects 0.000 description 2
- 238000010168 coupling process Methods 0.000 description 2
- 238000005859 coupling reaction Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000002123 temporal effect Effects 0.000 description 2
- 230000007704 transition Effects 0.000 description 2
- 101001115218 Homo sapiens Ubiquitin-40S ribosomal protein S27a Proteins 0.000 description 1
- 235000005135 Micromeria juliana Nutrition 0.000 description 1
- 241000246354 Satureja Species 0.000 description 1
- 235000007315 Satureja hortensis Nutrition 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 230000015556 catabolic process Effects 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 238000006731 degradation reaction Methods 0.000 description 1
- 239000004744 fabric Substances 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 239000002184 metal Substances 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/067—Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0604—Improving or facilitating administration, e.g. storage management
- G06F3/0607—Improving or facilitating administration, e.g. storage management by facilitating the process of upgrading existing storage systems, e.g. for improving compatibility between host and storage device
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0646—Horizontal data movement in storage systems, i.e. moving data in between storage devices or systems
- G06F3/0647—Migration mechanisms
Abstract
The invention discloses a method and related equipment for seamless migration of simple storage service by using index objects, wherein the method comprises the following steps: in response to a request for migrating user data of a user of a storage server from a remote SMSC to the storage server, creating and storing index objects in a storage device layer of the storage server by using an index object-based SMSC migration management module among a plurality of program modules running on a host device in the storage server in an indexing stage, and migrating respective access control lists of a plurality of normal objects to the storage server; and in the data phase, triggering one or more migration agents to migrate the object data by using the index object-based SMM.
Description
Technical Field
The present invention relates to memory control, and more particularly, to a method and apparatus (apparatus) for seamless migration of simple storage service (S3) using index objects, wherein examples of the apparatus may include: the storage server as a whole, the host device within the storage server, the processing circuitry within the host device, and at least one processor/processor core (e.g., central processing unit/CPU core) running one or more program modules in the processing circuitry corresponding to the method, although the invention is not limited thereto.
Background
The server may be used for cloud computing (e.g., a storage server may be used to implement remote storage (e.g., a cloud capable of storing data for a user). When a user using an old public cloud (public cloud) finds that the public cloud is almost full or has a slow access speed, the user may need to search for a new cloud, and a data migration method has been proposed in the prior art to attempt to copy data from the old public cloud to another public cloud, however, problems may occur, for example, the data migration method typically copies data including all data before a start time point in the old public cloud, then performs an endpoint switch, and the like, and thus may erroneously copy data that should be deleted (for example, data that has been deleted by the user after the start time point) to another public cloud. Furthermore, according to the data migration method, it may take a long time (for example, several months) to copy data first, and particularly, this data migration method is not suitable for a private cloud (private cloud), and if a user wishes to use the private cloud as a new cloud, it seems that a suitable solution for migrating data from an old cloud to the private cloud cannot be found in the related art, and therefore, a novel architecture is required for seamless migration of an easy storage service for the private cloud (for example, seamless migration of an easy storage service from any one public/private cloud to a private cloud) to allow the user to normally and smoothly access data during data migration.
Disclosure of Invention
Therefore, an object of the present invention is to disclose a method for seamless migration of a simple storage service by using index objects, and disclose related devices (e.g. a storage server, a master device in the storage server, etc.) to solve the above problems.
It is another object of the present invention to disclose a method for seamless migration of a thin provisioning service using index objects and related devices (e.g., provisioning servers, master devices in provisioning servers, etc.) to achieve optimal performance without introducing side effects or by a manner that is less likely to introduce side effects.
At least one embodiment of the present invention discloses a method for seamless migration of a thin provisioning service using index objects, wherein the method is applied to a provisioning server, and the method comprises: in response to a request for migrating user data of a user of the storage server from a remote simple storage service consistent (S3-compatible) server to the storage server, in an index stage (index stage) of a plurality of stages of migration of the user data, creating and storing index objects to a storage tier of the storage server by using an index-object-based simple storage service migration management module of a plurality of program modules running on a host device in the storage server as respective representatives of normal objects (normal objects) of the user data in the storage server, and migrating respective Access Control Lists (ACLs) of the normal objects to the storage server as respective access control lists of the index objects, to minimize a probability that any change of any of the plurality of normal objects affects the correctness of the migration of the user data, wherein the storage server comprises the host device and the storage device layer, the storage device layer comprises at least one storage device coupled to the host device, the host device is used to control the operation of the storage server, and the at least one storage device is used to store information for the storage server; and in a data phase (data phase) following the index phase of the plurality of phases, triggering one or more migration agents (migration agents) by the index-object-based SMPS migration management module to migrate respective object data of a set of normal objects of the plurality of normal objects from the remote SMPS to the storage tier of the storage server as respective replacements for a set of index objects of the plurality of index objects to complete the migration of the user data.
In addition to the above method, the present invention also discloses a host device, which may include a processing circuit for controlling the host device to perform index object-based easy storage service migration management in a storage server, wherein the storage server includes the host device and a storage device layer, the storage device layer includes at least one storage device coupled to the host device, the host device is used to control the operation of the storage server, and the at least one storage device is used to store information for the storage server, for example, in response to a request from a remote easy storage service compatible server to migrate user data of a user of the storage server to the storage server, in an index phase of a plurality of phases of migration of the user data, an index object-based easy storage service migration management module of a plurality of program modules running on the processing circuit creates and stores a plurality of index primers Loading into the storage device layer as a representation of each of a plurality of normal objects of the user data in the storage server, and migrating the access control list of each of the plurality of normal objects to the storage server as an access control list of each of the plurality of index objects to minimize a probability that any change in any of the plurality of normal objects affects the correctness of the migration of the user data; and in a data phase following the index phase of the plurality of phases, the index-object-based SMPS migration management module triggers one or more migration agents to migrate respective object data of a set of normal objects of the plurality of normal objects from the remote SMPS compliant server to the storage tier of the storage server as respective replacements for a set of index objects of the plurality of index objects to complete the migration of the user data.
In addition to the above method, the present invention also discloses a storage server, which may include a master device and a storage device layer, wherein the master device is used to control the operation of the storage server, for example, the master device may include a processing circuit for controlling the master device to perform index object-based easy storage service migration management in the storage server. In addition, the storage device layer may include at least one storage device for storing information for the storage server, for example, in response to a request to migrate user data of a user of the storage server from a remote SMPS server to the storage server, in an index phase of a plurality of phases of a migration of the user data, an index-object-based SMPS migration management module of a plurality of program modules running on the processing circuitry creates and stores a plurality of index objects to the storage device layer as respective representatives of a plurality of normal objects of the user data in the storage server, and migrates respective access control lists of the plurality of normal objects to the storage server as respective access control lists of the plurality of index objects to minimize any change in any of the plurality of normal objects from affecting the correctness of the migration of the user data The probability of sex; and in a data phase following the index phase of the plurality of phases, the index-object-based SMPS migration management module triggers one or more migration agents to migrate respective object data of a set of normal objects of the plurality of normal objects from the remote SMPS compliant server to the storage tier of the storage server as respective replacements for a set of index objects of the plurality of index objects to complete the migration of the user data.
The method and related apparatus of the present invention can utilize index artifacts for seamless migration of easy storage services, and after an index phase/period (e.g., a short period of time (period)), the method and related apparatus of the present invention can seamlessly migrate in a data phase/period with less overhead (overhead), wherein the overhead of the data phase/period is typically less than the overhead of the index phase/period.
Drawings
In order to make the aforementioned objects, features and advantages of the present invention comprehensible, embodiments accompanied with figures are described in detail below, wherein:
fig. 1 is a schematic diagram of a storage server according to an embodiment of the present invention.
FIG. 2 is a diagram illustrating some implementation details of a storage device in the storage server shown in FIG. 1 according to an embodiment of the invention.
FIG. 3 is a migration control scheme based on index artifacts for a method for seamless migration of a thin provisioning service using index artifacts according to an embodiment of the present invention.
FIG. 4 is a diagram illustrating some relevant operations of the migration control scheme based on index objects shown in FIG. 3 according to an embodiment of the present invention.
FIG. 5 is a diagram of some objects and some associated features with the index object based migration control scheme shown in FIG. 3, according to an embodiment of the present invention.
FIG. 6 is a diagram illustrating a migration agent control scheme for a method for seamless migration of a thin provisioning service using index artifacts according to an embodiment of the present invention.
FIG. 7 is a diagram illustrating an example of the Ceph object storage architecture with the index object-based SMPM migration management module shown in FIG. 1.
FIG. 8 is a diagram illustrating an object listing control scheme for a method for seamless migration of a thin provisioning service using indexed objects, according to an embodiment of the present invention.
FIG. 9 is a diagram illustrating an object header control scheme of a method for seamless migration of SMPS using indexed objects according to an embodiment of the present invention.
FIG. 10 is a diagram illustrating an object acquisition control scheme for a method for seamless migration of SMPS using indexed objects according to an embodiment of the present invention.
FIG. 11 is a diagram illustrating an object acquisition access control list control scheme for a method for seamless migration of SMSCs using index objects according to an embodiment of the present invention.
FIG. 12 is a diagram illustrating an object placement control scheme for a method for seamless migration of a thin provisioning service using indexed objects, according to an embodiment of the present invention.
FIG. 13 is a diagram illustrating an object placement access control list control scheme for a method for seamless migration of SMSC using indexed objects according to an embodiment of the invention.
FIG. 14 is a diagram illustrating an object deletion control scheme for a method for seamless migration of a thin storage service using indexed objects according to an embodiment of the present invention.
Element numbering in the figures:
50: master device
52: processing circuit
52P: program module
53: simple storage service migration management module based on index object
54: memory interface circuit
56: storage device
58: network interface circuit
90: multiple storage devices
100: storage device
110: memory controller
112: microprocessor
112C: program code
112M: read-only memory
114: control logic circuit
116: random access memory
118: transmission interface circuit
120: non-volatile memory
122-1 to 122-N: non-volatile memory element
T1-T3: point in time
S01 to S08, S10 to S14, S20 to S29, S30 to S36, S40 to S48, S50 to 58, S60 to S65, S70 to S75, S14A, S14B, S14C, S25A, S25B, S25C, S27A, S S27B, S27C, S29E, S36A, S36B, S36C, S45A, S45B, S45C, S56A, S56B, S56C, S64A, S64B, S64C, S74A, S74B, S74C: step (ii) of
Detailed Description
Embodiments of the present invention disclose a method and apparatus for seamless migration of a thin provisioning service using index objects, for example, the method and related apparatus can use a novel seamless migration architecture for the thin provisioning service to perform access control, and more particularly, to perform seamless migration of a thin provisioning service for a private cloud, for example, seamless migration of a thin provisioning service from any public/private cloud to a private cloud (e.g., destination cloud for data migration), so as to allow a user to access data normally and smoothly during the data migration process. For a better understanding, a first cloud, such as a trivial storage service compliant storage (e.g., it may be nearly full and access speed may be slow), and a second cloud, such as Bigtera storage implemented according to the method, may be examples of the either public/private cloud and the private cloud (e.g., target cloud), respectively, although the invention is not limited thereto. In addition, when the migration is started at the first time point T1, the method and related apparatus may create an INDEX artifact (e.g., a virtual INDEX object) in the second cloud before an INDEX phase, e.g., an INDEX phase (e.g., an INDEX period such as an INDEX period), started at the first time point T1, and may migrate (e.g., transfer) artifact DATA from the first cloud to the second cloud after a DATA phase, e.g., a DATA phase (e.g., a DATA period such as a DATA period), started at the second time point T2 (e.g., a time point when creation of the INDEX artifact is completed). In some embodiments, the storage services seamless migration architecture may utilize one or more migration agents (agents) for object migration, and further, the apparatus may comprise at least a portion (e.g., a portion or all) of a storage server, e.g., the apparatus may comprise a portion of a storage server, such as a master device in a storage server or associated control circuitry located within a master device (e.g., processing circuitry and associated circuitry running on one or more program modules corresponding to the method), in another example, the apparatus may comprise the entire storage server, according to some embodiments for a storage server, the INDEX period may represent a short time period (e.g., hours or days), and the DATA period may represent a long time period (e.g., weeks).
Fig. 1 is a diagram illustrating a storage server 10 according to an embodiment of the invention, in which the storage server 10 includes a host device 50 and at least one storage device, such as one or more storage devices (e.g., a plurality of storage devices 90), the plurality of storage devices 90 are coupled to the host device 50, according to the embodiment, the host device 50 may be used to control the operation of the storage server 10, and the plurality of storage devices 90 may be used to store information for the storage server 10. As shown in fig. 1, host device 50 may include processing circuitry 52 (e.g., at least one processor/processor core and associated circuitry such as Random Access Memory (RAM), bus, etc.) for controlling operations of host device 50, at least one storage interface circuit 54 for coupling to a plurality of storage devices 90 and to storage or memory devices located on host device 50 (e.g., one or more conventional Hard Disk Drives (HDDs) and/or one or more Solid State Drives (SSDs)), and a network interface circuit 58 for coupling host device 50 to at least one network. The storage or memory device may comprise at least one storage device (e.g., one or more storage devices), which may be collectively referred to as storage device 56, e.g., storage device 56 may comprise a set of storage devices, wherein one storage device of the set of storage devices may be used as a system disk for host device 50 and the other storage devices of the set of storage devices may be used to store user data for host device 50, although the invention is not limited thereto, in another example storage device 56 may comprise one storage device and this storage device may be used as a system disk for host device 50.
According to the embodiment, the processing circuit 52 (especially, the index-based trivial storage service migration management module 53 corresponding to the method) running the program module 52P may control the operation of the host device 50 according to the method, for example, control the host device 50 to perform index-based trivial storage service migration management in the storage server 10, and the storage interface circuit 54 may conform to one or more standards (e.g., one or more Serial Advanced Technology Attachment (Serial ATA or SATA) standards, external Component Interconnect (PCI) standard, Peripheral Component Interconnect Express (PCIe) standard, Non-Volatile storage Memory Express (NVMe) standard, Non-Volatile storage external access (NVMe-over-Fabrics, nvef) standard, Small Computer System Interface (SCSI) standards, Universal Flash Storage (UFS) standards, etc.), and may communicate according to one or more standards to allow processing circuitry 52 running program module 52P to access Storage device 56 and Storage devices 90 via Storage Interface circuitry 54, and network Interface circuitry 58 may be used to provide wired or wireless network connectivity, and one or more client devices corresponding to one or more users may access (e.g., read or write) user data in Storage server 10 (e.g., Storage device 56 and Storage devices 90 therein) via the wired or wireless network connectivity. For better understanding, the main device 50 shown in the left half of fig. 1 and the related circuits/modules/devices (e.g., the processing circuit 52 and the storage interface circuit 54 running the program module 52P) in the architecture can be taken as examples of the main device and the related circuits/modules/devices (e.g., the processing circuit and the related circuits corresponding to the method running one or more program modules) therein, respectively.
In the architecture shown in fig. 1, the storage server 10 may be illustrated as including a host device 50 and a plurality of storage devices 90 coupled to the host device 50, but the present invention is not limited thereto, for example, the host device 50 may further include a housing/casing (e.g., a computer casing, which may be made of metal and/or one or more other materials) for mounting components of the host device 50 (e.g., the processing circuit 52, the storage interface circuit 54, the network interface circuit 58, etc. shown in fig. 1) and at least a portion (e.g., a portion or all) of the plurality of storage devices 90, and for example, the storage server 10 may further include at least one switching circuit (e.g., one or more switching circuits) coupled between the host device 50 and at least a portion (e.g., a portion or all) of the plurality of storage devices 90 for switching signals (signal) between the host device 50 and at least a portion of the plurality of storage devices 90 switching).
According to some embodiments, processing circuitry 52 or storage interface circuitry 54 running program module 52P may configure at least a portion (e.g., a portion or All) of the plurality of storage devices 90 to form a storage pool architecture, although the invention is not limited in this respect and according to some embodiments, processing circuitry 52 or storage interface circuitry 54 running program module 52P may configure at least a portion (e.g., a portion or All) of the plurality of storage devices 90 to form a Redundant Array of Independent Disks (RAID) of storage server 10 (e.g., an All Flash Array, a)).
Fig. 2 is a diagram illustrating some implementation details of the storage device in the storage server 10 shown in fig. 1, wherein the storage device 100 may be used as an example of any one of the storage devices 90, and particularly, each of the storage devices 90 may be implemented according to the architecture of the storage device 100, but the invention is not limited thereto. The memory device 100 may be used to provide a memory space for the host device 50, in which one or more client devices may access (e.g., read or write) user data under the control of the host device 50, examples of the host device 50 may include but are not limited to: examples of the storage device 100 include, but are not limited to, a personal computer (e.g., a desktop computer or a notebook computer): solid state drives, and various types of embedded memory devices (e.g., conforming to the universal flash memory storage or Embedded Multimedia Memory Card (EMMC) standards, etc.). According to the present embodiment, the memory device 100 may comprise a controller (e.g., the memory controller 110) and further comprises a nonvolatile memory 120, wherein the controller is used for controlling the operation of the memory device 100 and accessing the nonvolatile memory 120, and the nonvolatile memory 120 is used for storing information, the nonvolatile memory 120 may comprise at least one nonvolatile memory element (e.g., one or more nonvolatile memory elements), such as a plurality of nonvolatile memory elements 120-1, 120-2, …, and 122-N, wherein "N" may represent a positive integer greater than one, for example, the nonvolatile memory 120 may be a flash memory, and the plurality of nonvolatile memory elements 120-1, 120-2, …, and 122-N may be a plurality of flash memory chips (chips) or a plurality of flash memory dies (die), the present invention is not limited thereto.
According to the present embodiment, the memory controller 110 may be configured to control the access of the non-volatile memory 120, so as to allow the host device 50 to access the non-volatile memory 120 through the memory controller 110 for access management in the storage server 10, as shown in fig. 2, the memory controller 110 may include a processing circuit (e.g., the microprocessor 112), a memory unit (e.g., a read-only memory (ROM) 112M), a control logic circuit 114, a random access memory 116, and a transmission interface circuit 118, which may be coupled to each other through a bus. The random access memory 116 is implemented by a Static Random Access Memory (SRAM), but the invention is not limited thereto, the random access memory 116 can be used to provide an internal storage space for the memory controller 110, for example, the random access memory 116 can be used as a buffer memory to buffer data, in addition, the rom 112M of the embodiment is used to store the program codes 112C, and the microprocessor 112 is used to execute the program codes 112C to control the operation of the memory controller 110 to control the access of the nonvolatile memory 120, so as to allow the host device 50 to access the nonvolatile memory 120 through the memory controller 110. It is noted that in some examples, the program code 112C may be stored in the ram 116 or any type of memory, and the control logic 114 may be used to control the non-volatile memory 120 and may include data protection circuitry (not shown) for protecting data and/or performing error correction, but the invention is not limited thereto, and the transmission interface circuit 118 may conform to a particular communication standard (e.g., serial advanced technology attachment (sata) standard, external component interconnect (pci) standard, fast external component interconnect (spi) standard, non-volatile memory storage external access standard, scsi standard, usb flash memory storage standard, etc.) and may communicate according to the particular communication standard, e.g., for the memory device 100 to communicate with the host device 50, wherein the memory interface circuit 54 may conform to the particular communication standard, to communicate with the storage device 100 for the host device 50.
According to some embodiments, the host device 50 may transmit a plurality of host commands and a plurality of corresponding logical addresses to the memory controller 110 to access the memory device 100, the memory controller 110 receives the host commands and the logical addresses, converts the host commands into a plurality of memory operation commands (which may be referred to as operation commands for brevity), and further controls the nonvolatile memory 120 via the operation commands to read, write/program, etc. on a memory cell (e.g., a data page) having a plurality of physical addresses within the nonvolatile memory 120, wherein the physical addresses may be associated with the logical addresses, when the memory controller 110 transmits a command to one of the nonvolatile memory elements 122-N ("N" may represent [1, n) interval, at least one block of a plurality of blocks of the non-volatile memory device 122-N, wherein each of the blocks may comprise a plurality of pages (e.g., data pages), may be erased, and an access operation (e.g., read or write) may be performed on one or more pages.
According to some embodiments, the storage server 10 (e.g., the host device 50) may operate according to a Ceph solution such that the storage server 10 is software defined as part of a distributed storage system, and optimally, multiple storage systems (e.g., multiple storage servers {10}, such as the storage server 10) may be used to form resource pools (resource pools) of the distributed storage system to support various types of access such as having certain features (features) (e.g., fault tolerance, automatic failover control, etc.), such as block device type (e.g., used in the distributed storage system to access an emulated block device), file system type (e.g., used in the distributed storage system to access a file system), and object type (e.g., used in the distributed storage system to access an object within an object namespace (object namespace)), the present invention is not limited thereto. For example, a file may be stored as an object in the object namespace in the distributed storage system, and for example, a data block corresponding to an address may be stored as an object in the object namespace in the distributed storage system, in order to perform features of the distributed storage system, the storage server 10 may be used to store additional information (e.g., metadata (metadata) and logs (journal)) in addition to data (e.g., user data).
FIG. 3 is a migration control scheme based on index artifacts for a method for seamless migration of a thin storage service using index artifacts, wherein the method is applicable to the storage server 10 according to an embodiment of the present invention. The storage server 10 may serve the role of the second cloud and may be considered as the destination (destination) of the data migration, and the remote scad compliant server may serve the role of the first cloud and may be considered as the source (source) of the data migration, for example, before the first time point T1, the user may access (e.g., read or write) data in the remote scad compliant server and find that his/her personal storage capacity in the remote scad compliant server is almost exhausted and may suffer from the performance degradation of the remote scad compliant server.
In response to a request for migrating user data of a user of the storage server 10 from the remote rdps to the storage server 10, during an INDEX phase (e.g., INDEX phase) of multiple phases of the migration of user data (e.g., migrating user data from the remote rdps to the storage server 10 according to the request of the user), the INDEX object-based rdps migration management module 53 in the program module 52P running on the processing circuit 52 may create and store multiple INDEX objects (e.g., virtual objects without object data) to a storage layer (storage device layer) of the storage server 10 as respective representatives of multiple normal objects (normal objects) of the user data in the storage server 10, and migrate respective Access Control lists (Access Control lists) of the multiple normal objects, ACL) to the storage server 10 as a respective access control list of a plurality of index objects as if the plurality of index objects were a plurality of normal objects to minimize the chance that any change in any of the plurality of normal objects affects the correctness of the migration of the user data, wherein the storage device layer comprises at least one storage device (e.g., one or more storage devices), such as the plurality of storage devices 90, coupled to the host device 50. By migrating the respective access control lists of the plurality of normal objects to the storage server 10 as the respective access control lists of the plurality of index objects, the index object-based SMM 53 may simulate (emulate) the plurality of normal objects using the plurality of index objects, for example, the destination (e.g., the storage server 10) may start migrating at a first time point T1, and after the first time point T1, when the user transmits any other request to the destination (e.g., the storage server 10) to attempt any change to any normal object, the storage server 10 may selectively proxy (forward) (e.g., forward) any other request to the source (e.g., the remote SMPS) and respond(s) corresponding to any other request based on whether one or more predetermined conditions are met (e.g., whether any other request meets one or more predetermined rules) response) from the source agent to the user.
During a DATA phase (e.g., DATA phase) following the index phase of the multiple phases, the SMPS migration management module 53 may trigger one or more other program modules (e.g., one or more migration agents) to migrate respective object DATA of a set of normal objects of the multiple normal objects from the remote SMPS compliant server to the storage tier of the storage server 10 as respective replacements for a set of index objects of the multiple index objects to complete the migration of the user DATA, e.g., a destination (e.g., the storage server 10) may begin the DATA phase at a second time T2 and complete the migration at a time T3, assuming that during the DATA phase, the user transmits any other request to the destination (e.g., the storage server 10) to attempt any change to any of the normal objects, similarly, based on whether one or more predetermined conditions are met (e.g., whether any other requirement meets one or more predetermined rules), storage server 10 may selectively proxy (e.g., forward) any other requirement to a source (e.g., remote SIMS server) and forward (e.g., forward) the response corresponding to any other requirement from the source proxy to the user, e.g., when any other requirement is a Provisioning (Get-object) requirement (e.g., "GetObject"), storage server 10 may proxy (e.g., forward) the requirement between the user and the source for the requirement, and particularly for some of its respective Provisioning data not yet migrated to the destination (e.g., during migration, when storage server 10 receives a Provisioning requirement, the respective Provisioning data of these indexed objects has not yet been copied to the destination, and needs to be replicated to the destination) proxies (e.g., forwards) the request. For a better understanding, the set of normal objects may include all of the plurality of normal objects and the set of index objects may include all of the plurality of index objects when any other request is not present (e.g., the user does not send any other request during time points T1 and T3).
One or more migration agents may be implemented by one or more of the following program modules, so long as the implementation of the present invention is not hindered:
(1) one or more of program modules 52P running on processing circuitry 52;
(2) for the case of a multi-node architecture having multiple processing circuits {52} in the master device 50, one or more program modules running on one or more other processing circuits in the multiple processing circuits {52}, wherein the multiple processing circuits {52} include processing circuits 52; and
(3) in the case of a multi-host architecture having multiple masters {50} in storage server 10, one or more program modules running on one or more corresponding processing circuits of one or more other masters {50} of the multiple masters {50}, where the multiple masters {50} comprise a master 50; the present invention is not limited thereto.
According to some embodiments, the method and related apparatus may be used to utilize a software library of Ceph (e.g., a "librados" software library) for certain basic operations for the architecture of Ceph, wherein the software library of Ceph may be used to provide client applications direct access to a reliable automated distributed object storage (RADS) storage system based on artifacts, and provides a basis for certain Ceph features, such as reliable automated distributed object storage (RBD) block devices, reliable automated distributed object storage gateways (RGW), Ceph file systems, etc., and reliable automated distributed object storage may be part of a Ceph distributed storage system, and may be viewed as a utility (utility) that interacts with a Ceph object store cluster (cluster), although the invention is not so limited.
FIG. 4 is a diagram illustrating some operations related to the migration control scheme based on index objects shown in FIG. 3, in which a first cloud and a second cloud (labeled "SIMS" and "Bigtera" for better understanding) are used as examples of the remote SIMS and storage server 10, respectively, and a user can access the remote SIMS and storage server 10 through a client device (such as SIMS client, labeled "S3 client" for simplicity), respectively, for example, each of the remote SIMS and storage server 10 can provide SIMS and storage services and can be SIMS, and especially can provide object storage to process a large amount of unstructured data, and may be accessed through an Application Program Interface (API) based on presentation state transfer (REST), such as a web service application interface (web API) that complies with the presentation state transfer architecture constraints, and by utilizing a hypertext transfer protocol (HTTP)/hypertext transfer security protocol (HTTPs) protocol. For some te-ads features, each remote te-ads compliant server and storage server 10 may be configured to store one or more objects in a bucket (e.g., container of objects), and any object of the one or more objects may include a file and optionally any metadata describing the file, wherein an object prefix (prefix) of any object may include one or more symbols "/", to support a folder concept (folder concept) to simulate a folder, and further, each remote te-ads compliant server and storage server 10 may have at least one Ceph object gateway (labeled "gateway" for brevity), such as an application built on librados to provide at least one object storage interface of a Ceph storage cluster with a gateway representing a layer state transition, wherein the at least one Ceph object gateway supports remote thin provisioning server and the storage server 10 is thin provisioning and Swift provisioning. As shown in the lower half of fig. 4, during the migration of user data (for simplicity, denoted as "data" in the open arrow), the user may send (by using the easy storage service client) any other request (e.g., GET request) to the storage server 10 to request a portion of the data of the user data, the storage server 10 may proxy (e.g., forward) any other request (e.g., GET request) to the remote easy storage service compatible server, and proxy (e.g., forward) the response corresponding to the any other request (e.g., a portion of data from the remote easy storage service compatible server to the user (e.g., the user's easy storage service client), denoted as "data" on some of the arrows indicated by the dashed lines for simplicity). For the sake of brevity, the detailed description of the similar contents of the present embodiment will not be repeated here.
According to some embodiments, in the INDEX phase (e.g., INDEX period), each INDEX object of the plurality of INDEX objects may be created to have certain attributes (attributes) of a source (src) object (e.g., a corresponding object in the first cloud, such as a normal object, wherein the INDEX object is in a role as a representation of the normal object), and the attributes may include a last-modified-time (mtime) of the source object (e.g., a source-modified-time (src-mtime), such as bigtera-scr-mtime, a size of the source object (e.g., a source-size, such as bigtera-src-size), and an entity tag (entity tag, Etag) of the source object (e.g., a file hash (5) for integrity check, such as MD, sub), for example, a source entity tag (src-etagt), such as bigtera-src-etag).
FIG. 5 is a diagram of some objects and some associated features of the indexed object based migration control scheme shown in FIG. 3. during user data migration, a source (e.g., a remote SMR compliant server) may include one or more buckets (e.g., one or more containers of objects), such as remote buckets, and the destination (e.g., storage server 10) may include one or more buckets (e.g., one or more containers of objects), such as the bucket shown above a remote bucket, wherein the request and the response respectively represent the request received from a user (e.g., the user's SMR) by the storage server 10 and the response returned to the user (e.g., the user's SMR) by the storage server 10, for example, the remote reservoir may contain a plurality of normal objects, small circles being shown in the remote reservoir for better understanding. In addition, the repository of the destination may include at least a portion (e.g., a portion or all) of a plurality of index objects, and in particular one or more normal objects, depending on the process of the migration (e.g., a percentage of completion of the migration, such as 1%, 2%, etc.) and/or the presence of any other requirements.
Before migrating the object data of any of the at least a portion of the plurality of index objects, the storage server 10 may set the destination repository to have the same repository name as the remote repository name (e.g., remote repository's repository name), and to have the same user credential as the remote user credential (e.g., remote repository's user credential), and in particular to have fake use information such as fake use information (e.g., size, number of objects, etc.) as temporary user information for the migration phase (e.g., index phase and data phase). Initially, before migrating the item data of any index item, the storage server 10 may create a plurality of index items having temporary attributes, such as virtual (dummy) index items (e.g., virtual items without item data), in a repository of a destination, when a normal item of a plurality of normal items has been migrated to the destination or an operation has been performed on the destination for the normal item (e.g., the item data of the normal item has been migrated or a user has overwritten the normal item), the normal item may be present in the repository (as shown in fig. 5), and further, for each index item of the plurality of index items, the temporary attributes may include:
(1) a source item last modification time (e.g., a source item last modification time), such as bigtera-src-mtime;
(2) a source object size (e.g., the size of the source object), such as bigtera-src-size;
(3) a source item entity tag (e.g., an entity tag of a source item), such as bigtera-src-etag;
any one of these temporary attributes (e.g., each attribute) may be implemented as an extended attribute (x-attribute/XATTR), but the present invention is not limited thereto, and once the artifact data of the source artifact is migrated to the destination or the user overwrites the artifact data at the destination, the destination (e.g., storage server 10) may remove the above listed temporary attributes, and for brevity, the detailed description of the similarity of this embodiment will not be repeated here.
According to some embodiments, taking the normal object shown in the upper right of fig. 5 as an example of a source object, when object data of the source object is migrated to a destination or a user overwrites the object data at the destination, the storage server 10 may invalidate (e.g., by removing a temporary attribute of) an index object serving as a representation of the normal object and/or replace the index object with the normal object in the repository of the destination, but the invention is not limited thereto.
Generally, changing an access control list of an object does not change the modification time of the object, and the related art data migration method does not seem to take this into account when it is proposed, and thus cannot properly handle deleted objects, read-only objects, and the like. The method and related apparatus of the present invention may suitably operate in any of a variety of conditions, further details of which will be described in the following examples.
Fig. 6 is a schematic diagram illustrating a migration agent control scheme of a method for seamless migration of a thin provisioning service using an index object according to an embodiment of the present invention, in response to a request for migrating user data from a remote thin provisioning service compatible server to a storage server 10, the storage server 10 may utilize a thin provisioning service migration management module 53 based on the index object to perform a pre-processing of migration of the user data during a set-up phase (e.g., a setup period) before an index phase in a plurality of phases, for example, the pre-processing may include operations of steps S01 to S05, but the present invention is not limited thereto.
In step S01, the index object-based thin provisioning service migration management module 53 may, for example, immediately initiate a thin provisioning service migration for a group of thin provisioning service users (e.g., a single user, such as the user described above, or a plurality of users including the user described above) upon a request trigger.
In step S02, the index-artifact-based thin provisioning service migration management module 53 may migrate the respective user credentials and the respective settings (e.g., quota) of the set of thin provisioning service users.
In step S03, the index-artifact-based thin store service migration management module 53 may create one or more repositories in the destination for each user of the set of users, and set up relevant migration information, e.g., the migration information may include agent parameters (e.g., remote user credentials to proxy (e.g., forward)) and other agent parameters, to direct the destination (e.g., the storage server 10) to proxy (e.g., forward) any other requirements to the source (e.g., a remote thin store service compatible server), and to proxy (e.g., forward) a response corresponding to the any other requirements from the source to the user.
In step S04, for each of the one or more banks, the index-artifact-based simplified storage service migration management module 53 may migrate the bank access control list (bucket ACL), the bank metadata (bucket metadata), and the bank quota (bucket quota) of a corresponding remote bank of the source end as the bank access control list, the bank metadata, and the bank quota of the one or more banks.
In step S05, the index-based te migration management module 53 may switch at least one endpoint to access user data from at least one cluster of remote ess compatible servers to at least one cluster of storage servers 10.
In step S06, after the setup phase, the INDEX artifact-based easy storage service migration management module 53 may create an INDEX bucket task (e.g., INDEX task) for each of the one or more buckets to perform the migration of the corresponding remote bucket of the source to the one or more buckets, e.g., the INDEX artifact-based easy storage service migration management module 53 may set the bucket to the INDEX phase (e.g., INDEX phase).
For better understanding, any one of the one or more migration agents (e.g., each agent) may be considered as a program module that actually performs the object migration, e.g., the INDEX object-based thin provisioning service migration management module 53 may create a plurality of job types (e.g., a plurality of INDEX jobs) corresponding to the INDEX phase and a plurality of DATA jobs (e.g., a plurality of DATA jobs)) and, in particular, transfer and/or assign the plurality of job types to the one or more migration agents to perform the DATA phase and the INDEX phase operations, respectively. Assuming that the one or more agents comprise a plurality of agents, the plurality of agents may be respectively implemented to run on a plurality of cluster nodes, wherein an agent runs on each of the plurality of cluster nodes, so that the index-artifact-based trivial storage service migration management module 53 may concurrently migrate user data (e.g., respective artifact data of the artifact) from a source to a destination in parallel by means of one or more migration agents (e.g., a plurality of migration agents).
In step S07, the INDEX-based SMPS migration management module 53 may execute INDEX work by one of the one or more migration agents to perform the following operations for each source object (e.g. each of the plurality of normal objects):
(1) creating a virtual object (e.g., an object without object data) having temporal attributes (e.g., source object last modification time, such as bigtera-src-mtime; source object size, such as bigtera-src-size; and source object physical tag, such as bigtera-src-etag) as one of a plurality of index objects, wherein the temporal attributes may be dummy attributes (e.g., dummy last modification time, dummy physical tag, such as dummy file hash and dummy size) of each object from source, but not real attributes of each object from source, and thus may be considered special attributes;
(2) the source end migrates the object access control list of the object to the destination;
wherein all index items of the plurality of index items are created after all operations of step S07 are completed.
When the indexing work (e.g., INDEX work) is complete, the INDEX artifact-based trivial storage service migration management module 53 may create a DATA work (e.g., DATA work) for each of the one or more volumes to perform a migration from the corresponding remote volume of the source to such one of the one or more volumes.
In step S08, for each of the one or more repositories, the index-artifact-based thin store service migration management module 53 may perform DATA work by an agent of the one or more migration agents to perform the following for each artifact of the source (e.g., each of the plurality of normal artifacts in the absence of any other requirement; or each normal artifact of the set of normal artifacts in the presence of any other requirement that may result in overwriting an artifact in the destination as a normal artifact in the repository):
(1) migrating the object data of the object from the source to the destination;
where the migration agent may migrate the most recent version of the bank quota (e.g., the original bank quota for the corresponding remote bank in the absence of any other requirements; or the bank quota updated in response to any changes made by the user in the presence of any other requirements) as the bank quota for the bank at the end of the job, e.g., the index object based trivial store service migration management module 53 may set the bank to a DATA phase (e.g., a DATA phase).
Of all banks corresponding to the set of SIMS users, while one of the banks is being processed in the INDEX phase (e.g., INDEX phase), another of the banks may be being processed in the DATA phase (e.g., DATA phase). Assuming that the one or more banks include multiple banks, while one bank of the multiple banks is being processed in the INDEX phase (e.g., INDEX phase), another bank of the multiple banks may be being processed in the DATA phase (e.g., DATA phase), and for brevity, the details of the similarity of this embodiment are not repeated here.
According to some embodiments, the INDEX phase (e.g., INDEX phase) may be much shorter than the DATA phase (e.g., DATA phase), in which the INDEX-based SMM 53 may perform INDEX work via the migration agent in step S07 to first list remote objects (e.g., source-side normal objects) for creating virtual objects and migrating object access control lists. Because any other requirement may exist in the indexing phase (e.g., the INDEX phase), different situations need to be considered, such as a first situation in which the INDEX object has not yet been created, and a second situation in which the storage server 10 is creating the INDEX object, etc., for example, processing for a PUT/POST requirement should be done at a local location (e.g., a destination, such as the storage server 10), and the storage server 10 can do the processing for the PUT/POST requirement and at the same time proxy the PUT/POST requirement to a remote location (remote site) (e.g., a source, such as a remote Simplex storage services consistent server) to make the results of the repository usage and List (List-Objects) requirements (e.g., ListObjects request) correct. In the event that any other request exists in the INDEX phase (e.g., INDEX phase), the user may overwrite one or more objects in that phase, and if an INDEX object is overwritten by the user, the object is no longer an INDEX object, and thus the migration agent will not migrate object DATA for that object in the DATA phase (e.g., DATA phase), and further, all INDEX objects may have been created (including their object access control list) prior to the DATA phase (e.g., DATA phase). In the DATA phase (e.g., DATA phase), the index-based SMPS migration management module 53 may perform DATA work by the migration agent in step S08 to list local objects (e.g., objects of the local location, such as index objects) and migrate the object DATA for each local object. Generally speaking, most operations may be normally performed, except for GET/HEAD object requests, in order to solve the contention hazard (competing hazard), the storage server 10 may utilize an atomic operation to create an index object and migrate data (e.g., object data of normal objects from the source) by means of the index object, for example, when the migration agent is creating the index object or migrating data about the index object, if a target object of a PUT operation already exists or does not have the above-mentioned temporary attribute, the PUT operation is cancelled, which represents that the user has overwritten the object. For proxying (e.g., forwarding) INDEX items located in a reliable automated distributed item storage gateway, where any other requirement exists, when a client device (e.g., a thin storage service client) operates on an INDEX item or the repository of the destination is in an INDEX stage, it may be desirable that the destination (e.g., storage server 10) can proxy the any other requirement to a remote location (e.g., a source end, such as a remote thin storage service compliant server) and transmit a proxy response (e.g., a response corresponding to the any other requirement) back to the client device, and similar contents of these embodiments will not be repeated here in detail for the sake of brevity.
Fig. 7 is a schematic diagram of an example of the Ceph object storage architecture of fig. 1 with the index object-based storage service migration management module 53, and the program module 52P of fig. 1 running on the processing circuit 52 may include a Ceph object gateway module (labeled "Ceph object gateway" in fig. 7 for brevity) including sub-modules such as a reliable automated distributed object storage gateway module (labeled "RADOS gateway" for brevity and better understanding) and a library of libraddos software (labeled "libraddos" for brevity), and the index object-based storage service migration management module 53 is implemented in the reliable automated distributed object storage gateway module, according to the present embodiment, a client device (labeled "client") may transmit hypertext transfer protocols (such as the storage service or Swift api) (for brevity, denoted "HTTP (S3 or Swift API)") to access the storage server 10, the storage server 10 may be implemented by a multi-node architecture, a multi-host architecture, etc., but the present invention is not limited thereto.
As shown in fig. 7, a reliable automated distributed item storage (e.g., an item-based storage system for reliable automated distributed item storage) of the storage server 10 may include a plurality of item storage device (OSD) nodes (labeled "OSD nodes" for brevity) coupled to the host device 50 of fig. 1 via at least one public network (e.g., one or more public networks), and may further include other partial architectures of the Ceph item storage architecture, such as at least one cluster network (e.g., one or more cluster networks) coupled to the plurality of item storage device nodes, an odd number of monitors (monitors) forming an arbitration (Quorum), and so forth. The at least one public network may be used to achieve a first predetermined speed such as one billion bits per second (labeled "1G" for brevity) and the at least one clustered network may be used to achieve a second predetermined speed such as one hundred gigabits per second (labeled "10G" for brevity), wherein the more bits per second the second predetermined speed, the better, the plurality of object storage device nodes may comprise some solid state drives for use with journals (e.g., multiple types of journals, such as object storage device journals, etc.), and more conventional hard drives, and particularly non-fault tolerant disk array drive architectures such as bonded disk/drives (JBOD), and similar contents of these embodiments will not be repeated herein for brevity.
For application program interfaces/operations for the SIMS presentation layer state transition, examples of object-related operations for SIMS actions may include, but are not limited to:
(1) an item listing request (e.g., Listobjects request);
(2) a Head-Object request (e.g., a Head Object request to obtain the Object properties);
(3) an artifact get request (e.g., GetObject request);
(4) a Get-Object-ACL request (e.g., getObjectaACLrequest);
(5) an Object Put (Put-Object) requirement (e.g., Put Object request);
(6) creating a multi-component Upload (Create-Multipart-Upload) request (e.g., Create mullttipartupload request);
(7) completing a multiple-component Upload (Complete-Multipart-Upload) request (e.g., Complete-Multipart Upload request);
(8) object Place Access control List (Put-Object-ACL) requests (e.g., PutObjectACLrequest);
(9) a Delete-Object request (e.g., Delete Object request);
some of the subsequent embodiments may focus on operations related to the items listed above.
FIG. 8 is a diagram illustrating a scenario listing control scheme for a method for seamless migration of SMPS using index scenarios according to an embodiment of the present invention, as shown in FIG. 8, a workflow for handling one or more scenario listing requests during user data migration may include steps S10-S14, wherein S14 may include sub-steps such as steps S14A-S14C.
In step S10, the storage server 10 (e.g., the abbreviated storage service migration management module 53 based on index artifacts) may receive an artifact list requirement from the user among the one or more artifact list requirements.
In step S11, in response to the item listing request, the INDEX-based savese migration management module 53 may check whether a repository (e.g., destination repository) corresponding to the item listing request in at least one repository in the storage tier of the storage server 10, which may represent at least one container (e.g., one or more containers) for the item, is being processed in the INDEX phase (e.g., INDEX phase), and if so, proceed to step S12; if not, then step S14 is entered (e.g., step S14A), for example, the item list requests that the item of the bank be listed.
In step S12, in response to the repository being processed in the indexing phase (e.g., INDEX phase), the INDEX-artifact-based thin store service migration management module 53 may control the storage server 10 to proxy (e.g., forward) the artifact list to a source peer (e.g., a remote thin store service consistent server).
In step S13, the index-artifact-based thin provisioning service migration management module 53 may control the storage server 10 to forward (e.g., forward) a response agent corresponding to the artifact listing request from the source (e.g., remote thin provisioning service compliant server) to the user.
In step S14, in response to the repository not being processed in the INDEX phase (e.g., INDEX phase), the INDEX-based easy storage service migration management module 53 may control the storage server 10 to perform at least one normal action according to the item listing request without proxying (e.g., forwarding) the item listing request to the source (e.g., remote easy storage service compatible server), for example, the INDEX-based easy storage service migration management module 53 may control the storage server 10 to check whether a local item listing result (e.g., a local item listing result of the local location, such as a result of performing the item listing request at the local location) is passed back to the user selectively according to an associated access control list.
In step S14A, the SMSC based on index object migration management module 53 may check whether the access control list check is passed, and if so, proceed to step S14B; if not, step S14C is entered, for example, when the access control list indicates that the user has the right to apply the object listing request to the repository (e.g., the right to execute the acquisition result of the object listing request), the index object-based SMS migration management module 53 may determine that the access control list check passes, and for example, when the access control list indicates that the user does not have the right to apply the object listing request to the repository (e.g., the right to execute the acquisition result of the object listing request), the index object-based SMS migration management module 53 may determine that the access control list check does not pass.
In step S14B, the index-artifact-based thin provisioning service migration management module 53 may control the provisioning server 10 to return the result of the local artifact listing to the user.
In step S14C, the index artifact-based thin provisioning service migration management module 53 may control the provisioning server 10 to prevent the local artifact listing from being returned to the user.
For better understanding, the method may be described by using the workflow shown in fig. 8, but the present invention is not limited thereto, and according to some embodiments, one or more steps in the workflow shown in fig. 8 may be added, deleted, or modified, for example, at least a portion (e.g., a portion or all) of the workflow may be executed multiple times, and particularly, any step including the checking operation may be executed multiple times to respectively generate multiple checking results to respectively trigger the execution of some subsequent steps.
FIG. 9 is a schematic diagram of an object header control scheme of a method for seamless migration of a compact storage service using index objects according to an embodiment of the present invention, as shown in FIG. 9, a workflow for handling one or more object header requests during user data migration may include steps S20-S28, S29A, and S29B, wherein step S25 may include sub-steps (e.g., S25A-S25C), and step S27 may include sub-steps (e.g., S27A-S27C).
In step S20, the storage server (e.g., the abbreviated storage service migration management module 53 based on index objects) may receive an object header request of one or more object header requests from the user, for example, the object header request requests to obtain an object attribute of an object in the storage of the destination, and the object may be regarded as a target object of the object header request, and the object attribute may include:
(1) an item last modification time, such as the last modification time of the item;
(2) an item physical tag, such as a physical tag of the item (e.g., a file hash);
(3) an object size, such as the size of the object;
wherein these attributes can be retrieved by an HTTP HEAD request.
In step S21, in response to the object header request, the INDEX-based SMM 53 may check whether a bank (e.g., destination bank) corresponding to the object header request in the at least one bank in the storage layer of the storage server 10 is being processed in the INDEX stage (e.g., INDEX stage), and if so, go to step S22; if not; step S26 is entered.
In step S22, in response to the storage being processed in the INDEX phase (e.g., INDEX phase), the SMPS migration manager 53 may check whether the object (e.g., the target object requested by the object header) exists, and if so, go to step S23; if not, go to step S28.
In step S23, in response to the existence of the object, the index-based thin provisioning migration management module 53 may check whether the object (e.g., the target object requested by the object header) is still an index object, and if so, go to step S24; if not, go to step S27 (e.g., step S27A).
In step S24, in response to the object still being an index object, the index-based easy storage service migration management module 53 may check whether the object access control list of the object is migrated; if so, go to step S25 (e.g., step S25A); if not, then step S29A is entered, for example, there is usually a short time delay between creating a virtual object (such as the virtual object described above) and migrating the object access control list, so the index-based SMM 53 can perform the checking operation of step S24 in the workflow to comply with the access control policy.
In step S25, the index-based storage service migration management module 53 may control the storage server 10 to perform at least one special action according to the object header requirement, for example, the index-based storage service migration management module 53 may control the storage server 10 to selectively check whether the special attribute (e.g., source modification time, such as bigtera-src-mtime, source entity tag, such as bigtera-src-etag, and source size, such as bigtera-src-size) recorded in the index object (e.g., the index object mentioned in step S23) passes back to the user according to an associated access control list.
In step S25A, the SMSC based on index object migration management module 53 may check whether the access control list check is passed, and if so, proceed to step S25B; if not, step S25C is entered, for example, when the access control list indicates that the user has the right to apply the item header requirement to the item (e.g., the right to obtain the result of executing the item header requirement), the SMM 53 based on the index item may determine that the access control list check passes, and for example, when the access control list indicates that the user does not have the right to apply the item header requirement to the item (e.g., the right to obtain the result of executing the item header requirement), the SMM 53 based on the index item may determine that the access control list does not pass.
In step S25B, the index-artifact-based thin provisioning service migration management module 53 may control the provisioning server 10 to return the attributes recorded in the special attributes of the index artifact (e.g., the index artifact mentioned in step S23) to the user.
In step S25C, the index artifact-based thin provisioning service migration management module 53 may control the provisioning server 10 to prevent the attributes recorded in the special attributes of the index artifact (e.g., the index artifact mentioned in step S23) from being returned to the user.
In step S26, in response to the container not being processed in the INDEX stage (e.g., INDEX stage), the SMM 53 may check whether the object (e.g., the target object requested by the object header) is still an INDEX object, and if so, go to step S25 (e.g., step S25A); if not, then step S27 is entered (e.g., step S27A), and for part of the workflow between steps S26 and S27, in the DATA phase (e.g., DATA phase), all object access control lists have been migrated, so there is no need to acquire the remote object access control list again.
In step S27, in response to the object (e.g., the target object requested by the object header) no longer being an index object (e.g., the object becoming a normal object), the index-based ess migration manager 53 may control the storage server 10 to perform at least one normal action according to the object header request, e.g., the index-based ess migration manager 53 may control the storage server 10 to check whether local metadata of the object (e.g., metadata of the object at a local location) is passed to selectively return to the user according to a relevant access control list.
In step S27A, the SMSC based on index object migration management module 53 may check whether the access control list check is passed, and if so, proceed to step S27B; if not, then step S27C is entered, e.g., the operation of step S27A may be similar to the operation of step S25A.
In step S27B, the index-based thin provisioning migration management module 53 may control the storage server 10 to return the local metadata of the object to the user.
In step S27C, the index-based thin provisioning migration management module 53 may control the storage server 10 to prevent the local metadata of the object from being returned to the user.
In step S28, the index-artifact-based thin store service migration management module 53 may control the storage server 10 to proxy (e.g., forward) the request (e.g., an artifact header request) to a source/remote location (e.g., a remote thin store service consistent server) to cause the remote location to check the artifact access control list.
In step S29, the index-artifact-based thin provisioning migration management module 53 may control the storage server 10 to proxy (e.g., forward) a response corresponding to the artifact header request to the user.
In step S29E, the SMPS migration management module 53 may control the storage server 10 to send back an error notification (e.g. '503 Service Unavailability'), for example, since the migration agent will migrate the ACCESS List in a very short time, this operation may notify the client to try again later.
For better understanding, the method may be described by using the workflow shown in fig. 9, but the present invention is not limited thereto, and according to some embodiments, one or more steps in the workflow shown in fig. 9 may be added, deleted, or modified, for example, at least a portion (e.g., a portion or all) of the workflow may be executed multiple times, and particularly, any step including the checking operation may be executed multiple times to respectively generate multiple checking results to respectively trigger the execution of some subsequent steps.
FIG. 10 is a diagram illustrating an object retrieval control scheme of a method for seamless migration of SMPS using indexed objects according to an embodiment of the present invention, and as shown in FIG. 10, a workflow for handling one or more object retrieval requests during user data migration may include steps S30-S36, wherein step S36 may include sub-steps (e.g., S36A-S36C).
In step S30, the storage server 10 (e.g., the abbreviated storage service migration management module 53 based on indexed artifacts) receives an artifact acquisition request from a user, wherein the artifact acquisition request acquires artifact data having attributes, such as artifact data of an artifact in the storage of the destination and the artifact attribute of the artifact, and the artifact can be regarded as a target artifact of the artifact acquisition request.
In step S31, in response to the object obtaining request, the INDEX-based thin provisioning migration management module 53 may check whether a bank (e.g., destination bank) corresponding to the object obtaining request in the at least one bank in the storage tier of the storage server is being processed in the INDEX stage (e.g., INDEX stage), and if so, go to step S32; if not, the flow proceeds to step S33.
In step S32, in response to the storage being processed in the INDEX phase (e.g., INDEX phase), the INDEX-based thin provisioning migration management module 53 may check whether the object (e.g., the target object requested by the object acquisition) exists, and if so, go to step S33; if not, the flow proceeds to step S34. For the partial workflow between steps S32 and S34, examples of reasons why the object does not exist may include: the index item has not yet been created; and the index object has been deleted by the user.
In step S33, the index-based savory service migration management module 53 may check whether the object (e.g., the target object requested by the object acquisition request) is still an index object, and if so, proceed to step S34; if not, the flow proceeds to step S36 (e.g., step S36A).
In step S34, the index artifact-based thin store service migration management module 53 may control the storage server 10 to proxy (e.g., forward) the request (e.g., the artifact acquisition request) to a source/remote location (e.g., a remote thin store service consistent server).
In step S35, the index-artifact-based thin provisioning service migration management module 53 may control the storage server 10 to forward (e.g., forward) a response agent corresponding to the artifact retrieval request to the user.
In step S36, in response to the object (e.g., the target object of the object acquisition request) no longer being an index object (e.g., the object becoming a normal object), the index-based sms migration management module 53 may control the storage server 10 to perform at least one normal action according to the object acquisition request, e.g., the index-based sms migration management module 53 may control the storage server 10 to check whether to selectively return local data of the object (e.g., the data of the object at the local location) to the user according to a relevant access control list.
In step S36A, the SMSC based on index object migration management module 53 may check whether the access control list check is passed, and if so, proceed to step S36B; if not, step S36C is entered, for example, when the access control list indicates that the user has the right to apply the object acquisition request to the object (e.g., the right to obtain the result of executing the object acquisition request), the SMS based on index object 53 may determine that the access control list check passes, and for example, when the access control list indicates that the user does not have the right to apply the object acquisition request to the object (e.g., the right to obtain the result of executing the object acquisition request), the SMS based on index object 53 may determine that the access control list does not pass.
In step S36B, the index-based thin provisioning migration management module 53 may control the storage server 10 to return the local data of the object to the user.
In step S36C, the index-based thin provisioning migration management module 53 may control the provisioning server 10 to prevent the local data of the object from being returned to the user.
For better understanding, the method may be described by using the workflow shown in fig. 10, but the present invention is not limited thereto, and according to some embodiments, one or more steps in the workflow shown in fig. 10 may be added, deleted, or modified, for example, at least a portion (e.g., a portion or all) of the workflow may be executed multiple times, and particularly, any step including the checking operation may be executed multiple times to respectively generate multiple checking results to respectively trigger the execution of some subsequent steps.
FIG. 11 is a diagram illustrating an object acquisition access control list control scheme of a method for seamless migration of SMR using index objects according to an embodiment of the present invention, and as shown in FIG. 11, a workflow for handling one or more object acquisition access control list requests during user data migration may include steps S40-S48, wherein step S45 may include sub-steps (e.g., S45A-S45C).
In step S40, the storage server 10 (e.g., the abbreviated storage service migration management module 53 based on indexed objects) receives an object acquisition access control list request from a user among one or more object acquisition access control list requests, for example, the object acquisition access control list request requests acquisition of an object access control list (e.g., an access control list of a storage in the destination storage), and the object can be regarded as a target object of the object acquisition access control list request.
In step S41, in response to the object get access control list request, the INDEX object based SMR management module 53 may check whether a bank (e.g., destination bank) corresponding to the object get access control list request in the at least one bank in the storage tier of the storage server 10 is being processed in the INDEX phase (e.g., INDEX phase), and if so, proceed to step S42; if not, the flow proceeds to step S45 (e.g., step S45A).
In step S42, in response to the storage being processed in the INDEX phase (e.g., INDEX phase), the SMPM 53 may check whether the object (e.g., the target object requested by the object acquisition AC) exists, and if so, go to step S43; if not, the flow proceeds to step S46.
In step S43, the index-based thin provisioning migration management module 53 may check whether the object (e.g., the target object requested by the object acquisition access control list) is still an index object, and if so, go to step S44; if not, the flow proceeds to step S45 (e.g., step S45A).
In step S44, in response to the object still being an index object, the SMM 53 may check whether the object ACC list of the object is migrated, and if so, go to step S45 (e.g., step S45A); if not, the flow proceeds to step S48.
In step S45, the index-based storage service migration management module 53 may control the storage server 10 to perform at least one normal operation according to the request for obtaining the ac list, for example, the index-based storage service migration management module 53 may control the storage server 10 to check whether the ac list of the object (e.g., the ac list of the object in the home location) is passed back to the user according to a relevant ac list.
In step S45A, the SMSC based on index object migration management module 53 may check whether the access control list check is passed, and if so, proceed to step S45B; if not, step S45C is entered, for example, when the access control list indicates that the user has the right to apply the object-get access control list request to the object (e.g., the right to perform the result of the object-get access control list request), the index-object-based SMM 53 may determine that the access control list check passes, and for example, when the access control list indicates that the user does not have the right to apply the object-get access control list request to the object (e.g., the right to perform the result of the object-get access control list request), the index-object-based SMM 53 may determine that the access control list check does not pass.
In step S45B, the sms migration management module 53 based on the index object can control the storage server 10 to return the local access control list of the object to the user.
In step S45C, the index-based thin provisioning migration management module 53 may control the provisioning server 10 to prevent the local access control list of the object from being returned to the user.
In step S46, the index artifact-based thin store service migration management module 53 may control the storage server 10 to proxy (e.g., forward) the request (e.g., the artifact get access control list request) to the originating source/remote location (e.g., a remote thin store service consistent server).
In step S47, the index-based SMM 53 may control the storage server 10 to forward (e.g., forward) a response agent corresponding to the object acquire access control list request to the user.
In step S48, the index-artifact-based easy storage Service migration management module 53 may control the storage server 10 to return an error notification (e.g., '503 Service Unavailable').
For better understanding, the method may be described by using the workflow shown in fig. 11, but the present invention is not limited thereto, and according to some embodiments, one or more steps in the workflow shown in fig. 11 may be added, deleted, or modified, for example, at least a portion (e.g., a portion or all) of the workflow may be executed multiple times, and particularly, any step including the checking operation may be executed multiple times to respectively generate multiple checking results to respectively trigger the execution of some subsequent steps.
FIG. 12 is a diagram illustrating an object placement control scheme of a method for seamless migration of SMPS using indexed objects according to an embodiment of the present invention, as shown in FIG. 12, a workflow for handling one or more object data write-related requests (e.g., one or more object placement requests, one or more create-multi-component upload requests, and one or more complete-multi-component upload requests) during user data migration may include steps S50-S58, wherein step S56 may include sub-steps (e.g., S56A-S56C).
In step S50, the storage server 10 (e.g., the index-artifact-based thin provisioning service migration management module 53) may receive an artifact data write-related request (e.g., an artifact placement/creation multi-component upload/completion multi-component upload request, such as an artifact placement request, a creation multi-component upload request, or a completion multi-component upload request) among one or more artifact data write-related requests (e.g., one or more placement requests, one or more creation multi-component upload requests, and one or more completion multi-component upload requests) from a user, for example, the artifact data write-related request requests to write an artifact to the repository of the destination, and the artifact may be considered a target artifact of the artifact data write-related request.
In step S51, in response to the object data write-related request, the SMPS migration management module 53 may check whether a bank (e.g., destination bank) corresponding to the object data write-related request in the at least one bank in the storage layer of the storage server 10 is being processed in the INDEX stage (e.g., INDEX stage), and if so, proceed to step S52; if not, the flow proceeds to step S56 (e.g., step S56A).
In step S52, the index-artifact-based thin store service migration management module 53 may control the storage server 10 to proxy (e.g., forward) the request (e.g., the artifact data write-related request) to the source peer (e.g., the remote thin store service consistent server) so that the remote thin store service consistent server checks the artifact access control list.
In step S53, the SMPS migration management module 53 may control whether the request (e.g., the object data write-related request) is successful without any error, and if so (e.g., a response received from the source, such as a response corresponding to the object data write-related request, indicating no error occurred), go to step S54; if not (e.g., the response received from the source, such as the response corresponding to the object data write-related request, is an error response indicating one or more errors), step S57 is entered.
In step S54, the SMPS migration management module 53 may check whether the object (e.g., the target object requested by the object data write) exists, and if so, proceed to step S55; if not, the flow proceeds to step S56 (e.g., step S56A).
In step S55, the SMSC migration management module 53 may check whether the object access control list of the object is migrated, and if so, proceed to step S56 (e.g., step S56A); if not, the flow proceeds to step S58.
In step S56, the index-based storage-ready service migration management module 53 may control the storage server 10 to perform at least one normal operation according to the object-data-writing-related request, for example, the index-based storage-ready service migration management module 53 may check whether to selectively write data (e.g., the object data carried by the object-data-writing-related request) into the object data of the object according to a related access control list.
In step S56A, the SMSC based on index object migration management module 53 may check whether the access control list check is passed, and if so, proceed to step S56B; if not, step S56C is entered, for example, when the access control list indicates that the user has the permission to apply the object data write-related request to the object (the permission to write the object data carried by the object data write-related request), the SMM 53 based on the index object may determine that the access control list check passes, and for example, when the access control list indicates that the user does not have the permission to apply the object data write-related request to the object (the permission to write the object data carried by the object data write-related request), the SMM 53 based on the index object may determine that the access control list check does not pass.
In step S56B, the index-based savepoint migration management module 53 may control the storage server 10 to write the data (e.g., the object data carried by the object data write-related request) into the object data of the object.
In step S56C, the index-based savepoint migration management module 53 may control the storage server 10 to prevent the data (e.g., the object data carried by the object data write-related request) from being written into the object data of the object.
In step S57, the index-based SMM 53 may control the storage server 10 to write (e.g., forward) the error response agent corresponding to the object data write-related request to the user.
In step S58, the index-based easy storage service migration management module 53 may control the storage server 10 to migrate the item access control list of the item from the source to the destination.
For better understanding, the method may be described by using the workflow shown in fig. 12, but the present invention is not limited thereto, and according to some embodiments, one or more steps in the workflow shown in fig. 12 may be added, deleted, or modified, for example, at least a portion (e.g., a portion or all) of the workflow may be executed multiple times, and particularly, any step including the checking operation may be executed multiple times to respectively generate multiple checking results to respectively trigger the execution of some subsequent steps.
FIG. 13 is a diagram illustrating an example of a TAB control scheme for a method for seamless migration of SMSC using indexed objects, as shown in FIG. 13, wherein a workflow for handling one or more TAB requests during user data migration may include steps S60-S65, wherein step S64 may include sub-steps (e.g., S64A-S64C).
In step S60, the storage server 10 (e.g., the abbreviated storage service migration management module 53 based on index objects) receives an object placement access control list request from a user for one or more object placement access control list requests, for example, the object placement access control list requests writing or updating of an object access control list (e.g., an access control list of an object in a destination storage), and the object can be regarded as a target object of the object placement access control list request.
In step S61, in response to the object placement access control list request, the SMPS migration management module 53 may check whether a bank (e.g., destination bank) corresponding to the object placement access control list request is being processed in the INDEX phase (e.g., INDEX phase) in the at least one bank in the storage tier of the storage server 10, and if so, proceed to step S62, and if not, proceed to step S64 (e.g., step S64A).
In step S62, the index-artifact-based thin store service migration management module 53 may control the storage server 10 to proxy (e.g., forward) the request (e.g., an artifact placement access control list request) to the source peer (e.g., a remote thin store service consistent server).
In step S63, the SMPS migration management module 53 may check whether the request (e.g., the DRAC request) is successful without any error, and if so (e.g., a response received from the source, such as a response corresponding to the DRAC request, indicating no error occurred), proceed to step S64 (e.g., step S64A); if not (e.g., the response received from the source, such as the response corresponding to the object placement access control list request, is an error response (error response) indicating one or more errors), then step S65 is entered.
In step S64, the index object-based sm management module 53 may control the storage server 10 to perform at least one normal operation according to the object placement ac list request, for example, the index object-based sm management module 53 may control the storage server 10 to check whether the object placement ac list request carries a new/latest ac list according to an associated ac list to selectively update the object ac list (e.g., the ac list of the target object) with the new/latest ac list carried by the object placement ac list request.
In step S64A, the SMSC based on index object migration management module 53 may check whether the access control list check is passed, and if so, proceed to step S64B; if not, step S64C is entered, for example, when the access control list indicates that the user has the right to apply the object-placing access control list request to the object (e.g., the right to write or update the object access control list), the index-object-based SMM 53 may determine that the access control list check passes, and for example, when the access control list indicates that the user does not have the right to apply the object-placing access control list request to the object (e.g., the right to write or update the object access control list), the index-object-based SMM 53 may determine that the access control list check does not pass.
In step S64B, the index-based thin provisioning migration management module 53 may control the storage server 10 to update the object ac with the new ac carried by the object placement ac request (e.g., the ac of the target object).
In step S64C, the index-based SMS migration management module 53 may control the storage server 10 to forward (e.g., forward) the error response agent corresponding to the object placement access control list request to the user.
For better understanding, the method may be described by using the workflow shown in fig. 13, but the present invention is not limited thereto, and according to some embodiments, one or more steps in the workflow shown in fig. 13 may be added, deleted, or modified, for example, at least a portion (e.g., a portion or all) of the workflow may be executed multiple times, and particularly, any step including the checking operation may be executed multiple times to respectively generate multiple checking results to respectively trigger the execution of some subsequent steps.
FIG. 14 is a schematic diagram of a artifact deletion control scheme of a method for seamless migration of SMES using indexed artifacts according to an embodiment of the present invention, and as shown in FIG. 14, a workflow for handling one or more artifact deletion requests during user data migration may include steps S70-S75, wherein step S74 may include sub-steps (e.g., S74A-S74D).
In step S70, the storage server 10 (e.g., the index-based thin provisioning migration management module 53) may receive an item deletion request from the user for one or more item deletion requests, e.g., for deleting an item in the destination storage, and the item may be regarded as a target item for the item deletion request.
In step S71, in response to the artifact deletion request, the SMS migration management module 53 may check whether a bank (e.g., destination bank) corresponding to the artifact deletion request among the at least one bank in the storage tier of the storage server 10 is being processed in the INDEX phase (e.g., INDEX phase), and if so, proceed to step S72, and if not, proceed to step S74 (e.g., step S74A).
In step S72, the index artifact-based thin store service migration management module 53 may control the storage server 10 to proxy (e.g., forward) the request (e.g., artifact deletion request) to the source peer (e.g., remote thin store service consistent server).
In step S73, the SMPM based on indexed artifacts may check whether the request (e.g., artifact deletion request) is successful without any error, and if so (e.g., a response received from the source, such as a response corresponding to the artifact deletion request, indicating no error occurred), proceed to step S74 (e.g., step S74A); if not (e.g., the response received from the source, such as the response corresponding to the item delete request, is an error response indicating one or more errors), then step S75 is entered.
In step S74, the index-based storage migration management module 53 may control the storage server 10 to perform at least one normal action according to the item deletion request, for example, the index-based storage migration management module 53 may control the storage server 10 to check whether the item (e.g., the target item) is selectively deleted at least according to an associated access control list, and for example, the index-based storage migration management module 53 may control the storage server 10 to selectively delete the item (e.g., the target item) at least according to whether the item exists, preferably, the index-based storage migration management module 53 may control the storage server 10 to selectively delete the item (e.g., the target item) according to whether a first condition and a second condition are satisfied, wherein the first condition represents that the access control list check passes and the second condition represents that the object exists.
In step S74A, the SMSC based on index object migration management module 53 may check whether the access control list check is passed, and if so, proceed to step S74B; if not, step S74D is entered, for example, when the access control list indicates that the user has the right to apply the object delete request to the object (e.g., the right to delete the object), the index-based SMS migration management module 53 may determine that the access control list check passes, and for example, when the access control list indicates that the user does not have the right to apply the object delete request to the object (e.g., the right to delete the object), the index-based SMS migration management module 53 may determine that the access control list check does not pass.
In step S74B, the SMPS migration management module 53 may check whether the object (e.g., the target object requested by the object deletion) exists, and if so, go to step S74C; if not, the flow proceeds to step S74D.
In step S74C, the index-artifact-based thin provisioning service migration management module 53 may control the storage server 10 to delete the artifact (e.g., the target artifact).
In step S74D, the index-artifact-based thin provisioning service migration management module 53 may control the storage server 10 to prevent deletion of the artifact (e.g., the target artifact).
In step S75, the index-artifact-based thin provisioning migration management module 53 may control the storage server 10 to forward (e.g., forward) the error response agent corresponding to the artifact deletion request to the user.
For better understanding, the method may be described by using the workflow shown in fig. 14, but the present invention is not limited thereto, and according to some embodiments, one or more steps in the workflow shown in fig. 14 may be added, deleted, or modified, for example, at least a portion (e.g., a portion or all) of the workflow may be executed multiple times, and in particular, any step including a checking operation may be executed multiple times to respectively generate multiple checking results to respectively trigger execution of some subsequent steps. In addition, the combination of steps S74A and S74B may be used to check whether the first condition and the second condition are both satisfied, and the conditions checked by steps S74A and S74B, respectively, may be exchanged, for example, the index artifact based thin provisioning service migration management module 53 may check whether the second condition is satisfied in step S74A, check whether the first condition is satisfied in step S74B, and proceed to step S74C if the first condition and the second condition are both satisfied (e.g., the access control list check is passed and the artifact exists); otherwise, the routine proceeds to step S74D.
Watch 1
Table one illustrates some examples of simple behavior of reliable automated distributed object store gateway actions of the store server 10 in the INDEX phase (e.g., INDEX phase) and DATA phase (e.g., DATA phase), where such requirements, such as GET object, GET object Access control List, HEAD object, PUT object Access control List, DELETE object, List object, and upload object (listed in the left half of Table one), represent one or more object acquisition requirements, one or more object acquisition Access control List requirements, one or more object header requirements, one or more object Place Access control List requirements, one or more object DELETE requirements, one or more object listing requirements, and one or more object DATA write-related requirements (e.g., one or more object Place requirements, one or more create Multi-component upload requirements, And one or more requirements to complete a multi-component upload). Under the control of the index artifact based thin provisioning service migration management module 53, the storage server 10 may operate correctly and efficiently, and for a better understanding, some of the operations described in the above embodiments may be summarized simply in the "index phase" and "data phase" fields of Table one.
For example, in the indexing phase, when any other request represents an item acquisition request, if the target item does not exist or has a last modification time (e.g., source modification time), the storage server 10 may proxy the request to the source; when any other request represents any of the object get access control list request and the object header request, if the target object does not exist, the storage server 10 may proxy the request to the source end; when any other request represents an object place ac request, the storage server 10 may proxy the request to the source, which writes the object ac carried by the request to a local location (e.g., a destination, such as the storage server 10); when any other requirement represents an item deletion requirement, the storage server 10 may proxy the requirement to the source peer and delete the local item (e.g., the target item) if the target item exists; when any other requirement represents an item listing requirement, the storage server 10 may proxy the requirement to the source; and when any other requirement represents an item data write-related requirement (e.g., an item placement requirement, a create multi-part upload requirement, or a complete multi-part upload requirement), the storage server 10 may proxy the requirement to the source peer and write the item data carried by the requirement to a local location (e.g., a destination, such as the storage server 10). In the data phase, when any other request represents an item acquisition request, if the target item has a last modification time (e.g., a source modification time), the storage server 10 may proxy the request; and when any other requirement represents any of the remaining requirements in all of these requirements, the storage server 10 may allow the processing of this requirement without proxying the requirement to the source.
Because the indexing phase only takes a short time, and because the index-artifact-based thin provisioning service migration management module 53 can properly control the storage server 10 during user migration by means of various control schemes of the method, the storage server 10 can simply perform thin provisioning service seamless migration (e.g., thin provisioning service seamless migration from any public/private cloud to a private cloud) to the private cloud to allow users to normally and smoothly access data during data migration, under the control of the index-artifact-based thin provisioning service migration management module 53, the storage server 10 can correctly migrate user data while properly handling any other requirements (if any) and thus does not generate any erroneous data (e.g., erroneous data due to related art incorrect migration management, typically found in the case where the user changes something (e.g., writes, overwrites, deletes, etc.) during migration), thereby making migration of the user seamless, and for the sake of brevity, the details of the similarities of this embodiment will not be repeated here.
The above-mentioned embodiments are merely preferred embodiments of the present invention, and all equivalent changes and modifications made by the claims of the present invention should be covered by the scope of the present invention.
Claims (18)
1. A method for seamless migration of a SIMS by using index objects, which is applied to a storage server, comprises:
in response to a request for migrating a user data of a user of the storage server from a remote SMPS server to the storage server, creating and storing index objects in a storage device layer of the storage server in an index stage of a plurality of stages of migration of the user data by using an index object-based SMPS migration management module of a plurality of program modules running on a host device in the storage server as respective representatives of a plurality of normal objects of the user data in the storage server, and migrating respective access control lists of the plurality of normal objects to the storage server as respective access control lists of the plurality of index objects, so as to minimize a probability that any change of any of the plurality of normal objects affects the correctness of the migration of the user data, the storage server comprises the main device and the storage device layer, wherein the storage device layer comprises at least one storage device coupled to the main device, the main device is used for controlling the operation of the storage server, and the at least one storage device is used for storing information for the storage server; and
in a data phase following the index phase of the plurality of phases, triggering one or more migration agents using the index-object-based SMM to migrate respective object data of a set of normal objects of the plurality of normal objects from the remote SMPS compliant server to the storage tier of the storage server as respective replacements for a set of index objects of the plurality of index objects to complete the migration of the user data.
2. The method of claim 1, further comprising:
in response to the request for migrating the user data from the remote SMPS to the storage server, utilizing the SMPS migration management module based on the index object to perform the pre-processing of the migration of the user data in a setup phase prior to the index phase of the plurality of phases.
3. The method of claim 2, wherein the preprocessing comprises switching at least one endpoint for accessing the user data from at least one cluster of the remote SMPS to at least one cluster of the storage servers.
4. The method of claim 1, wherein the respective access control list of the plurality of normal objects is migrated to the storage server as the respective access control list of the plurality of index objects, the plurality of index objects being the plurality of normal objects, to minimize a probability that any change in any of the plurality of normal objects affects the correctness of the migration of the user data.
5. The method of claim 1, wherein a workflow for processing one or more object list requests during the migration of the user data comprises:
receiving an item list request from the user among the one or more item list requests;
checking, in response to the object list request, whether a bin corresponding to the object list request among at least one bin in the storage tier is being processed in the indexing stage, wherein the at least one bin represents at least one container of an object; and
in response to the repository being processed in the indexing phase, controlling the storage server to proxy the object list request to the remote SMSC.
6. The method of claim 5, wherein the workflow for processing the one or more object list requests during the migration of the user data further comprises:
after controlling the repository server to proxy the object list request to the remote SMPS server, controlling the repository server to proxy a response corresponding to the object list from the remote SMPS server to the user.
7. The method of claim 5, wherein the step of checking whether the bank is being processed in the indexing stage is performed a plurality of times to generate a first check result and a second check result, respectively, wherein the first check result indicates that the bank is being processed in the indexing stage, and the second check result indicates that the bank is not being processed in the indexing stage; controlling the storage server to proxy the object list request to the remote SMSC compatible server in response to the first check result; and the workflow for processing the one or more item list requests during the migration of the user data further comprises:
and controlling the storage server to perform at least one normal action according to the object list request in response to the second check result without proxying the object list request to the remote SMPS server.
8. The method of claim 7, wherein the at least one normal action comprises:
controlling the storage server to selectively return a local item list result to the user based on whether an associated access control list check passes, wherein if the associated access control list check passes, the storage server returns the local item list result to the user, otherwise, the storage server prevents return of the local item list result to the user.
9. The method of claim 1, wherein a workflow for processing one or more object header requests during the migration of the user data comprises:
receiving an object head request from the user among the one or more object head requests;
checking, in response to the object head request, whether a bin corresponding to the object head request in at least one bin in the storage device layer is being processed in the indexing stage, wherein the at least one bin represents at least one container of an object;
checking whether a target object requested by the object header exists in response to the storage being processed in the indexing stage;
checking whether the target object is still an index object or not in response to the existence of the target object; and
in response to the target object still being the index object, checking whether an object access control list of the target object is migrated to the storage server, so as to selectively perform at least one predetermined action according to the object header request without proxying the object header request to the remote SMPS compliant server.
10. The method of claim 9, wherein the at least one predetermined action comprises:
controlling the storage server to selectively return the plurality of attributes recorded in the plurality of temporary attributes of the index object to the user according to whether an associated access control list check passes, wherein if the associated access control list check passes, the storage server returns the plurality of attributes recorded in the plurality of temporary attributes of the index object to the user, otherwise, the storage server prevents the plurality of attributes recorded in the plurality of temporary attributes of the index object from being returned to the user.
11. The method of claim 1, wherein a workflow for processing one or more object acquisition requests during the migration of the user data comprises:
receiving an object acquisition request from the user among the one or more object acquisition requests;
checking, in response to the object acquisition request, whether a bin corresponding to the object acquisition request in at least one bin in the storage tier is being processed in the indexing stage, wherein the at least one bin represents at least one container of the object;
checking whether a target object of the object acquisition request exists in response to the storage being processed in the indexing stage;
checking whether the target object is still an index object or not in response to the existence of the target object; and
and controlling the storage server to perform at least one normal action according to the object acquisition request in response to the target object still being the index object, without proxying the object acquisition request to the remote SIMS server.
12. The method of claim 11, wherein the at least one normal action comprises:
controlling the storage server to selectively return local data of the target object to the user according to an associated access control list check, wherein if the associated access control list check passes, the storage server returns the local data to the user, otherwise, the storage server prevents returning the local data to the user, wherein the local data represents object data of the target object in the storage server.
13. The method of claim 1, wherein a workflow for processing one or more object acquisition access control list requests during the migration of the user data comprises:
receiving an object get access control list request from the user among the one or more object get access control list requests;
checking, in response to the object get access control list request, whether a bank corresponding to the object get access control list request among at least one bank in the storage device layer is being processed in the indexing stage, wherein the at least one bank represents at least one container of the object;
checking whether a target object requested by the object acquisition access control list exists in response to the storage being processed in the indexing stage;
checking whether the target object is still an index object or not in response to the existence of the target object; and
and checking whether an object access control list of the target object is migrated to the storage server or not in response to the target object still being the index object, so as to selectively perform at least one normal action according to the object acquisition access control list request without requiring the object acquisition access control list request to be proxied to the remote SIMS compatible server.
14. The method of claim 13, wherein the at least one normal action comprises:
controlling the storage server to selectively return a local access control list of the target object to the user according to an associated access control list check, wherein if the associated access control list check passes, the storage server returns the local access control list to the user, otherwise, the storage server prevents returning the local access control list to the user, wherein the local access control list represents an access control list of the target object in the storage server.
15. The method of claim 1, wherein the storage server is implemented as a private cloud and the remote SIMS compatible server is implemented as a public cloud or another private cloud.
16. A host device includes:
a processing circuit for controlling the host device to perform index-based simple storage service migration management in a storage server, wherein the storage server comprises the host device and a storage device layer, the storage device layer comprises at least one storage device coupled to the host device, the host device is used to control the operation of the storage server, and the at least one storage device is used to store information for the storage server, wherein:
in response to a request to migrate user data of a user of the storage server from a remote SMSC to the storage server, in an indexing stage of a plurality of stages of migration of the user data, an index object-based SMPS migration management module of a plurality of program modules running on the processing circuitry creates and stores index objects to the storage tier, as respective representatives of a plurality of normal objects of the user data in the storage server, and migrating the respective access control lists of the plurality of normal objects to the storage server as respective access control lists of the plurality of index objects, to minimize the probability that any change to any of the plurality of normal objects affects the correctness of the migration of the user data; and
in a data phase following the index phase of the plurality of phases, the index-object-based SMPS migration management module triggers one or more migration agents to migrate respective object data of a set of normal objects of the plurality of normal objects from the remote SMPS server to the storage tier of the storage server as respective replacements for a set of index objects of the plurality of index objects to complete the migration of the user data.
17. The host device of claim 16, further comprising:
and the shell is used for installing a plurality of components of the main device and the at least one storage device, wherein the plurality of components of the main device comprise the processing circuit.
18. A storage server, comprising:
a host device for controlling the operation of the storage server, the host device comprising:
a processing circuit for controlling the host device to perform index-based easy storage service migration management in the storage server; and
a memory device layer, comprising:
at least one storage device coupled to the host device for storing information for the storage server;
wherein:
in response to a request to migrate user data of a user of the storage server from a remote SMSC to the storage server, in an indexing stage of a plurality of stages of migration of the user data, an index object-based SMPS migration management module of a plurality of program modules running on the processing circuitry creates and stores index objects to the storage tier, as respective representatives of a plurality of normal objects of the user data in the storage server, and migrating the respective access control lists of the plurality of normal objects to the storage server as respective access control lists of the plurality of index objects, to minimize the probability that any change to any of the plurality of normal objects affects the correctness of the migration of the user data; and
in a data phase following the index phase of the plurality of phases, the index-object-based SMPS migration management module triggers one or more migration agents to migrate respective object data of a set of normal objects of the plurality of normal objects from the remote SMPS server to the storage tier of the storage server as respective replacements for a set of index objects of the plurality of index objects to complete the migration of the user data.
Applications Claiming Priority (4)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US202063017001P | 2020-04-29 | 2020-04-29 | |
| US63/017,001 | 2020-04-29 | ||
| US17/167,095 US11516287B2 (en) | 2020-04-29 | 2021-02-04 | Method and apparatus for performing simple storage service seamless migration using index objects |
| US17/167,095 | 2021-02-04 |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113568567A true CN113568567A (en) | 2021-10-29 |
| CN113568567B CN113568567B (en) | 2023-10-24 |
Family
ID=78161177
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110360172.XA Active CN113568567B (en) | 2020-04-29 | 2021-04-02 | Method for seamless migration of simple storage service by index object, main device and storage server |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN113568567B (en) |
| TW (2) | TWI769796B (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110109891A (en) * | 2018-01-18 | 2019-08-09 | 伊姆西Ip控股有限责任公司 | Method, equipment and computer program product for Data Migration |
| CN110196851A (en) * | 2019-05-09 | 2019-09-03 | 腾讯科技（深圳）有限公司 | A kind of date storage method, device, equipment and storage medium |
| CN110806839A (en) * | 2018-08-06 | 2020-02-18 | 慧荣科技股份有限公司 | Storage control method, memory device, storage controller and storage server |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110765156A (en) * | 2018-07-09 | 2020-02-07 | 慧荣科技股份有限公司 | Linked list searching device and method |
| CN111522499B (en) * | 2019-02-01 | 2023-08-08 | 香港商希瑞科技股份有限公司 | Operation data reading device and reading method thereof |
-
2021
- 2021-04-02 CN CN202110360172.XA patent/CN113568567B/en active Active
- 2021-04-27 TW TW110115045A patent/TWI769796B/en active
- 2021-04-27 TW TW111119488A patent/TWI805380B/en active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110109891A (en) * | 2018-01-18 | 2019-08-09 | 伊姆西Ip控股有限责任公司 | Method, equipment and computer program product for Data Migration |
| CN110806839A (en) * | 2018-08-06 | 2020-02-18 | 慧荣科技股份有限公司 | Storage control method, memory device, storage controller and storage server |
| CN110196851A (en) * | 2019-05-09 | 2019-09-03 | 腾讯科技（深圳）有限公司 | A kind of date storage method, device, equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| TW202141289A (en) | 2021-11-01 |
| TWI805380B (en) | 2023-06-11 |
| TWI769796B (en) | 2022-07-01 |
| TW202234258A (en) | 2022-09-01 |
| CN113568567B (en) | 2023-10-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113568566B (en) | Method for seamless migration of simple storage service by index object, main device and storage server | |
| JP4809040B2 (en) | Storage apparatus and snapshot restore method | |
| KR101827239B1 (en) | System-wide checkpoint avoidance for distributed database systems | |
| KR101833114B1 (en) | Fast crash recovery for distributed database systems | |
| US7293145B1 (en) | System and method for data transfer using a recoverable data pipe | |
| US9996421B2 (en) | Data storage method, data storage apparatus, and storage device | |
| US11403269B2 (en) | Versioning validation for data transfer between heterogeneous data stores | |
| US9354907B1 (en) | Optimized restore of virtual machine and virtual disk data | |
| US9817587B1 (en) | Memory-based on-demand data page generation | |
| US11093387B1 (en) | Garbage collection based on transmission object models | |
| US11516287B2 (en) | Method and apparatus for performing simple storage service seamless migration using index objects | |
| CN113157487A (en) | Data recovery method and apparatus thereof | |
| CN113568567B (en) | Method for seamless migration of simple storage service by index object, main device and storage server | |
| US11645333B1 (en) | Garbage collection integrated with physical file verification | |
| TWI726403B (en) | Method for enhancing speed of incremental backup, bridge device, and storage system | |
| US20170308542A1 (en) | File system configuration data storage | |
| CN112286869A (en) | Snapshot creating method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |