WO2019127234A1 - 一种对象迁移的方法、设备和系统 - Google Patents

一种对象迁移的方法、设备和系统 Download PDF

Info

Publication number
WO2019127234A1
WO2019127234A1 PCT/CN2017/119464 CN2017119464W WO2019127234A1 WO 2019127234 A1 WO2019127234 A1 WO 2019127234A1 CN 2017119464 W CN2017119464 W CN 2017119464W WO 2019127234 A1 WO2019127234 A1 WO 2019127234A1
Authority
WO
WIPO (PCT)
Prior art keywords
migration
bucket
policy
storage server
attribute
Prior art date
Application number
PCT/CN2017/119464
Other languages
English (en)
French (fr)
Inventor
田树刚
白平昌
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Priority to PCT/CN2017/119464 priority Critical patent/WO2019127234A1/zh
Publication of WO2019127234A1 publication Critical patent/WO2019127234A1/zh
Priority to US16/911,974 priority patent/US11573725B2/en

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0628Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0646Horizontal data movement in storage systems, i.e. moving data in between storage devices or systems
    • G06F3/0647Migration mechanisms
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0602Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/061Improving I/O performance
    • G06F3/0611Improving I/O performance in relation to response time
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0602Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/0614Improving the reliability of storage systems
    • G06F3/0617Improving the reliability of storage systems in relation to availability
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0628Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0629Configuration or reconfiguration of storage systems
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0628Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0646Horizontal data movement in storage systems, i.e. moving data in between storage devices or systems
    • G06F3/0647Migration mechanisms
    • G06F3/0649Lifecycle management
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0628Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0653Monitoring storage devices or systems
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0668Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/067Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0668Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/0671In-line storage system
    • G06F3/0683Plurality of storage devices
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44Arrangements for executing specific programs

Definitions

  • Embodiments of the present invention relate to the field of storage technologies, and in particular, to a method, device, and system for an object storage service.
  • the Object Storage Service is an object-based mass storage service that provides customers with massive, secure, highly reliable, and low-cost data storage capabilities.
  • OBS is a service for Internet access. It provides a Web service interface based on HTTP/HTTPS protocol. Users can connect to the Internet computer anytime and anywhere, and access it through the client or Software Development Kit (SDK). Manage data stored in OBS.
  • SDK Software Development Kit
  • user files are stored on the cloud in the form of objects.
  • the stored carrier is named as a bucket. Users can create, modify, maintain, or delete buckets, and upload, download, and delete objects in the bucket. Wait for the operation to manage your own data.
  • the system's advanced functions and value-added services are all configured on the Bucket to provide advanced functions and value-added services at the granularity of the Bucket.
  • the user can manually migrate the objects between different buckets according to actual business conditions, and in some business situations, the user cannot perceive the operation of the object, for example, The frequency of being accessed, so that manual migration across the bucket cannot be completed based on the operation of the object.
  • the embodiments of the present invention provide a method, a device, a system, and a storage medium for an object storage service, which can automatically perform object migration between different buckets, thereby improving the efficiency of object migration.
  • an embodiment of the present application provides an object migration method, where the method is applied to an object storage service system, where the object storage service system includes an object storage server and a storage client, where the object storage server manages multiple bucket buckets. Each of the buckets includes one or more objects, and the storage client uses at least two of the plurality of buckets.
  • the object migration method is performed by the object storage server.
  • the object storage server acquires an object migration policy of the source bucket, the object migration policy indicates a condition that the object migrates from the source bucket in the plurality of buckets to the destination bucket, and the object storage server determines the number in the source bucket Whether an object satisfies the object migration policy of the source bucket, and if so, migrates the first object to the destination bucket according to the object migration policy.
  • the object storage server may determine, according to the pre-configured object migration policy, whether the object migration process needs to be performed for different objects, and migrate the object that satisfies the object migration policy to the destination bucket, without storing the client according to the basis.
  • the manual operation of the user indicates that the object migration process is performed, and the object migration efficiency is improved.
  • the object storage server receives a migration policy configuration request that is sent by the client and carries a migration policy parameter, where the migration policy parameter includes a bucket identifier of the source bucket corresponding to the object migration policy.
  • the migration attribute of the object may be the importance of the object, or may be the frequency at which the object is accessed, or in the case where the plurality of buckets are distributed in a plurality of different regions, the migration attribute of the object may also be The frequency at which an object is accessed by an access requester within a Region, the certain Region being different from the Region to which the destination bucket belongs.
  • the object storage server After receiving the migration policy configuration request, the object storage server performs an object migration policy configuration on the source bucket according to the migration policy parameter.
  • objects that need to be managed in a centralized manner can be placed in a bucket that is configured with more value-added functions or services, and other objects (for example, The object that is accessed less frequently or less importantly is placed in the source bucket configured with less value-added functions or services, thereby optimizing resource allocation and utilization, and achieving resource saving effects accordingly.
  • Buckets configured with more value-added functions or services require more resources, such as network resources, processing resources, and/or storage resources.
  • the migration policy configuration request further includes a priority parameter, where the priority parameter is associated with a migration attribute of the object.
  • the priority parameter is configured to: when the source bucket is configured with multiple object migration policies, and the first object respectively meets the multiple object migration policies, the object storage server selects according to the priority parameter The object migration strategy of the source bucket.
  • multiple object migration policies can be set for the same bucket without conflicts, and further fine-grained management of object ownership management for different buckets can be performed, and further Optimize resource allocation and utilization and conserve resources.
  • the object storage server further receives an object access request of the storage client, requests access to an object in the source bucket, and checks whether the source bucket configures the object according to the object access request. And migrating the policy; and after determining that the source bucket configures the object migration policy, reading the object migration policy.
  • This embodiment provides a foreground service startup mode to start the object migration processing operation in the foregoing embodiment.
  • the object migration processing operation in the above embodiment is started, for example, by storing a manner in which the client sends a specific object access.
  • the startup method mainly performs object migration processing for a specific object or an object in a specific bucket, and does not extensively check objects in all buckets managed by the object storage server, and objects that satisfy the object migration policy.
  • the migration process is relatively efficient, with a relatively short duration and less impact on other services.
  • the object migration processing flow can be started according to the activity level of the specific object. For the active object, the object migration processing flow of the object is started frequently, and for the inactive object, the object migration processing of the object is processed. The number of starts of the process is correspondingly reduced, so that the object migration process can be handled in a targeted manner, thereby saving processing resources.
  • the object storage server periodically checks whether each of the buckets managed by the object storage server respectively configures the object migration policy according to a preset time, and determines the source bucket configuration. After the object migration policy, the object migration policy is read.
  • the embodiment provides the background timing to start the object migration processing operation in the foregoing embodiment, for example, periodically starting according to a preset time
  • the background timing startup scenario is an object migration processing flow for the object in the bucket managed by the object storage server.
  • the migration processing of the completed object can be completed in a timely manner and completely, so that when the client accesses the object, the migrated object can be accessed in time, and the object migration processing is not performed after the user accesses, thereby further optimizing the resource and utilization.
  • the background timing start scenario is to perform scan check and object migration processing on objects in all buckets managed by the object storage server within a specific time period, if a lot of buckets are involved, the resource may be occupied slightly. bigger.
  • the object storage server may set scan detection and object migration processing for a certain number of buckets instead of all, and at another time. Read other buckets in the segment for scan check and object migration processing, so as to reduce the impact on resource utilization or other services.
  • the attribute value of the migration attribute is a duration of time that the object is stored in the source bucket.
  • the object storage server first acquires the current time of the object storage server, and then compares the time difference between the current time of the object storage server and the time when the first object is uploaded to the source bucket. And in the case that it is determined that the time difference is greater than or equal to the attribute value of the migration attribute, the first object is migrated into the destination bucket.
  • Buckets configured with more value-added functions or services require more resources, such as network resources, processing resources, and/or storage resources.
  • the attribute value of the migration attribute is a frequency value that the object is accessed.
  • the object storage server first counts the accessed frequency of the first object in the source bucket, and determines that the accessed frequency of the first object is greater than or equal to the attribute value of the migration attribute. The first object is then migrated into the destination bucket.
  • the attribute value of the migration attribute is a frequency value that the object is accessed.
  • the object storage server first counts the accessed frequency of the first object in the source bucket, and determines that the accessed frequency of the first object is less than or equal to the attribute value of the migration attribute. The first object is then migrated into the destination bucket.
  • Bucket By placing objects that are frequently accessed at a destination that is configured with more value-added functions or services, objects that are accessed less frequently are placed in the source that is configured with fewer value-added functions or services.
  • resource allocation and utilization can be optimized, and resource saving effects can be achieved accordingly.
  • Buckets configured with more value-added functions or services require more resources, such as network resources, processing resources, and/or storage resources.
  • a migration attribute of the object is an importance degree of the object and a frequency at which the object is accessed
  • an attribute value of the migration attribute is respectively stored in the source as the object
  • the object storage server acquires the current time of the object storage server, and compares the time difference between the current time of the object storage server and the time when the first object is uploaded to the source bucket.
  • the object storage server further needs to count the accessed frequency of the first object in the source bucket.
  • the object storage server determines that the time difference value is greater than or equal to a duration threshold of the object stored in the source bucket and the accessed frequency of the first object is greater than or equal to a frequency at which the object is accessed.
  • the first object is migrated into the destination bucket after the threshold.
  • objects with high frequency of access and high importance are placed in the bucket of the purpose of adding more value-added functions or services, and will be The objects with low access frequency and low importance are placed in the source bucket configured with less value-added functions or services, thereby further refining the bucket problem of the management object attribution, further optimizing resource allocation and utilization, and conserving resources. .
  • the attribute value of the migration attribute is the object is within the certain Region.
  • the frequency threshold for accessing the requester is the object storage server.
  • the object storage server counts the number of times the first object in the source bucket is accessed by an access requester in the certain Region, and the first in the source bucket. The number of times an object was accessed by an access requester within a plurality of different Regions.
  • the attribute value of the migration attribute is that the object is within the certain Region.
  • the object storage server counts the access frequency of the first object in the source bucket accessed by an access requester in the certain Region, and determines the content in the source bucket. Relocating the first object to the destination bucket after the access frequency of the first object accessed by the access requester in the certain Region is greater than or equal to the attribute value of the migration attribute, and the destination bucket belongs to The Region is for the certain Region.
  • the object frequently accessed by the access requester in a certain Region is migrated to the Region, thereby better solving the problem of the near access, thereby saving network resources. And improve access effectiveness.
  • the object storage server further determines whether to switch the statistical period, and after determining that the new statistical period needs to be switched, the source in the foregoing embodiment is counted by using the new statistical period.
  • the resource allocation can be performed more dynamically and flexibly by updating the statistical period according to the specific situation and counting the related statistical operations in the foregoing embodiment according to the new statistical period.
  • the object storage server in a scenario that the object access request of the storage client is a startup object migration process, the object storage server, according to the object access request of the storage client, the The number of times the first object is accessed is incremented by one to obtain the current number of times the first object in the source bucket is accessed. Then, the object storage server counts the accessed frequency of the first object in the source bucket according to the current number of accesses of the first object in the source bucket.
  • the statistics are more accurate and more accurately reflect the access situation of the object, thereby making the management of the object-owned bucket more refined, and the resource allocation and utilization are more optimized.
  • the object storage server actively notifies the storage client after migrating the first object into the destination bucket, or receives the access sent by the storage client. Notifying the first object to migrate to the destination bucket after the object access request of the first object; or notifying the first after receiving the object query request of the first object sent by the storage client An object migrates to the destination bucket.
  • the storage location of the object created and used by the storage client can be known in advance, so that when the object is subsequently accessed, the access is directly accessed according to the queried location. Improve access efficiency.
  • an embodiment of the present application provides a method for an object storage service, where the method is applied to an object storage service system, where the object storage service system includes an object storage server and a storage client, where the object storage server manages multiple A bucket bucket, each of the buckets including one or more objects including a bucket created or used by the storage client.
  • the method is performed by a storage client, in which the storage client determines a migration policy parameter, where the migration policy parameter includes an identifier of a source bucket corresponding to the object migration policy, a migration attribute of the object, and an attribute of the migration attribute. The value, as well as the identity of the destination bucket.
  • the storage client sends a migration policy configuration request carrying the migration policy parameter to the object storage server, and requests the object storage server to configure an object migration policy for the source bucket.
  • the object migration policy represents a condition for the object to migrate from the source bucket to the destination bucket.
  • the storage client sends a migration policy configuration request carrying the migration policy parameter to the object storage server, so that the object storage server configures the corresponding object migration policy for the bucket created or used by the storage client.
  • the object storage server can determine whether the object migration processing needs to be performed for different objects in different buckets according to the configured object migration policy, and the object that satisfies the object migration policy is migrated to the destination bucket, without storing the client according to the manual of the user.
  • the operation instruction performs object migration processing, which improves the object migration efficiency.
  • the migration attribute of the object may be an importance degree of the object; or may be a frequency at which the object is accessed; or, the plurality of buckets are distributed in multiple different regions.
  • the migration attribute of the object may also be a frequency at which the object is accessed by an access requester within a Region, wherein the certain Region is different from the Region to which the destination bucket belongs.
  • the storage client can place objects that need to be managed in a centralized manner (for example, objects with high frequency of access or high importance) in a bucket that is configured with more value-added functions or services. And other objects (for example, objects that are accessed less frequently or less important) are centralized into the source buckets configured with fewer value-added functions or services, thereby optimizing resource allocation and utilization, and correspondingly achieving The effect of saving resources. Buckets configured with more value-added functions or services require more resources, such as network resources, processing resources, and/or storage resources.
  • the migration policy configuration request further includes a priority parameter, where the priority parameter is associated with a migration attribute of the object.
  • the priority parameter is used in a scenario in which the source bucket is configured with multiple object migration policies, and an object satisfies the multiple object migration policies respectively, and the object storage server selects the source according to the priority parameter. Bucket's object migration strategy.
  • multiple object migration policies can be set for the same bucket without conflicts, and further fine-grained management of object ownership management for different buckets can be performed, and further Optimize resource allocation and utilization and conserve resources.
  • the storage client determines the migration policy parameter for each bucket, including the source bucket, according to a configuration of each bucket created or used by the storage client.
  • the configuration of the bucket includes a value-added function or service of each bucket.
  • the value-added function or service of the bucket includes at least one of the following: network acceleration, data high reliability, stored QoS, storage performance, near access of data, or multi-version control of data.
  • the storage client determines the migration policy parameters of each bucket according to the configuration of each bucket, so that a better object migration policy can be configured, and the object is migrated between buckets with different levels of value-added functions or services, and the bucket attachment of the object can be performed. More refined management is carried out to achieve optimal allocation and utilization of resources.
  • the storage client updates the migration policy parameter according to a change in a value-added function or service of the bucket, and sends a migration policy configuration update request to the object storage server to request the object.
  • the storage server updates the object migration policy of the source bucket configuration
  • the storage client updates the migration policy parameter to the object storage server according to the change of the configuration of the bucket created and used by the storage client, so as to update and adjust the object migration policy accordingly, so that the object's bucket management and The configuration and utilization of resources is more dynamic and flexible.
  • the storage client sends the migration policy configuration request to the object storage server by using a bucket configuration interface between the storage client and the object storage server.
  • the storage client receives a notification sent by the object storage server, and the notification informs the storage client that the first object in the source bucket is migrated to the destination bucket. in.
  • the storage client queries the object storage server for an object migration situation in the source bucket.
  • an embodiment of the present application provides an object storage server in a hardware form, where the object storage server manages a plurality of bucket buckets, and each of the buckets includes one or more objects.
  • the object storage server includes a policy acquisition module, a policy determination module, and a migration module.
  • the policy acquisition module is configured to obtain an object migration policy of the source bucket, where the object migration policy indicates a condition for the object to migrate from the source bucket to the destination bucket.
  • the policy determining module is configured to determine whether the first object in the source bucket satisfies the object migration policy acquired by the policy acquiring module.
  • the migration module is configured to migrate the first object to the destination bucket if the policy determination module determines that the first object satisfies the object migration policy.
  • the object storage server may determine, according to the pre-configured object migration policy, whether the object migration process needs to be performed for different objects, and migrate the object that satisfies the object migration policy to the destination bucket, without storing the client according to the basis.
  • the manual operation of the user indicates that the object migration process is performed, and the object migration efficiency is improved.
  • the object storage server further includes a receiving module and a policy configuration module.
  • the receiving module is configured to receive a migration policy configuration request that is sent by the client and that carries a migration policy parameter, where the migration policy parameter includes an identifier of the source bucket, a migration attribute of the object, an attribute value of the migration attribute, and the The identifier of the destination bucket.
  • the migration attribute of the object may be the importance of the object, or may be the frequency at which the object is accessed, or in the case where the plurality of buckets are distributed in a plurality of different regions, the object
  • the migration attribute may also be a frequency at which the object is accessed by an access requester within a Region, the Certain Region being different from the Region to which the destination bucket belongs.
  • the policy configuration module is configured to configure an object migration policy for the source bucket according to the migration policy configuration request.
  • objects that need to be managed in a centralized manner can be placed in a bucket that is configured with more value-added functions or services, and other objects (for example, The object that is accessed less frequently or less importantly is placed in the source bucket configured with less value-added functions or services, thereby optimizing resource allocation and utilization, and achieving resource saving effects accordingly.
  • Buckets configured with more value-added functions or services require more resources, such as network resources, processing resources, and/or storage resources.
  • the embodiment of the present application provides a storage client in a hardware form, where the storage client uses an object storage server to manage at least another bucket in a plurality of bucket buckets, each of the buckets including one or more objects.
  • the storage client includes a migration policy determination module and a sending module.
  • the migration policy determining module is configured to determine a migration policy parameter, where the migration policy parameter includes an identifier of a source bucket corresponding to the object migration policy, a migration attribute of the object, an attribute value of the migration attribute, and an identifier of the destination bucket.
  • the sending module is configured to send, to the object storage server, a migration policy configuration request that carries the migration policy parameter, and request the object storage server to perform an object migration policy configuration on the source bucket, where the object migration policy indicates an object from the The condition that the source bucket migrates to the destination bucket.
  • the storage client sends a migration policy configuration request carrying the migration policy parameter to the object storage server, so that the object storage server configures the corresponding object migration policy for the bucket created or used by the storage client.
  • the object storage server can determine whether the object migration process needs to be performed for different objects in different buckets according to the configured object migration policy, and migrate the object that satisfies the object migration policy to the destination bucket without storing the client according to the user.
  • Manual operation indicates object migration processing, which improves object migration efficiency.
  • the migration attribute of the object may be the importance of the object, or may be the frequency at which the object is accessed; or, the multiple buckets are distributed in multiple different regions.
  • the migration attribute of the object may also be the frequency at which the object is accessed by the access requester in a certain Region, and the certain Region is different from the Region to which the destination bucket belongs.
  • the storage client can place objects that need to be managed in a centralized manner (for example, objects with high frequency of access or high importance) in a bucket that is configured with more value-added functions or services. And other objects (for example, objects that are accessed less frequently or less important) are centralized into the source buckets configured with fewer value-added functions or services, thereby optimizing resource allocation and utilization, and correspondingly achieving The effect of saving resources. Buckets configured with more value-added functions or services require more resources, such as network resources, processing resources, and/or storage resources.
  • the migration policy determination module determines the migration policy parameter for the source bucket according to the configuration of each bucket used by the storage client.
  • the configuration of the bucket includes a value-added function or service of each bucket, and the value-added function or service of the bucket includes at least one of the following: network acceleration, high data reliability, stored QoS, and near access of data. Or multi-version control of data.
  • the storage client determines the migration policy parameters of each bucket according to the configuration of each bucket, so that a better object migration policy can be configured, and the object is migrated between buckets with different levels of value-added functions or services, and the bucket attachment of the object can be performed. More refined management is carried out to achieve optimal allocation and utilization of resources.
  • the migration policy determining module updates the migration policy parameter according to the value-added function of the bucket or a change of the service, and sends the update to the object storage server by using the sending module.
  • the migration policy configuration update request of the migration policy parameter requests the object storage server to update the object migration policy configured by the source bucket.
  • the storage client updates the migration policy parameter to the object storage server according to the change of the configuration of the bucket created and used by the storage client, so as to update and adjust the object migration policy accordingly, so that the object's bucket management and The configuration and utilization of resources is more dynamic and flexible.
  • an embodiment of the present application provides an object storage service system, where the object storage service system includes an object storage server, a data persistence component, a metadata database, and a plurality of hard disks.
  • the object storage server is configured to manage a plurality of bucket buckets, each of the buckets includes one or more objects, and receives a migration policy configuration request that carries a migration policy parameter sent by the storage client, according to the migration policy parameter.
  • the migration policy parameter includes an identifier of a source bucket corresponding to the object migration policy, a migration attribute of the object, an attribute value of the migration attribute, and an identifier of the destination bucket.
  • a data persistence component is used to store the plurality of buckets.
  • the metadata database is used to save the object migration policy.
  • the plurality of hard disks are used to provide physical storage space.
  • the object storage server may perform the configuration of the object migration policy on the source bucket according to the migration policy configuration request that is sent by the storage client and carries the migration policy parameter. Then, according to the object migration policy, the object that meets the object migration policy in the source bucket is processed by a migration operation process, for example, according to the object migration policy, whether the object migration processing needs to be performed for different objects in the bucket And migrate the objects that satisfy the object migration policy to the destination bucket. Therefore, the storage client does not need to perform object migration processing according to the manual operation instruction of the user, thereby improving object migration efficiency.
  • the migration attribute of the object may be the importance of the object, or may be the frequency at which the object is accessed, or when the plurality of buckets are distributed in a plurality of different regions,
  • the migration attribute of the object may also be the frequency with which the object is accessed by an access requester within a Region, the certain Region being different from the Region to which the destination bucket belongs.
  • objects that need to be managed in a centralized manner can be placed in a bucket that is configured with more value-added functions or services, and other objects (for example, The object that is accessed less frequently or less importantly is placed in the source bucket configured with less value-added functions or services, thereby optimizing resource allocation and utilization, and achieving resource saving effects accordingly.
  • Buckets configured with more value-added functions or services require more resources, such as network resources, processing resources, and/or storage resources.
  • the migration policy configuration request further includes a priority parameter, where the priority parameter is associated with a migration attribute of the object.
  • the priority parameter is included in the object migration policy.
  • the priority parameter is configured to: when the source bucket is configured with multiple object migration policies, and the first object respectively meets the multiple object migration policies, the object storage server selects according to the priority parameter The object migration strategy of the source bucket.
  • multiple object migration policies can be set for the same bucket without conflicts, and further fine-grained management of object ownership management for different buckets can be performed, and further Optimize resource allocation and utilization and conserve resources.
  • an embodiment of the present application provides an object storage server, where the object storage server includes a processor, the processor is coupled to a memory, and the processor implements the program on the memory to implement the first aspect as described above. Or based on any of the various possible implementations of the first aspect.
  • the embodiment of the present application provides a storage client, where the storage client includes a processor, the processor is coupled to a memory, and the processor implements the program on the memory to implement the second aspect as described above. Or based on any of the various possible implementations of the second aspect.
  • an embodiment of the present application provides an object storage service system, where the object storage service system includes any one of the object storage servers according to the third aspect or based on various possible implementation manners of the third aspect, a data persistence component, a metadata database, and a plurality of hard disks; a data persistence component for storing the plurality of buckets; the metadata database for saving the object migration policy; the plurality of hard disks for providing physical storage.
  • the embodiment of the present application provides a storage system, where the storage system includes any one of the object storage servers according to the foregoing third aspect or the various possible implementations of the third aspect, and the fourth aspect or Any one of the various possible implementations of the fourth aspect is a storage client.
  • the embodiment of the present application provides a computer readable storage medium, where the program storage code is stored in the computer storage medium, and the program code is used to indicate that the first aspect is performed.
  • the embodiment of the present application provides a computer readable storage medium, where the program code is stored in the computer storage medium, and the program code is used to indicate a method in performing any possible implementation manner of the foregoing second aspect.
  • FIG. 1 is a schematic structural diagram of a storage system according to an embodiment of the present disclosure
  • FIG. 2 is a flowchart of an object migration method according to an embodiment of the present invention.
  • FIG. 3 is a flowchart of another method for configuring an object migration policy according to an embodiment of the present invention.
  • FIG. 4 is a flowchart of a practical application scenario of an object migration method according to an embodiment of the present invention.
  • FIG. 5 is a flowchart of a practical application scenario of another object migration method according to an embodiment of the present invention.
  • FIG. 6 is a flowchart of a method for object migration according to an embodiment of the present invention.
  • FIG. 7 is a flowchart of still another object migration method according to an embodiment of the present invention.
  • FIG. 8 is a flowchart of still another method for object migration according to an embodiment of the present invention.
  • FIG. 9 is a schematic structural diagram of an object storage server according to an embodiment of the present disclosure.
  • FIG. 10 is a schematic structural diagram of a storage client according to an embodiment of the present disclosure.
  • FIG. 11 is a schematic structural diagram of another object storage server according to an embodiment of the present disclosure.
  • FIG. 12 is a schematic structural diagram of a storage client according to an embodiment of the present invention.
  • the network architecture and the service scenario described in the embodiments of the present invention are used to more clearly illustrate the technical solutions of the embodiments of the present invention, and do not constitute a limitation of the technical solutions provided by the embodiments of the present invention.
  • the technical solutions provided by the embodiments of the present invention are equally applicable to similar technical problems.
  • the storage system 100 includes an object storage service system 110 and a storage client 112 .
  • the object storage service system 110 includes an object storage server 102, a data persistence component 104, a metadata database 106, and a hard disk server 108.
  • the object storage service system 110 provides an object storage service to a storage client.
  • the user uses the object storage service through the storage client 112, for example, the user performs various operations on the data or files stored in the cloud through the storage client 112, such as creating, modifying, maintaining, or deleting the bucket, and uploading and downloading. , delete objects in the bucket and other operations to manage their own data.
  • the object storage server 102 is configured to provide an object storage interface and an implementation of the interface function, provide an interface for the interaction between the storage client 112 and the object storage service system 110, manage multiple buckets, and accept various operation requests of the storage client. For example, an operation request such as creating, modifying, maintaining, or deleting a bucket, or uploading, downloading, and deleting an object in a bucket, processing according to the operation request, and returning a processing result to the storage client.
  • the object storage server 102 is further configured to receive a migration policy configuration request that carries a migration policy parameter sent by the client 112, configure an object migration policy for the source bucket according to the migration policy parameter, and configure the object
  • the migration policy is saved in the metadata database, and the objects in the bucket are migrated according to the object migration policy.
  • the data persistence component 104 is configured to virtualize the hard disk space, provide a logical space in the form of a bucket, provide a data storage interface to the object storage server 102, and store data of the user and the storage client;
  • the metadata database 106 is configured to provide a metadata storage interface to the object storage server 102, and store metadata of the object storage service system and control data.
  • the metadata database 106 is configured to provide a metadata storage interface to the object storage server 102, and store metadata of the object storage service system and control data.
  • the hard disk server 108 is configured to mount all the hard disks of the object storage service system, and provides physical storage space for upper layer components, such as the object storage server 102, the data persistence component 104, and the metadata database 106.
  • the object storage server 102 is equivalent to the object storage service system 110.
  • FIG. 2 shows an object migration method provided in an embodiment of the present invention.
  • the method is applicable to the object storage service system 100 shown in FIG. 1 above, the object storage service system 100 includes an object storage server and a storage client, and the object storage server manages a plurality of bucket buckets, each of the buckets including One or more objects, the storage client using at least two of the plurality of buckets.
  • the object migration method includes the following processing:
  • the object storage server obtains an object migration policy corresponding to the source bucket.
  • the object storage server obtains an object migration policy previously configured for the source bucket from a metadata database in the object storage service system, the object migration policy indicating an object from the plurality of The condition that the source bucket in the Bucket migrates to the destination bucket.
  • an object migration policy previously configured for the source bucket from a metadata database in the object storage service system, the object migration policy indicating an object from the plurality of The condition that the source bucket in the Bucket migrates to the destination bucket.
  • the object migration policy may be composed of at least the following migration policy parameters: an identifier of the source bucket, a migration attribute of the object, an attribute value of the migration attribute, and a bucket identifier of the destination bucket.
  • the migration attribute of the object includes an importance degree of the object or a frequency at which the object is accessed, or in a case where a plurality of buckets are distributed in a plurality of different regions, the migration attribute of the object includes the object being The frequency or frequency of access by an access requestor within a Region (e.g., Region 1) that is different from the Region to which the destination bucket belongs (e.g., Region 2).
  • the frequency used in the embodiment of the present application represents a ratio of two different frequencies.
  • the frequency that an object is accessed by an access requester in a certain Region refers to the object.
  • the identifier of the bucket does not necessarily represent a specific bucket.
  • the identifier of the bucket may represent a type of bucket composed of a plurality of destination buckets having the same feature, for example, a bucket in Region 2. .
  • the migration attribute of the object can be differently set according to different business scenarios.
  • the above is merely an example, and does not mean that the present invention is limited to the attributes of the above three enumerated objects.
  • the migration attribute of different objects, the attribute value of the migration attribute, and the bucket identifier of the source bucket and the bucket identifier of the destination bucket associated with the two parameters constitute an object migration policy, which may be referred to as a first object.
  • the migration policy for example, the first object migration policy may include: a bucket identifier of the source bucket, an importance degree of the object, an attribute value related to the importance degree, and a bucket identifier of the destination bucket.
  • the second object migration policy includes: a bucket identifier of the source bucket, a frequency at which the object is accessed, a frequency threshold, and a bucket identifier of the destination bucket.
  • the third object migration policy includes: a bucket identifier of the source bucket, and a migration attribute of the object includes The frequency or frequency, frequency threshold or frequency threshold at which the object is accessed by an access requestor within a Region, and the bucket identifier of the destination bucket.
  • the migration policy parameter may further include a priority parameter, where the priority parameter is associated with the migration attribute of the object, such that each object has a corresponding priority in the previous policy.
  • the priority parameter is used to select an object migration policy of the source bucket according to the priority parameter in a scenario in which the source bucket is configured with multiple object migration policies and a certain object satisfies a plurality of the object migration policies. .
  • the object storage server determines whether the first object in the source bucket satisfies the object migration policy.
  • the object storage server determines whether the first object in the source bucket satisfies an object migration policy configured for the source bucket.
  • the specific judging process may have different judging processes according to different object migration policies, and may be specifically described with reference to the detailed description in the embodiments shown in FIG. 6-8 below.
  • Figure 6-8 illustrates the processing flow of three different object migration policies, respectively.
  • Figure 6 depicts the object migration processing flow of the first object migration policy with the frequency of the object being accessed as the migration attribute.
  • Figure 7 depicts The importance of the object is the second object migration policy object migration process flow of the migration attribute
  • FIG. 8 depicts the object migration process of the third object migration policy with the frequency of the object being accessed by the access requester within a certain Region as the migration attribute. Process.
  • step 206 If the result of the determination in step 204 is that the first object in the source bucket satisfies the object migration policy, the object storage server migrates the first object to the target bucket.
  • the object may be migrated to the destination bucket corresponding to the policy. For example, if the object storage server determines that the first object satisfies the importance degree of the set object and the attribute value related to the importance degree, the first object is migrated to the destination bucket corresponding to the destination bucket identifier corresponding to the strip object migration policy, Then, it is judged whether or not the other migration attribute, that is, the corresponding attribute value is satisfied.
  • the object storage server may continue to determine whether other object migration policies are met, and meet other object migration policies, and the destination bucket does not In the case of a conflict, the first object is migrated to the destination bucket represented by the identifier of the destination bucket in each eligible object migration policy.
  • the object storage server may continue to determine whether other object migration policies are met, and other destination migration policies are met, but the destination bucket exists.
  • the migration policy can be set to a priority, and the object migration is performed according to the object migration policy with a higher priority.
  • the setting of the priority parameter may be set in the migration policy parameter. For details, refer to the description of step 202 above and the detailed description of step 306 in the embodiment shown in FIG. 3 below. It can be understood that the priority of the object migration policy can also be set in other ways.
  • the object storage server notifies the storage client of the result of the object migration.
  • the object storage server after the first object is successfully migrated, notifies the storage client of the result of the object migration.
  • the notification message When the destination bucket identifier in the object migration policy indicates a specific destination bucket, the notification message carries the bucket identifier of the destination bucket and the object identifier of the first object.
  • the notification may include the bucket identifier of the destination bucket, the identifier of the first object, and the number of the specific destination bucket to which the migration is performed. Identification information such as name or address.
  • the object storage server does not immediately notify the storage client of the result of the object migration. In this case, it is difficult for the storage client to perceive that the storage object is migrated, so the objects in the source bucket before the migration may still be accessed.
  • the object storage server receives the object access request of the storage client, check whether the object corresponding to the object identifier carried in the object access request has been migrated to another bucket, and if it has been migrated to another bucket, notify the The storage destination bucket of the object of the client is stored.
  • the notification message may carry a bucket identifier of the destination bucket and an object identifier of the object.
  • the object storage server does not immediately notify the storage client of the result of the object migration. Rather, the storage client is notified of the result of the object migration by the storage client when the storage client actively queries the location information of the object.
  • the storage client may send an object query message to the object storage server at any time, and the object query message includes an object identifier of the object to be queried.
  • the object storage server receives the object query request of the storage client, check whether the object corresponding to the object identifier carried in the object query request has been migrated to another bucket, and if it has been migrated to another bucket, notify the The storage destination bucket of the object of the client is stored.
  • the notification message may carry a bucket identifier of the destination bucket and an object identifier of the object.
  • the storage client can actively query in advance, and can obtain the storage location of the object created and used by the storage client in advance, so that when the object is accessed subsequently, the access is directly accessed according to the queried location, thereby improving the access efficiency.
  • step 210 If the result of the determination in step 210 is that the first object in the source bucket does not satisfy the object migration policy, the first object is not migrated.
  • the object storage server may continue to determine whether other object migration policies are met, and only if the object migration policy is not met, If the first object in the source bucket does not satisfy the object migration policy of the source bucket, the first object is not migrated.
  • the first object satisfies more than one object migration policy
  • the migration according to the migration policy of the objects causes a conflict, for example, if priority is not set for each object migration policy Or, even if there is a conflict but there is a conflict, migration based on these object migration policies can lead to conflicts. Then this case is considered as an object migration policy in which the first object in the source bucket does not satisfy the source bucket.
  • the object storage server can determine whether the object migration process needs to be performed for different objects according to the pre-configured object migration policy, and does not need the storage client to perform object migration processing according to the manual operation instruction of the user, thereby improving
  • the efficiency of object migration can also solve the problem that users cannot automatically manage the object's belonging to the bucket.
  • objects that need to be managed in a centralized manner may be placed in a bucket in which a value-added function or service is configured, and other objects are used. (for example, objects that are accessed less frequently or less important) are placed in the source bucket configured with fewer value-added functions or services, thereby optimizing resource allocation and utilization, and achieving resource-saving effects accordingly.
  • Buckets configured with more value-added functions or services require more resources, such as network resources, processing resources, and/or storage resources.
  • the object storage server needs to migrate the policy according to the configuration object of the migration policy of the storage client, and then obtain the object migration corresponding to the source bucket according to the previously configured object migration policy.
  • Strategy For the processing flow of the specific configuration object migration policy, refer to Figure 3 below.
  • FIG. 3 shows a method for configuring an object migration policy provided in an embodiment of the present invention.
  • the method is applicable to the object storage service system 100 shown in FIG. 1 .
  • the method for configuring the object migration policy includes the following processing:
  • the storage client determines a migration policy parameter.
  • the object migration policy is composed of at least the following migration policy parameters: an identifier of the source bucket, a migration attribute of the object, an attribute value of the migration attribute, and the destination.
  • the migration attribute of the object includes: the importance degree of the object or the frequency at which the object is accessed; or in the case where a plurality of buckets are distributed in a plurality of different regions, the migration attribute of the object includes the object being a certain Region (for example, Region) 1) The frequency or frequency of access by the requester within the range, the certain Region is different from the Region to which the destination bucket belongs (for example, Region 2).
  • the storage client may determine the migration policy parameters of each bucket according to the configuration of each bucket created or used by the storage client, including determining the migration attribute of the object of each bucket and the corresponding attribute value.
  • the configuration of the bucket includes a value-added function or service configured by each bucket.
  • the value-added functions or services configured by the Bucket include at least one of the following: network acceleration, high data reliability, stored QoS, near access to data, or multi-version control of data.
  • Scenario 1 For example, the storage client establishes or uses two buckets, which are a first bucket and a second bucket, respectively, wherein the first bucket is configured with a higher level of storage QoS capability, for example, the first bucket configuration per second.
  • the number of transactions processed by the clock (Transactions Per Second, TPS) and the number of concurrent indicators are very high, and the number of external requests processed by the second bucket configuration (Transactions Per Second, TPS) and the number of concurrent indicators are not special.
  • the storage client may need to place objects with a high frequency of access in the first bucket as much as possible, and focus the objects with a small frequency of access into the second bucket.
  • resource allocation and utilization can be optimized, and the effect of saving resources can be achieved accordingly.
  • Buckets configured with more value-added functions or services require more resources, such as network resources, processing resources, and/or storage resources, so that higher fees are required, so corresponding savings can be made. More cost.
  • the storage client determines the accessed frequency of the object as the migration attribute, and the attribute value of the migration attribute, that is, the frequency of the object in the scene, may be accessed by the storage client according to the user's instruction, for example,
  • the commands determined by the command line or the user interface may also be determined by the storage client according to a specific scenario.
  • the migration policy parameter may be determined as: the source bucket identifier is the identifier of the second bucket, the frequency of the object being accessed, the frequency of the object being accessed is higher than 100 times/day, and the destination bucket identifier is the bucket identifier of the first bucket. Or determining the migration policy parameter as: the source bucket identifier is the identifier of the first bucket, the frequency of the object being accessed, the frequency of the object being accessed less than 100 times/day, and the destination bucket identifier being the bucket identifier of the second bucket.
  • Scenario 2 For example, the storage client establishes or uses two buckets, namely a first bucket and a second bucket, wherein the first bucket is configured with higher performance solid state storage as the underlying medium of the bucket, and the second bucket configuration. A lower performance general storage medium is used as the underlying medium for the bucket.
  • the storage client may need to place the object with a high frequency of access in the first bucket as much as possible, and place the object with a small frequency of access into the second bucket.
  • resource allocation and utilization can be optimized, and the effect of saving resources can be achieved accordingly.
  • the storage client determines the accessed frequency of the object as the migration attribute, and the attribute value of the migration attribute, that is, the frequency of the object in the scene, may be accessed by the storage client according to the user's instruction, for example,
  • the commands determined by the command line or the user interface may also be determined by the storage client according to a specific scenario.
  • the migration policy parameter may be determined as: the source bucket identifier is the identifier of the second bucket, the frequency of the object being accessed, the frequency of the object being accessed is higher than 100 times/day, and the destination bucket identifier is the bucket identifier of the first bucket. Or determining the migration policy parameter as: the source bucket identifier is the identifier of the first bucket, the frequency of the object being accessed, the frequency of the object being accessed less than 100 times/day, and the destination bucket identifier being the bucket identifier of the second bucket.
  • Scenario 3 For example, the storage client establishes or uses two buckets, which are a first bucket and a second bucket respectively, wherein the first bucket has a high configuration for data reliability, for example, a disaster tolerance requirement for data is relatively high. Requires remote disaster recovery; and the second bucket has no special requirements for data reliability, only to meet the general disaster tolerance conditions.
  • the storage client may need to place objects of higher importance as much as possible in the first bucket, and objects with lower importance into the second bucket.
  • resource allocation and utilization can be optimized, and the effect of saving resources can be achieved accordingly.
  • the storage client determines the importance level of the object as a migration attribute
  • the attribute value of the migration attribute may be determined by the storage client according to an instruction of the user, for example, a command input through a command line or a user interface, or may be stored by the storage client.
  • the client determines according to the specific scenario.
  • the attribute value may be set to the storage duration of the object, and the object exceeding a certain storage duration may be considered to be less important, thereby becoming unimportant.
  • the migration policy parameter may be determined as: the source bucket identifier is the identifier of the second bucket, the importance degree of the object, the storage duration of the object (for example, greater than or equal to 30 days), and the destination bucket identifier is the bucket identifier of the first bucket. .
  • determining the migration policy parameter as: the source bucket identifier is the identifier of the first bucket, the importance degree of the object, the storage duration of the object (for example, greater than or equal to 30 days), and the destination bucket identifier is the bucket identifier of the second bucket.
  • Scenario 4 For example, in order to solve the problem of nearby access, cloud storage vendors generally set up multiple regional Region-level data centers in different cities, and each Region can include multiple buckets. For example, Huawei Public Cloud deployed the first Region in Beijing and the second Region in Hongkong.
  • the same storage client can create or use buckets in multiple Regions. For example, the storage client creates two buckets, which are a first bucket and a second bucket, respectively, wherein the first bucket belongs to the first Region and the second bucket belongs to the second Region. If the object in the first bucket is often accessed by an access requester within the second Region. In order to better solve the problem of nearby access, it is necessary to migrate an object in the first bucket belonging to the first Region that is frequently accessed by the access requester in the second Region to the second bucket in the second Region. This can save network resources and improve access effectiveness.
  • the storage client determines the access frequency or frequency of the access requester in the second Region as the migration attribute.
  • the storage client may be instructed by the storage client according to the user.
  • the command input through the command line or the user interface may be determined by the storage client according to a specific scenario.
  • the attribute value can be set to: the ratio of the number of accesses of the object from the access requester within the second Region to the number of times the object is accessed by access requestors from all Regions (for frequency cases), or 100 times/day ( For the case of frequency).
  • the migration policy parameter can be determined as: the source bucket identifier is the identifier of the first bucket, the access frequency of the object by the access requester in the second Region, and the object is accessed by the access requester from the second Region.
  • the number of times is the ratio of the number of times the object is accessed by access requestors from all Regions (eg, greater than or equal to 20%), and the destination bucket is identified as the bucket identifier of the second bucket.
  • the source bucket identifier is the identifier of the first bucket, the access frequency of the object by the access requester in the second Region, and the frequency at which the object is accessed by the access requester from the second Region (eg, greater than Or equal to 100 times/day), and the destination bucket identifier is the bucket identifier of the second bucket.
  • scenario 4 is an example of determining the migration policy parameters after considering the configuration in each case:
  • the storage client establishes or uses two buckets, which are a first bucket and a second bucket, wherein the first bucket is configured with more value-added functions or services, such as network acceleration, high data reliability, and high Horizontal storage of Qos, data access, etc.
  • the second bucket is configured with fewer value-added features or services and even only basic functions and services.
  • the storage client may need to comprehensively consider the accessed frequency of the object and the importance of the object or even the object is accessed by the access requester within the certain Region to determine the frequency to be accessed.
  • the set attribute value, the set attribute value of the importance level of the object, and the object whose object is accessed by the access requester within the certain Region to access the set frequency value of the frequency or frequency are placed in the first bucket.
  • the object that does not meet the above conditions is placed in the second bucket, so that the configuration and utilization of resources can be optimized to achieve better resource saving effects.
  • the storage client may be based on other scenarios, such as network acceleration, storage performance, data access, or data.
  • For version control, etc. perform migration policy parameters similar to the above method. It is to be understood that the similar processing in other scenarios can be easily conceived by those skilled in the art on the basis of reading the embodiments of the present application, which are all within the scope of the present application.
  • the storage client can update the corresponding modification or update migration policy parameters according to the changes of the buckets created and used or the configuration of these buckets.
  • modification or update of the migration policy parameters refer to the above process for determining the migration policy parameters, which will not be described here.
  • the storage client requests a configuration of the migration policy from the object storage server after determining the migration policy parameter.
  • the migration policy configuration request message may be sent to the object storage server by using a newly added bucket configuration interface of the storage client and the object storage server.
  • the migration policy configuration request message can adopt the OBS protocol.
  • the migration policy configuration request message includes the migration policy parameters as shown in Table 1 below:
  • the storage client can update the corresponding modification or update migration policy parameters according to the change of the created or used bucket or the configuration of the buckets, and then send a request to the object storage server to notify the object storage server to modify or update before. Configuration of the object migration strategy.
  • the object migration policy may be configured in the object storage server or the object storage service system by other means, and those skilled in the art may Other configurations may be readily apparent upon reading the above-described embodiments, which are all within the scope of the present application.
  • the object storage server configures an object migration policy for the source bucket.
  • the object storage server performs configuration of the object migration policy according to the migration policy parameter included in the migration policy configuration request message. Specifically, the object storage server creates data in the metadata database (if there is no object migration policy for a certain bucket) or modifies (if a bucket already has an object migration policy), the specific examples are as shown in Tables 2 and 3 below. Show.
  • the object storage server configures an object migration policy for Bucket1.
  • the records created or modified in the metadata database are as shown in Table 2 below:
  • Table 2 Object Storage Policy for Object Storage Server Configuration for Bucket 1
  • the object storage server configures an object migration policy for Bucket 2, and the records created or modified in the metadata database are as shown in Table 3 below:
  • the object storage server notifies that the storage client object migration policy is configured successfully.
  • the object storage server can notify the storage client object migration policy configuration succeeding through the bucket configuration interface in the foregoing step 304, or notify the storage client object migration through other messages in the OBS protocol.
  • the policy configuration is successful.
  • the object storage server may start the steps in the foregoing embodiment shown in FIG. 2 in the following two situations or scenarios, namely, a foreground service startup scenario and a background timer startup.
  • a foreground service startup scenario and a background timer startup.
  • the scenario, the following two scenarios are described in Figure 4 and Figure 5 below.
  • FIG. 4 shows an actual application scenario of an object migration method provided in the embodiment of the present application.
  • the method can be applied to the object storage service system 100 shown in FIG. 1 and applied to the method flow shown in FIG. 2 and FIG. 3, as shown in FIG. 4, the object migration method includes the following processing:
  • the storage client sends an object access request to the object storage server by using the OBS protocol, and requests to access the first object of the first bucket created by the storage client, where the object access request may include the first bucket.
  • the bucket identifier and the identifier of the first object may include the first bucket.
  • the object storage server checks whether the corresponding bucket is configured with an object migration policy.
  • the object storage server after receiving the object access request sent by the storage client, the object storage server checks whether the bucket corresponding to the bucket identifier is configured with object migration according to the bucket identifier carried in the storage client.
  • the object storage server may query the object database to query the object migration policy corresponding to each bucket, such as querying whether the metadata database has the records as shown in Table 2 or Table 3 above. Object migration policy configuration records for different buckets.
  • the object storage server when the object access request is carried not by the first object, but other objects in the bucket, the object storage server is in accordance with the object identifier of the object. After determining that the object belongs to the bucket, it also checks whether the bucket is configured with an object migration policy.
  • the object migration processing for the object in the bucket is not performed.
  • the flow for example, does not execute the object migration processing flow in the embodiment corresponding to FIG. 2.
  • the object migration policy configuration corresponding to the bucket corresponding to the bucket identifier is recorded in the metadata database after the query, the object corresponding to the bucket is obtained according to the record.
  • the specific migration process refer to step 202 in the foregoing embodiment shown in FIG. 2, and details are not described herein again.
  • the subsequent object migration processing process is performed according to the first object migration policy. For details, refer to the processing steps 204-210 in the foregoing embodiment shown in FIG. 2, where No longer.
  • the first object migration policy may include the following migration policy parameters as shown in Table 2 above:
  • the source bucket identifier is: the identifier of the first bucket, that is, bucket 1;
  • the migration attribute of an object is: the frequency at which the object is accessed;
  • Attribute value less than 100 times / day;
  • the bucket identifier of the destination bucket is: the identifier of the second bucket, that is, bucket 2.
  • the migration policy parameter may further include a priority parameter of 1, indicating a first priority.
  • the operation of the subsequent object migration processing process according to the first object migration policy may include determining whether the first object in the first bucket is satisfied according to the identifier of the first object and the migration policy parameter of the first object migration policy.
  • the first object migration policy For details, refer to the processing step 204 in the foregoing embodiment shown in FIG. 2, and details are not described herein again.
  • the subsequent object migration processing process is performed according to the first object migration policy.
  • the second object migration policy may be the policy 2 as shown in Table 2 above, including the following migration policy parameters:
  • the source bucket identifier is: the identifier of the first bucket, that is, bucket 1;
  • the migration attribute of an object is: the importance of the object;
  • Attribute value the storage time of the object is greater than or equal to 30 days;
  • the bucket identifier of the destination bucket is: the identifier of the second bucket, that is, bucket 2.
  • the migration policy parameter may further include a priority parameter of 1, indicating a first priority.
  • the operation of the subsequent object migration processing process according to the second object migration policy may include determining whether the first object in the first bucket is satisfied according to the identifier of the first object and the migration policy parameter of the second object migration policy.
  • the second object migration policy For details, refer to the processing step 204 in the foregoing embodiment shown in FIG. 2, and details are not described herein again.
  • the processing of the objects refers to the above processing, which will not be repeated here.
  • the embodiment shown in FIG. 4 is an object migration processing process in a foreground service startup scenario, that is, an object migration process for a specific object (such as the first object in this embodiment) is started according to an object access service of the storage client. Process.
  • the object storage server may further expand the judgment range, that is, in addition to determining whether the first object satisfies the object migration policy corresponding to the first bucket, the first Whether other objects in the Bucket satisfy the object migration policy corresponding to the first bucket, and perform migration or not processing according to the judgment result.
  • the specific process is the same as above, and will not be repeated here.
  • the object migration processing flow can be started according to the activity level of the specific object.
  • the object migration processing flow of the object is also started frequently, and for an inactive object, the number of times the object migration processing flow of the object is started is reduced, so that the object migration process can be processed in a targeted manner. Can save processing resources.
  • the object migration processing process is started by the foreground service, if the first object requested by the storage client satisfies the object migration policy of the first bucket, the first object is performed. Before migrating to the destination bucket, returning the first object to the storage client according to the access request of the storage client.
  • FIG. 5 shows an actual application scenario of an object migration method provided in the embodiment of the present application.
  • the method is applicable to the object storage service system 100 shown in FIG. 1 above, and to the method flow shown in FIGS. 2 and 3.
  • the difference between the embodiment of the foreground service startup scenario shown in FIG. 4 is that, in this embodiment, the background timing startup scenario is performed. Once the background task is started, the object in the bucket managed by the object storage server is checked. 5, including the following processing:
  • the object storage server may set a startup task on the day after tomorrow, start the task periodically according to a certain period or a set period, and scan and check the object in the bucket managed or maintained by the storage server.
  • the object storage server checks, by the bucket, whether each bucket is configured with an object migration policy.
  • the object storage server checks each bucket managed by the object storage server by bucket, and determines whether each bucket is configured with an object migration policy. For example, the object storage server may query the object database to query the object migration policy corresponding to each bucket, such as querying whether the metadata database has the records as shown in Table 2 or Table 3 above. Object migration policy configuration records for different buckets.
  • object migration processing is not performed on the object in the bucket.
  • the object migration processing flow for the object in the bucket is not performed, for example, the map is not executed. 2 The object migration processing flow in the corresponding embodiment.
  • the object storage server determines that a certain bucket is configured with an object migration policy, further obtain an object migration policy corresponding to the bucket.
  • the object migration policy configuration corresponding to the bucket corresponding to the bucket identifier is recorded in the metadata database after the query, the object migration corresponding to the bucket is obtained according to the record.
  • the specific acquisition process reference may be made to step 202 in the foregoing embodiment shown in FIG. 2, and details are not described herein again.
  • the subsequent object migration processing flow is performed separately for each object in the bucket according to the first object migration policy.
  • the subsequent object migration processing flow is performed separately for each object in the bucket according to the first object migration policy.
  • the object storage server After the operations of the object migration process are performed on all the objects in the bucket, the object storage server further processes the objects in the buckets configured with the object migration policy. The specific process is the same as above, and is not described here. Until all objects of all buckets managed by the object storage server are executed.
  • the background timing startup scenario is an object migration processing process for the objects in the bucket managed by the object storage server
  • the migration processing of the completed object can be completed in time and in a complete manner, so that the migration can be accessed in time when the client accesses the object. After the object, without waiting for the user to access the object migration process, further optimizing the resources and utilization.
  • the background timing start scenario is to perform scan check and object migration processing on objects in all buckets managed by the object storage server in a specific time period, if a lot of buckets are involved, the resources may be occupied. Slightly larger.
  • the object storage server may set scan detection and object migration processing for a certain number of buckets instead of all, and at another time. Read other buckets in the segment for scan check and object migration processing, so as to reduce the impact on resource utilization or other services.
  • Figure 6-8 illustrates the processing flow of three different object migration strategies. They are: the first object migration strategy with the frequency of the object being accessed as the migration attribute ( Figure 6), the second object migration strategy of the migration attribute ( Figure 7), and the object being within a certain Region. The frequency of accessing the requester is the third object migration policy of the migration attribute ( Figure 8).
  • Figure 6-8 illustrates the processing flow of three different object migration strategies. They are: the first object migration strategy with the frequency of the object being accessed as the migration attribute ( Figure 6), the second object migration strategy of the migration attribute ( Figure 7), and the object being within a certain Region.
  • the frequency of accessing the requester is the third object migration policy of the migration attribute ( Figure 8).
  • FIG. 6 shows an object migration method provided in an embodiment of the present invention.
  • the method determines whether to perform object migration processing based on the accessed frequency of the object.
  • the method can be applied to the object storage service system 100 shown in FIG. 1 and applied to the method flow shown in FIG. 2, FIG. 3 and FIG. 4.
  • the method for configuring the object migration policy includes the following: deal with:
  • the storage client sends an object access request to the object storage server by using the OBS protocol, and requests to access the first object of the first bucket created by the storage client, where the object access request may include the first bucket.
  • the bucket identifier and the identifier of the first object For details, refer to step 402 above, and details are not described herein again.
  • the object storage server after receiving the object access request sent by the storage client, acquires an object migration policy corresponding to the first bucket, and determines that the object migration policy is an object.
  • the first object migration policy whose access frequency is the migration attribute of the object (refer to step 202, step 306, and step 408 above) may further determine whether to switch to a new statistical period.
  • the statistical period refers to a unit of the frequency at which the statistical object is accessed, for example, the frequency of the object being accessed in days or hours.
  • the new statistical period is used as the current statistical period, for example, the previous statistics in units of one hour are switched to one day.
  • the statistics are then counted according to the new statistical period for the number of times the first object was accessed before the current visit, and then the following step 608 is performed.
  • the statistics mentioned in the present application may be based on statistics within a certain period, or may be statistics on the entire life cycle of the object.
  • the number of accesses of the object in the current statistical period is increased by one.
  • the object storage server performs the accessed frequency statistics of the first object in a current statistical period. According to the above example, if the object storage server determines that it is not necessary to switch to a new statistical period, the frequency of access of the first object is directly counted in days, and the number of accesses is incremented by one according to the current access request of the storage client. If the object storage server determines that it is necessary to switch to the new statistical period, since the statistical period has been reset in step 606, in this step, the object storage server is directly based on the statistics of the above step 606 (based on the statistical period after the switching), according to The current access request of the storage client adds 1 to the number of accesses to obtain the updated number of times of accessing the first object.
  • steps 604 and 606 may also be performed without going to the step 602.
  • the object storage server obtains an attribute value of the migration attribute of the object according to the first object migration policy, for example, a frequency threshold for the object to be accessed.
  • the object storage server determines whether the number of times of accessing the updated first object is lower than an attribute value included in the first object migration policy, that is, a frequency threshold for the object in the above example to be accessed, such as less than 100 Days/times.
  • the object storage server migrates the first object to the first The destination bucket corresponding to the destination bucket identifier included in the object migration policy.
  • the object storage server may further determine whether the accessed number of the first object is higher than an attribute value included in an object migration policy, such as low. In the case of 100 days/time, and in the case that the number of times the first object is accessed is higher than the attribute value included in the certain object migration policy, the object migration process is performed; otherwise, the migration process is not performed, specifically Refer to the description of step 306 above. I will not repeat them here.
  • objects with high access frequency can be placed in the destination bucket with more value-added functions or services, and objects with small access frequency are concentrated to be configured with less value-added functions. Or in the source bucket of the service, thereby optimizing resource allocation and utilization, and correspondingly achieving resource saving effects.
  • Buckets configured with more value-added functions or services require more resources, such as network resources, processing resources, and/or storage resources.
  • FIG. 7 shows an object migration method provided in an embodiment of the present invention.
  • the method determines whether to perform object migration processing according to the importance of the object.
  • the method can be applied to the object storage service system 100 shown in FIG. 1 and applied to the method flow shown in FIG. 2, FIG. 3 and FIG. 4, as shown in FIG. 7, the method for configuring the object migration policy includes the following: deal with:
  • the storage client sends an object access request to the object storage server by using the OBS protocol, and requests to access the first object of the first bucket created by the storage client, where the object access request may include the first bucket.
  • the bucket identifier and the identifier of the first object For details, refer to step 402 above, and details are not described herein again.
  • the object storage server after receiving the object access request sent by the storage client, acquires an object migration policy corresponding to the first bucket, and determines that the object migration policy is an object.
  • the current time of the system is obtained by the second object migration policy of the migration attribute of the object (refer to step 202, step 306, and step 408 above), and the current time of the object storage service system, for example, T1.
  • the object storage server determines that the object migration policy is a second object migration policy that is a migration attribute of an object
  • the identifier of the first object included in the object access request is determined. Determining a time at which the first object is uploaded to the object storage service system, such as T2. Then, the object storage server subtracts the T1 from the T2 to obtain a difference between the current time of the system and the first object upload time, and determines the difference according to the attribute value included in the second object migration policy. Whether the value is greater than or equal to the attribute value, and the attribute value is, for example, a duration threshold in which the object is stored in the source bucket.
  • the object storage server migrates the first object to the destination included in the second object migration policy.
  • the Bucket identifies the destination bucket.
  • a data with a high degree of importance may become a less important data after a certain period of time, and may eventually become Archive data, then place objects of higher importance in source buckets with more value-added features or services, and lower-important objects into buckets with less value-added features or services.
  • resource allocation and utilization can be optimized, and the effect of saving resources can be achieved accordingly.
  • Buckets configured with more value-added functions or services require more resources, such as network resources, processing resources, and/or storage resources.
  • FIG. 8 shows an object migration method provided in an embodiment of the present invention.
  • the method determines whether to perform object migration processing based on the accessed frequency of the object.
  • the method can be applied to the object storage service system 100 shown in FIG. 1 and applied to the method flow shown in FIG. 2, FIG. 3 and FIG. 4.
  • the method for configuring the object migration policy includes the following: deal with:
  • the storage client sends an object access request to the object storage server by using the OBS protocol, and requests to access the first object of the first bucket created by the storage client, where the object access request may include the first bucket.
  • the bucket identifier and the identifier of the first object For details, refer to step 402 above, and details are not described herein again.
  • the object storage server after receiving the object access request sent by the storage client, acquires an object migration policy corresponding to the first bucket, and determines that the object migration policy is third.
  • the object migration policy, the migration attribute of the object included in the third object migration policy is the frequency of the object being accessed by the access requester in a certain region (for the specific process, refer to step 202, step 306, and step 408 above), and further determining whether to switch.
  • the certain Region is the Region to which the storage client belongs, and the Region to which the storage client belongs is different from the Region to which the first bucket belongs.
  • the statistical period refers to a unit of the frequency at which the statistical object is accessed, for example, the frequency of the object being accessed in days or hours.
  • the new statistical period is used as the current statistical period, for example, the previous statistics in units of one hour are switched to one day. And then counting, according to the new statistical period, the number of times the first object is accessed by the access requester in the Region to which the storage client belongs before the current access of the storage client, and then performing the following steps 808.
  • the object storage server also needs to count the number of times the object is accessed by the access requester in all Regions within the current statistical period.
  • the object storage server statistically counts the number of times the first object is accessed by an access requester from all Regions during the current statistical period.
  • the object storage server determines that there is no need to switch to a new statistical period, the number of times the first object is accessed by access requesters from all Regions is directly counted in days.
  • all Regions in this embodiment refer to each Region to which each access requester has accessed the object, and does not include one or more Regions, and accesses within the one or more Regions. The requester has not accessed the object as described.
  • the statistics mentioned in the present application may be based on statistics within a certain period, or may be statistics on the entire life cycle of the object.
  • the object storage server counts, in the current statistical period, the number of times the first object is accessed by an access requester from within the Region to which the storage client belongs. According to the above example, if the object storage server determines that there is no need to switch to a new statistical period, the number of times the first object is accessed by the access requester from within the Region to which the storage client belongs is directly counted in days, and The number of accesses is increased by one according to the current access request of the storage client.
  • the object storage server determines that it is necessary to switch to the new statistical period, since the statistics period has been reset in step 806, in this step, the object storage server is directly based on the statistics of the above step 806 (based on the statistical period after the switching), according to The current access request of the storage client adds 1 to the number of accesses to obtain the number of times the updated first object is accessed by an access requester from within the Region to which the storage client belongs.
  • steps 804 and 806 can also be skipped from step 802 to step 808 without performing.
  • the object storage server also needs to count the access requester in all the Regions in the current statistical period. The number of visits.
  • determining that the updated first object is accessed by an access requester from within the Region to which the storage client belongs is that the first object is accessed by an access requester in all Regions. Whether the proportion of the number of accesses exceeds a set attribute value, and the attribute value of the third object migration policy is that the object is within the Region (the Region to which the storage client belongs in the present embodiment)
  • the frequency threshold for accessing the requester's access for example 20%.
  • the object storage server determines that the updated first object is accessed by an access requester from within the Region to which the storage client belongs, the first object is owned by the first object. After the proportion of the number of access requester accesses in the Region exceeds the set attribute value (for example, 20%), the object storage server migrates the first object to the destination bucket identifier included in the third object migration policy. The corresponding destination bucket.
  • the access frequency of the access requester in a certain Region may be used as a criterion for determining whether to migrate.
  • scenario 4 and step 306 in step 302 in the foregoing embodiment shown in FIG. The Policy 4 in Table 2 and Table 3 will not be repeated here.
  • the object frequently accessed by the access requester in a certain Region is migrated to the Region, thereby better solving the problem of the near access, and further Can save network resources and improve access effectiveness.
  • FIG. 9 is a schematic structural diagram of an object storage server according to an embodiment of the present application.
  • the object storage server in the embodiment of the present application may be implemented by software.
  • the object storage server is a software module or a program, and may also be implemented by hardware.
  • the object storage server is An object storage server in a hardware form, such as various physical servers; and may also be implemented by a combination of software and hardware. Accordingly, the object storage server is installed to enable the object storage server in the embodiment described in the foregoing FIG. 1-8.
  • a software storage module of an executed operation or an object storage server of a hardware form of a computer program such as an object storage server with a processor built-in software module or computer program.
  • the object storage server may be applied to the foregoing embodiment shown in FIG. 1-8, the object storage server manages a plurality of bucket buckets, each of the buckets includes one or more objects, and the object storage server includes: The receiving module 902, the policy configuration module 904, the policy obtaining module 906, the policy determining module 908, the migration module 910, and the sending module 912.
  • the receiving module 902 is configured to receive a migration policy configuration request sent by the storage client, where the migration policy configuration request includes a bucket identifier of the source bucket corresponding to the object migration policy, a migration attribute of the object, and a location An attribute value of the migration attribute, and a bucket identifier of the destination bucket; wherein the migration attribute of the object includes an importance degree of the object or a frequency at which the object is accessed, or is distributed in the plurality of buckets In the case of Regions of different regions, the migration attribute of the object includes the frequency with which the object is accessed by an access requester within a Region, and the certain Region is different from the Region to which the destination bucket belongs.
  • the receiving module 902 For a detailed detailed processing function or a step that can be performed by the receiving module 902, reference may be made to the detailed description of step 304 in the foregoing embodiment shown in FIG.
  • the policy configuration module 904 is configured to configure an object migration policy for the source bucket according to the migration policy configuration request. For detailed detailed processing functions or steps that can be performed by the policy configuration module 904, reference may be made to the detailed description of step 306 in the embodiment shown in FIG. 3 above.
  • the policy acquisition module 906 is configured to acquire an object migration policy, where the object migration policy indicates a condition that an object migrates from a source bucket in the plurality of buckets to a destination bucket, where the object migration policy corresponds to the source bucket;
  • the policy judging module 908 is configured to determine whether the first object in the source bucket satisfies the object migration policy; the specific detailed processing function of the policy judging module 908 or the steps that can be performed may be implemented by referring to the foregoing FIG. 2 A detailed description of steps 204, 206, and 210 in the example, a detailed description of steps 604-610 in the embodiment shown in FIG. 6, a detailed description of steps 704-706 in the embodiment shown in FIG. 7, and FIG. A detailed description of steps 804-810 in the embodiment.
  • the migrating module 910 is configured to migrate the first object to the destination bucket if the first object satisfies the object migration policy; and the specific processing function of the migration module 910 may be performed.
  • steps reference may be made to the detailed description of step 206 in the embodiment shown in FIG. 2, the detailed description in steps 612-614 in the embodiment shown in FIG. 6, and the details of steps 708-710 in the embodiment shown in FIG. A detailed description of steps 812-814 in the embodiment shown in FIG. 8 is described.
  • the sending module 912 is configured to notify the storage client of the result of the object migration after the object migration succeeds; the specific detailed processing function or the steps that can be performed by the policy sending module 912 can refer to the implementation shown in FIG. 2 above. A detailed description of step 208 in the example and step 308 in the embodiment shown in FIG. 3 above.
  • module division in the object storage server shown in FIG. 9 and the interaction relationship between the modules are only an example. In specific implementation, the method may be based on the method embodiments shown in FIG. 1-8. Modules inside the object storage server are merged or re-partitioned according to related functions.
  • FIG. 10 is a schematic structural diagram of a storage client according to an embodiment of the present application.
  • the storage client in the embodiment of the present application may be implemented by software.
  • the storage client is a software module, such as a software development kit (SDK); or may be implemented by hardware, correspondingly
  • SDK software development kit
  • the storage client is a storage client in a hardware form, such as a personal PC, a mobile terminal, a fixed terminal device, a smart card, a chip, a circuit device, or a thin terminal; and can also be implemented by a combination of software components, and correspondingly, the storage
  • the client is a storage client, such as a built-in software module or computer program, in the form of a physical hardware in which a software module or computer program capable of implementing the operations performed by the storage client in the embodiment described above in the embodiments of Figures 1-8 is implemented.
  • Various storage clients with processors such as personal PCs with built-in software modules or computer programs, mobile terminals, fixed terminal devices,
  • the storage client may be applied in the embodiment shown in FIG. 1-8 above, the storage client uses an object storage server to manage at least another bucket in a plurality of bucket buckets, each of the buckets including one or more
  • the storage client includes: a migration policy determination module 1002, a sending module 1004, and a receiving module 1006.
  • the migration policy determination module 1002 is configured to determine a migration policy parameter, where the policy parameter includes a bucket identifier of a source bucket corresponding to the object migration policy, a migration attribute of the object, an attribute value of the migration attribute, and a destination bucket. Bucket logo.
  • the sending module 1004 is configured to send a migration policy configuration request to the object storage server to request the object storage server to perform an object migration policy configuration on the source bucket, where the migration policy configuration request includes the migration policy parameter
  • the object migration policy represents a condition for the object to migrate from the source bucket to the destination bucket; the migration attribute of the object includes an importance degree of the object or a frequency at which the object is accessed, or is distributed in the plurality of buckets In the case of a plurality of different regions, the migration attribute of the object includes the frequency with which the object is accessed by an access requester within a certain Region, the certain Region being different from the Region to which the destination bucket belongs.
  • the receiving module 1006 is configured to receive a response message that the object migration policy returned by the object storage server is successfully configured.
  • the receiving module 1006 may be further configured to receive a notification message sent by the object storage server, to notify the storage client of the result of the object migration, which may be specifically referred to in the foregoing embodiment shown in FIG. 2 A detailed description of step 208.
  • the migration policy determination module 1002 may determine the migration policy parameter for the source bucket according to the configuration of each bucket used by the storage client, where the configuration of the bucket includes The value-added function or service of each bucket; the value-added function or service of the bucket includes at least one of the following: network acceleration, high data reliability, stored QoS, stored performance, nearby access of data, or multiple versions of data control.
  • the storage client further includes a bucket configuration interface 1008, and the sending module 1006 sends the migration policy configuration request to the object storage server through the bucket configuration interface 1008.
  • the device embodiments described above are merely illustrative.
  • the division of the modules is only a logical function division, and the actual implementation may have another division manner, such as multiple modules, units, or Components can be combined or one of the modules can be further divided into different functional modules.
  • the function of the policy configuration module in the object storage server in the above embodiment may also be merged into the policy acquisition module, or the policy configuration module, the policy acquisition module, and the policy determination module may be combined in one module.
  • the coupling or communication connection between the modules or devices shown or described in the figures may be indirect coupling or communication connections through some interfaces, devices or units, or electrical, mechanical or otherwise. The form of coupling or connection.
  • Modules in which the components are described as separate components may be physically separate or physically located in the same physical component.
  • the component named by the module may be a hardware unit, a software module or a logic unit, or a combination of hardware and software.
  • the module may be located in one network element or may be distributed to multiple network elements. These may select some or all of the units according to actual needs to achieve the purpose of the solution of the embodiment.
  • FIG. 11 is a schematic structural diagram of an object storage server 1100 involved in the embodiment of the present application.
  • the object storage server 1100 can be applied in the embodiment shown in Figures 1-8 above.
  • the functions or operational steps of the object storage server are implemented by one or more processors in a general purpose computer or server by executing program code in the memory.
  • the object storage server 1100 includes a transceiver 1110, a processor 1120, a random access memory 1140, a read only memory 1150, and a bus 1160.
  • the processor 1120 is coupled to the receiver 1110, the random access memory 1140, and the read only memory 1150 via the bus 1160.
  • the processor 1120 can be a general purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling the execution of the program of the present invention.
  • CPU central processing unit
  • ASIC application-specific integrated circuit
  • Bus 1160 can include a path for communicating information between the components described above.
  • the transceiver 1110 is configured to communicate with other devices or communication networks, such as an Ethernet, a radio access network (RAN), a wireless local area network (WLAN), etc., in the embodiment of the present invention, the transceiver 1110 can be used for Communicate with the storage client.
  • RAN radio access network
  • WLAN wireless local area network
  • the random access memory 1140 can load an application code that implements the object storage server in the embodiment shown in Figs. 2-8, and is controlled by the processor 1120 for execution.
  • booting is performed by the boot loader boot system in the basic input/output system or the embedded system in the read-only memory 1150, and the boot object storage server 1100 is put into a normal operation state.
  • the processor 1120 runs the application and the operating system in the random access memory 1140, so that the object storage server 1100 can perform the functions and operations in the embodiment shown in FIGS. 2-8.
  • the interaction with the storage client is completed by the transceiver 1110 under the control of the processor 1120, and the internal processing of the object storage server 1100 (for example, configuration of an object migration policy, acquisition of an object migration policy, determination of an object migration policy, And the migration processing of the object, etc.) is performed by the processor 1120.
  • the object storage server 1100 for example, configuration of an object migration policy, acquisition of an object migration policy, determination of an object migration policy, And the migration processing of the object, etc.
  • the present embodiment may also virtualize the virtual object implemented by the NFV technology based on the physical server combined with the network function.
  • a storage server which may be a virtual network device or a server.
  • Those skilled in the art can use the NFV technology to virtualize a plurality of object storage servers having the above functions on a physical server by reading the present application. I will not repeat them here.
  • FIG. 12 is a schematic structural diagram of a storage client 1200 involved in the embodiment of the present application.
  • the storage client 1200 can be applied in the embodiment shown in Figures 1-8 above.
  • the functions or operational steps of the storage client 1200 are implemented by one or more processors in a general purpose computer or server by executing program code in memory.
  • the pair of storage clients 1200 can include a transceiver 1210, a processor 1220, a random access memory 1240, a read only memory 1250, and a bus 1260.
  • the processor 1220 is coupled to the transceiver 1210, the random access memory 1240, and the read only memory 1250 via a bus 1260.
  • the processor 1220 can be a general purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling the execution of the program of the present invention.
  • CPU central processing unit
  • ASIC application-specific integrated circuit
  • Bus 1260 can include a path for communicating information between the components described above.
  • the transceiver 1210 is configured to communicate with other devices or communication networks, such as an Ethernet, a radio access network (RAN), a wireless local area network (WLAN), etc., in the embodiment of the present invention, the transceiver 1210 can be used for Communicate with the object storage server.
  • RAN radio access network
  • WLAN wireless local area network
  • the random access memory 1240 can load application code that implements the storage client in the embodiment shown in FIGS. 2-8, and is controlled by the processor 1220 for execution.
  • booting is initiated by the boot loader boot system in the basic input/output system or embedded system in the read-only memory 1250 to boot the storage client 1200 into a normal operating state.
  • the processor 1220 runs the application and operating system in the random access memory 1240, such that the storage client 1200 can perform the functions and operations in the embodiment shown in FIGS.
  • the interaction with the object storage server is completed by the transceiver 1210 under the control of the processor 1220.
  • the internal processing of the storage client 1200 eg, determining migration policy parameters, generating a migration policy configuration request message, etc. is completed by the processor 1220.
  • the present embodiment may also virtualize the virtual storage implemented by the NFV technology based on a physical server combined with a network function.
  • the client, the virtual storage client can be a virtual network device or a computer.
  • Those skilled in the art can use the NFV technology to virtualize a plurality of storage clients having the above functions on a physical server by reading the present application. This is not repeated here.
  • the embodiment of the present invention further provides a computer storage medium for storing computer software instructions used by the object storage server, which includes functions for executing the object storage server in the embodiment shown in FIG. 2-8. The procedures involved.
  • the embodiment of the present invention further provides a computer storage medium for storing computer software instructions used by the storage client, which includes functions for performing the storage client function in the embodiment shown in FIG. 2-8. program of.
  • embodiments of the present application can be provided as a method, apparatus (device), or computer program product. Therefore, the embodiments of the present application may take the form of a hardware embodiment, a software embodiment, or a combination of software and hardware. Moreover, the application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) including computer usable program code.
  • the computer program is stored/distributed in a suitable medium, provided with other hardware or as part of the hardware, or in other distributed forms, such as over the Internet or other wired or wireless telecommunication systems.
  • the computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture comprising the instruction device.
  • the apparatus implements the functions specified in one or more blocks of a flow or a flow and/or block diagram of the flowchart.
  • These computer program instructions can also be loaded onto a computer or other programmable data processing device such that a series of operational steps are performed on a computer or other programmable device to produce computer-implemented processing for execution on a computer or other programmable device.
  • the instructions provide steps for implementing the functions specified in one or more of the flow or in a block or blocks of a flow diagram.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Software Systems (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

本发明实施例涉及一种对象迁移方法,该方法应用于包括对象存储服务器和存储客户端的存储系统,该对象存储服务器管理多个桶Bucket,每个所述Bucket包括一或多个对象,该存储客户端使用所述多个Bucket中的至少两个Bucket。在该方法中,对象存储服务器获取源Bucket的对象迁移策略,所述对象迁移策略表示对象从所述多个Bucket中的源Bucket迁移至目的Bucket的条件,对象存储服务器在确定该源Bucket中的第一对象满足该源Bucket的对象迁移策略的情况下,根据该对象迁移策略将该第一对象迁移至所述目的Bucket。从而可以自动地在不同的Bucket之间进行对象的迁移,提高了对象迁移的效率。

Description

一种对象迁移的方法、设备和系统 技术领域
本发明实施例涉及存储技术领域,特别涉及一种对象存储服务的方法、设备和系统。
背景技术
对象存储服务(Object Storage Service,OBS)是一个基于对象的海量存储服务,为客户提供海量、安全、高可靠、低成本的数据存储能力。OBS是一项面向Internet访问的服务,提供了基于HTTP/HTTPS协议的Web服务接口,用户可以随时随地连接至Internet的电脑上,通过客户端或者软件开发工具包(Software Development Kit,SDK)访问和管理存储在OBS中的数据。在OBS场景下,用户文件以对象的形式在云上进行存储,存储的载体命名成桶(Bucket),用户可以通过创建、修改、维护或删除Bucket,以及通过上传、下载、删除Bucket中的对象等操作来管理自己数据。一般情况下,系统高级功能、增值服务等都是配置在Bucket上,以Bucket为粒度提供高级功能和增值服务。但在现有技术下,如果要对对象进行跨Bucket的迁移管理,只能根据实际业务情况由用户人工地在不同的Bucket之间迁移对象,而且有些业务场合用户无法感知对象的操作情况,例如被访问的频率,从而无法根据该对象的操作情况完成跨Bucket的人工迁移。
发明内容
本申请实施例提供了一种对象存储服务的方法、设备和系统和存储介质,可以自动地在不同的Bucket之间进行对象的迁移,提高了对象迁移的效率。
第一方面,本申请实施例提供了一种对象迁移方法,该方法应用于对象存储服务系统,所述对象存储服务系统包括对象存储服务器和存储客户端,该对象存储服务器管理多个桶Bucket,每个所述Bucket包括一个或多个对象,该存储客户端使用所述多个Bucket中的至少两个Bucket。该对象迁移方法由该对象存储服务器执行。在该方法中,对象存储服务器获取源Bucket的对象迁移策略,所述对象迁移策略表示对象从所述多个Bucket中的源Bucket迁移至目的Bucket的条件,对象存储服务器判断该源Bucket中的第一对象是否满足该源Bucket的对象迁移策略,如果满足,则根据该对象迁移策略将该第一对象迁移至所述目的Bucket。
通过本实施中的处理,对象存储服务器可以根据预先配置的对象迁移策略针对不同的对象判断是否需要进行对象迁移处理,并将满足对象迁移策略的对象迁移到目的Bucket,而不需要存储客户端根据用户的手动操作指示进行对象迁移处理,提高了对象迁移效率。
在一种可能的实现方式中,所述对象存储服务器接收存储客户端发送的携带迁移策略参数的迁移策略配置请求,该迁移策略参数包括所述对象迁移策略所对应的所述源Bucket的Bucket标识、对象的迁移属性、所述迁移属性的属性值、以及所述目的Bucket的Bucket标识。其中,该对象的迁移属性可以是对象的重要程度,或者也可以是对象被访问的频率,或者在所述多个Bucket分布在多个不同地区Region的场合下,该对象的迁移属性还可以是对象被某一Region内的访问请求者访问的频度,所述某一Region不同于所述目的Bucket所归属的Region。所述对象存储服务器收到该迁移策略配置请求后,根据该迁移策略参数对该源Bucket进行对象迁移策略的配置。
通过本实施例,可以将需要重点管理的对象(例如,被访问频率大或者重要程度较高的对象)都放置在配置了较多的增值功能或服务的目的Bucket中,而将其他对象(例如,被访问频率小或者重要程度较低的对象)集中放置到配置了较少的增值功能或服务的所述的源Bucket中,从而可以优化资源配置和利用,并相应达到节约资源的效果。因为配置了较多的增值功能或服务的Bucket的需要更多的资源的支持,如网络资源、处理资源,和/或存储资源等。
在一种可能的实现方式中,所述迁移策略配置请求还包括优先级参数,该优先级参数与所述对象的迁移属性相关联。该优先级参数用于在所述源Bucket配置有多个对象迁移策略且所述第一对象分别满足所述多个对象迁移策略的情况下,所述对象存储服务器根据所述优先级参数选择所述源Bucket的对象迁移策略。
通过对同一Bucket配置的不同对象迁移策略设置优先级,从而可以针对同一个Bucket设置多个对象迁移策略而不会产生冲突,进而还可以针对不同Bucket的对象归属管理进行更加精细化管理,并进一步优化资源配置和利用以及节约资源。
在一种可能的实现方式中,该对象存储服务器还接收所述存储客户端的对象访问请求,请求访问所述源Bucket中的对象,根据该对象访问请求检查所述源Bucket是否配置了所述对象迁移策略;并在确定所述源Bucket配置了所述对象迁移策略后,读取所述对象迁移策略。
本实施例提供了一种前台业务启动方式来启动上述实施例中的对象迁移处理操作。例如通过存储客户端发送具体的对象访问的方式来启动上述实施例中的对象迁移处理操作。该启动方式主要是针对特定的对象或者特定的Bucket中的对象进行对象迁移处理流程,而不会广泛地检查该对象存储服务器所管理的所有Bucket中的对象,并对所有满足对象迁移策略的对象进行迁移处理,处理效率相对较高,持续时间相对较短,对其他业务的影响较少。而且根据业务启动可以根据特定对象的活跃程度启动对象迁移处理流程,对于活跃的对象,则该对象的对象迁移处理流程的启动也频繁些,而对于不活跃的对象,则该对象的对象迁移处理流程的启动次数就会相应减少,从而可以有针对性的进行对象迁移流程的处理,进而可以节省处理资源。
在一种可能的实现方式中,所述对象存储服务器根据预设的时间定期检查所述对象存储服务器管理的各所述Bucket是否分别配置了所述对象迁移策略,并在确定所述源Bucket配置了所述对象迁移策略后,读取所述的对象迁移策略。
本实施例提供了后台定时启动上述实施例中的对象迁移处理操作,例如根据预设的时间定期启动,由于后台定时启动场景是针对该对象存储服务器所管理的Bucket中的对象进行对象迁移处理流程,可以比较及时和完整的完成对象的迁移处理,从而在存储客户端访问对象时,可以及时地访问迁移后的对象,而不用等到用户访问之后再进行对象迁移处理,进一步优化了资源和利用。
此外,由于后台定时启动场景是在一个特定时间段内针对该对象存储服务器所管理的所有Bucket中的对象进行扫描检查和对象迁移处理,如果涉及到的Bucket很多,则可能会对资源的占用稍微大些。为了不对资源的利用造成较大影响或对其他业务造成较大影响,对象存储服务器可以设定在不同时间段内针对一定数量而不是全部的Bucket进行扫描检查和对象迁移处理,而在另一时间段内读其他的Bucket进行扫描检查和对象迁移处理,从而可以减轻对资源的利用造成的影响或对其他业务造成的影响。
在一种可能的实现方式中,当所述对象的迁移属性为所述对象的重要程度时,所述迁移属性的属性值为所述对象存储在所述源Bucket中的时长。这种情况下,所述对象存储服务器 先获取所述对象存储服务器的当前时间,然后比较所述对象存储服务器的当前时间与所述第一对象上传至所述源Bucket的时间的时间差值,并在确定所述时间差值大于或等于所述迁移属性的属性值的情况下,将所述第一对象迁移至所述目的Bucket中。根据对象的生命周期确定对象的重要程度,例如,一份重要程度高的数据,经过一定的时间后可能会变成重要程度低的数据,最终可能会变成归档数据,然后将重要程度比较高的对象都放置在配置了较多的增值功能或服务的源Bucket中,而将重要程度较低的对象放置到配置了较少的增值功能或服务的目的Bucket中,从而可以优化资源配置和利用,并相应达到节约资源的效果。因为配置了较多的增值功能或服务的Bucket的需要更多的资源的支持,如网络资源、处理资源,和/或存储资源等。
在一种可能的实现方式中,当所述对象的迁移属性为所述对象被访问的频率时,所述迁移属性的属性值为所述对象被访问的频率值。这种情况下,所述对象存储服务器先统计所述源Bucket中的所述第一对象的被访问频率,并在确定所述第一对象的被访问频率大于或等于所述迁移属性的属性值后将所述第一对象迁移至所述目的Bucket中。
在一种可能的实现方式中,当所述对象的迁移属性为所述对象被访问的频率时,所述迁移属性的属性值为所述对象被访问的频率值。这种情况下,所述对象存储服务器先统计所述源Bucket中的所述第一对象的被访问频率,并在确定所述第一对象的被访问频率小于或等于所述迁移属性的属性值后将所述第一对象迁移至所述目的Bucket中。
通过将被访问频率大的对象都放置在配置了较多的增值功能或服务的目的Bucket中,而将被访问频率小的对象集中放置到配置了较少的增值功能或服务的所述的源Bucket中,从而可以优化资源配置和利用,并相应达到节约资源的效果。因为配置了较多的增值功能或服务的Bucket的需要更多的资源的支持,如网络资源、处理资源,和/或存储资源等。
在一种可能的实现方式中,当所述对象的迁移属性为所述对象的重要程度以及所述对象被访问的频率时,所述迁移属性的属性值分别为所述对象存储在所述源Bucket中的时长阈值以及所述对象被访问的频率阈值。这种情况下,所述对象存储服务器获取所述对象存储服务器的当前时间,并比较所述对象存储服务器的当前时间与所述第一对象上传至所述源Bucket的时间的时间差值。此外,所述对象存储服务器还需统计所述源Bucket中的所述第一对象的被访问频率。然后,所述对象存储服务器在确定所述时间差值大于或等于所述对象存储在所述源Bucket中的时长阈值且所述第一对象的被访问频率大于或等于所述对象被访问的频率阈值后将所述第一对象迁移至所述目的Bucket中。
通过综合对象的被访问频率以及对象的重要程度作为对象迁移的标准,将被访问频率大以及重要程度较高的对象都放置在配置了较多的增值功能或服务的目的Bucket中,而将被访问频率小以及重要程度较低的对象集中放置到配置了较少的增值功能或服务的所述的源Bucket中,从而进一步精细化管理对象归属的bucket问题,进一步优化资源配置和利用以及节约资源。
在一种可能的实现方式中,当所述对象的迁移属性为对象被某一Region内的访问请求者访问的频度,所述迁移属性的属性值为所述对象被所述某一Region内的访问请求者访问的频度阈值。在这种情况下,所述对象存储服务器统计所述源Bucket中的所述第一对象的被所述某一Region内的访问请求者访问的次数,以及所述源Bucket中的所述第一对象的被多个不同Region内的访问请求者访问的次数。然后,比较所述源Bucket中的所述第一对象的被所述某一Region内的访问请求者访问的次数与所述源Bucket中的所述第一对象的被多个不同Region内的访问请求者访问的次数获得比值,并在确定所述比值大于或等于所述迁移属性的 属性值后将所述第一对象迁移至所述目的Bucket中,所述目的Bucket所归属的Region为所述某一Region。
在一种可能的实现方式中,当所述对象的迁移属性为对象被某一Region内的访问请求者访问的频率,所述迁移属性的属性值为所述对象被所述某一Region内的访问请求者访问的频率阈值。这种情况下,所述对象存储服务器统计所述源Bucket中的所述第一对象的被所述某一Region内的访问请求者访问的访问频率,并在确定所述源Bucket中的所述第一对象的被所述某一Region内的访问请求者访问的访问频率大于或等于所述迁移属性的属性值后将所述第一对象迁移至所述目的Bucket中,所述目的Bucket所归属的Region为所述某一Region。
根据对象被某一Region内访问请求者的访问频度,将被某一Region内的访问请求者的频繁访问的对象迁移至该Region,从更好地解决就近访问的问题,进而可以节省网络资源以及提高访问效力。
在一种可能的实现方式中,所述对象存储服务器还进一步确定是否切换统计周期,并在确定需要切换成新的统计周期之后,以所述新的统计周期统计前述实施例中的所述源Bucket中的所述第一对象的被访问频率或者对象被某一Region内的访问请求者访问的频率。
通过根据具体情况适时地更新统计周期,并根据新的统计周期统计上述实施例中的相关统计操作,从而能更加动态及灵活地进行资源配置。
在一种可能的实现方式中,在以存储客户端的对象访问请求为启动对象迁移处理的场景下,所述对象存储服务器根据所述存储客户端的对象访问请求,将所述源Bucket中的所述第一对象的被访问次数加1后获得所述源Bucket中的所述第一对象的当前被访问次数。然后,所述对象存储服务器根据所述源Bucket中的所述第一对象的当前被访问次数统计所述源Bucket中的所述第一对象的被访问频率。
通过将本次访问请求也纳入统计范围,从而使得统计更精确,更能准确地反映对象的被访问情况,进而使得对象归属Bucket的管理更加精细化,资源配置和利用更加优化。
在一种可能的实现方式中,所述对象存储服务器在将所述第一对象迁移至所述目的Bucket中后,主动通知存储客户端;或者在收到所述存储客户端发送的访问所述第一对象的对象访问请求后,通知所述第一对象迁移至所述目的Bucket中;或者在收到所述存储客户端发送的查询所述第一对象的对象查询请求后,通知所述第一对象迁移至所述目的Bucket中。
通过对象存储服务器的主动通知或者该存储客户端提前主动查询,可以提前获知该存储客户端所创建和使用的对象的存储位置,从而可以在后续访问对象时,直接根据查询到的位置进行访问,提高了访问效率。
第二方面,本申请实施例提供了一种对象存储服务的方法,该方法应用于对象存储服务系统,所述对象存储服务系统包括对象存储服务器和存储客户端,所述对象存储服务器管理多个桶Bucket,每个所述Bucket包括一个或多个对象,该多个Bucket中包括了存储客户端所创建或使用的Bucket。该方法由存储客户端执行,在该方法中,该存储客户端确定迁移策略参数,所述迁移策略参数包括对象迁移策略所对应的源Bucket的标识、对象的迁移属性、所述迁移属性的属性值、以及目的Bucket的标识。然后,该存储客户端向所述对象存储服务器发送携带所述迁移策略参数的迁移策略配置请求,请求所述对象存储服务器对所述源Bucket配置对象迁移策略。该对象迁移策略表示对象从所述源Bucket迁移至目的Bucket的条件。
本实施中,存储客户端通过向对象存储服务器发送携带迁移策略参数的迁移策略配置请求,可以使得对象存储服务器对存储客户端创建或使用的Bucket配置相应的对象迁移策略。 这样对象存储服务器就可以根据配置的对象迁移策略针对不同Bucket中的不同对象判断是否需要进行对象迁移处理,并将满足对象迁移策略的对象迁移到目的Bucket,而不需要存储客户端根据用户的手动操作指示进行对象迁移处理,提高了对象迁移效率。
在一种可能的实现方式中,所述对象的迁移属性可以为所述对象的重要程度;或者也可以为所述对象被访问的频率;或者,在所述多个Bucket分布在多个不同地区Region的情况下,所述对象的迁移属性还可以为对象被某一Region内的访问请求者访问的频度,其中所述某一Region不同于所述目的Bucket所归属的Region。
存储客户端通过携带不同的迁移策略参数,可以将需要重点管理的对象(例如,被访问频率大或者重要程度较高的对象)都放置在配置了较多的增值功能或服务的目的Bucket中,而将其他对象(例如,被访问频率小或者重要程度较低的对象)集中放置到配置了较少的增值功能或服务的所述的源Bucket中,从而可以优化资源配置和利用,并相应达到节约资源的效果。因为配置了较多的增值功能或服务的Bucket的需要更多的资源的支持,如网络资源、处理资源,和/或存储资源等。
在一种可能的实现方式中,所述迁移策略配置请求还包括优先级参数,该优先级参数与所述对象的迁移属性相关联。该优先级参数用于在所述源Bucket配置有多个对象迁移策略且某一对象分别满足所述多个对象迁移策略的场景下,所述对象存储服务器根据所述优先级参数选择所述源Bucket的对象迁移策略。
通过对同一Bucket配置的不同对象迁移策略设置优先级,从而可以针对同一个Bucket设置多个对象迁移策略而不会产生冲突,进而还可以针对不同Bucket的对象归属管理进行更加精细化管理,并进一步优化资源配置和利用以及节约资源。
在一种可能的实现方式中,所述存储客户端根据该存储客户端所创建或使用的各Bucket的配置情况,针对所述各Bucket,包括所述源Bucket,确定所述迁移策略参数。所述Bucket的配置情况包括所述各Bucket的增值功能或服务。所述Bucket的增值功能或服务包括至少下述之一:网络加速、数据高可靠性、存储的Qos、存储性能、数据的就近访问或数据的多版本控制。
存储客户端根据各Bucket的配置情况确定各Bucket的迁移策略参数,从而可以配置较佳的对象迁移策略,将对象在具有不同等级的增值功能或服务的Bucket之间迁移,可以对对象的Bucket归属进行更加精细化的管理,进而达到资源的优化配置和利用。
在一种可能的实现方式中,所述存储客户端根据所述Bucket的增值功能或服务的变化更新所述迁移策略参数,并向所述对象存储服务器发送迁移策略配置更新请求以请求所述对象存储服务器对所述源Bucket配置的对象迁移策略进行更新
存储客户端根据该存储客户端所创建和使用的Bucket的配置情况的变化适时地向对象存储服务器更新迁移策略参数,使其相应地更新和调整对象迁移策略,从而使得,对象的Bucket归属管理以及资源的配置和利用更加动态和灵活。
在一种可能的实现方式中,所述存储客户端通过所述存储客户端与所述对象存储服务器之间的Bucket配置接口向所述对象存储服务器发送所述迁移策略配置请求。
在一种可能的实现方式中,所述存储客户端接收所述对象存储服务器发送的通知,所述通知告知所述存储客户端所述源Bucket中的第一对象以被迁移至所述目的Bucket中。
在一种可能的实现方式中,所述存储客户端向所述对象存储服务器查询所述源Bucket中的对象迁移情况。
第三方面,本申请实施例提供了一种硬件形态的对象存储服务器,该对象存储服务器管 理多个桶Bucket,每个所述Bucket包括一个或多个对象。所述对象存储服务器包括策略获取模块、策略判断模块以及迁移模块。其中,该策略获取模块用于获取源Bucket的对象迁移策略,所述对象迁移策略表示对象从所述源Bucket迁移至目的Bucket的条件。该策略判断模块用于确定所述源Bucket中的第一对象是否满足所述策略获取模块所获取的该对象迁移策略。该迁移模块用于在所述策略判断模块确定所述第一对象满足所述对象迁移策略的情况下,将所述第一对象迁移至所述目的Bucket中。
通过本实施中的处理,对象存储服务器可以根据预先配置的对象迁移策略针对不同的对象判断是否需要进行对象迁移处理,并将满足对象迁移策略的对象迁移到目的Bucket,而不需要存储客户端根据用户的手动操作指示进行对象迁移处理,提高了对象迁移效率。
在一种可能的实现方式中,所述对象存储服务器还包括接收模块和策略配置模块。
该接收模块用于接收存储客户端发送的携带迁移策略参数的迁移策略配置请求,所述迁移策略参数包括所述源Bucket的标识、对象的迁移属性、所述迁移属性的属性值、以及所述目的Bucket的标识。其中所述对象的迁移属性可以为所述对象的重要程度,或者也可以为所述对象被访问的频率,或者在所述多个Bucket分布在多个不同地区Region的情况下,所述对象的迁移属性还可以为对象被某一Region内的访问请求者访问的频度,所述某一Region不同于所述目的Bucket所归属的Region。该策略配置模块,用于根据所述的迁移策略配置请求对所述源Bucket配置对象迁移策略。
通过本实施例,可以将需要重点管理的对象(例如,被访问频率大或者重要程度较高的对象)都放置在配置了较多的增值功能或服务的目的Bucket中,而将其他对象(例如,被访问频率小或者重要程度较低的对象)集中放置到配置了较少的增值功能或服务的所述的源Bucket中,从而可以优化资源配置和利用,并相应达到节约资源的效果。因为配置了较多的增值功能或服务的Bucket的需要更多的资源的支持,如网络资源、处理资源,和/或存储资源等。
第四方面,本申请实施例提供了一种硬件形态的存储客户端,该存储客户端使用对象存储服务器管理多个桶Bucket中的至少另个Bucket,每个所述Bucket包括一个或多个对象。该存储客户端包括迁移策略确定模块、发送模块。其中,该迁移策略确定模块用于确定迁移策略参数,所述迁移策略参数包括对象迁移策略所对应的源Bucket的标识、对象的迁移属性、所述迁移属性的属性值、以及目的Bucket的标识。该发送模块用于向所述对象存储服务器发送携带该迁移策略参数的迁移策略配置请求,请求所述对象存储服务器对所述源Bucket进行对象迁移策略的配置,所述对象迁移策略表示对象从所述源Bucket迁移至目的Bucket的条件。
本实施中,存储客户端通过向对象存储服务器发送携带迁移策略参数的迁移策略配置请求,可以使得对象存储服务器对存储客户端创建或使用的Bucket配置相应的对象迁移策略。这样对象存储服务器就可以根据配置的对象迁移策略,针对不同Bucket中的不同对象判断是否需要进行对象迁移处理,并将满足对象迁移策略的对象迁移到目的Bucket,而不需要存储客户端根据用户的手动操作指示进行对象迁移处理,提高了对象迁移效率。
一种可能的实现方式中,所述对象的迁移属性可以为所述对象的重要程度,或者也可以为所述对象被访问的频率;或者,在所述多个Bucket分布在多个不同地区Region的情况下,所述对象的迁移属性还可以为对象被某一Region内的访问请求者访问的频度,所述某一Region不同于所述目的Bucket所归属的Region。
存储客户端通过携带不同的迁移策略参数,可以将需要重点管理的对象(例如,被访问 频率大或者重要程度较高的对象)都放置在配置了较多的增值功能或服务的目的Bucket中,而将其他对象(例如,被访问频率小或者重要程度较低的对象)集中放置到配置了较少的增值功能或服务的所述的源Bucket中,从而可以优化资源配置和利用,并相应达到节约资源的效果。因为配置了较多的增值功能或服务的Bucket的需要更多的资源的支持,如网络资源、处理资源,和/或存储资源等。
一种可能的实现方式中,所述迁移策略确定模块根据所述存储客户端所使用的各Bucket的配置情况,针对所述源Bucket确定所述迁移策略参数。其中,所述Bucket的配置情况包括所述各Bucket的增值功能或服务,所述Bucket的增值功能或服务包括至少下述之一:网络加速、数据高可靠性、存储的Qos、数据的就近访问或数据的多版本控制。
存储客户端根据各Bucket的配置情况确定各Bucket的迁移策略参数,从而可以配置较佳的对象迁移策略,将对象在具有不同等级的增值功能或服务的Bucket之间迁移,可以对对象的Bucket归属进行更加精细化的管理,进而达到资源的优化配置和利用。
在一种可能的实现方式中,所述迁移策略确定模块根据所述Bucket的增值功能或服务的变化更新所述迁移策略参数,并通过所述发送模块向所述对象存储服务器发送携带该更新后的迁移策略参数的迁移策略配置更新请求,请求所述对象存储服务器对所述源Bucket配置的对象迁移策略进行更新。
存储客户端根据该存储客户端所创建和使用的Bucket的配置情况的变化适时地向对象存储服务器更新迁移策略参数,使其相应地更新和调整对象迁移策略,从而使得,对象的Bucket归属管理以及资源的配置和利用更加动态和灵活。
第五方面,本申请实施例提供了一种对象存储服务系统,该对象存储服务系统包括对象存储服务器、数据持久化组件、元数据数据库、以及多个硬盘。其中,该对象存储服务器,用于管理多个桶Bucket,每个所述Bucket包括一个或多个对象,接收存储客户端的发送的携带迁移策略参数的迁移策略配置请求,根据所述的迁移策略参数为源Bucket配置对象迁移策略,并将所述对象迁移策略保存在所述元数据数据库中,以及根据所述对象迁移策略对所述源Bucket中的满足所述对象迁移策略的对象进行迁移处理;其中所述迁移策略参数包括对象迁移策略所对应的源Bucket的标识、对象的迁移属性、所述迁移属性的属性值、以及目的Bucket的标识。数据持久化组件用于存储所述多个Bucket。所述元数据数据库用于保存所述对象迁移策略。所述多个硬盘用于提供物理存储空间。
本实施中,对象存储服务器可以根据存储客户端发送的携带迁移策略参数的迁移策略配置请求对源Bucket进行对象迁移策略的配置。然后根据所述对象迁移策略对该源Bucket中的满足所述对象迁移策略的对象进行迁移操作流程的处理,例如,根据该对象迁移策略针对该Bucket中的不同的对象判断是否需要进行对象迁移处理,并将满足对象迁移策略的对象迁移到目的Bucket。从而,不需要存储客户端根据用户的手动操作指示进行对象迁移处理,提高了对象迁移效率。
在一种可能的实现方式中,该对象的迁移属性可以是对象的重要程度,或者也可以是对象被访问的频率,或者在所述多个Bucket分布在多个不同地区Region的场合下,该对象的迁移属性还可以是对象被某一Region内的访问请求者访问的频度,所述某一Region不同于所述目的Bucket所归属的Region。
通过本实施例,可以将需要重点管理的对象(例如,被访问频率大或者重要程度较高的对象)都放置在配置了较多的增值功能或服务的目的Bucket中,而将其他对象(例如,被访问频率小或者重要程度较低的对象)集中放置到配置了较少的增值功能或服务的所述的源 Bucket中,从而可以优化资源配置和利用,并相应达到节约资源的效果。因为配置了较多的增值功能或服务的Bucket的需要更多的资源的支持,如网络资源、处理资源,和/或存储资源等。
在一种可能的实现方式中,所述迁移策略配置请求还包括优先级参数,该优先级参数与所述对象的迁移属性相关联。相应的,所述对象迁移策略中包括该优先级参数。该优先级参数用于在所述源Bucket配置有多个对象迁移策略且所述第一对象分别满足所述多个对象迁移策略的情况下,所述对象存储服务器根据所述优先级参数选择所述源Bucket的对象迁移策略。
通过对同一Bucket配置的不同对象迁移策略设置优先级,从而可以针对同一个Bucket设置多个对象迁移策略而不会产生冲突,进而还可以针对不同Bucket的对象归属管理进行更加精细化管理,并进一步优化资源配置和利用以及节约资源。
第六方面,本申请实施例提供了一种对象存储服务器,该对象存储服务器包括处理器,所述处理器耦合至存储器,所述处理器执行所述存储器上的程序时实现如上述第一方面或基于该第一方面的各种可能的实现方式中的任一方法。
以上对象存储服务器的各实施方式的效果可以参考上面第一方面及其各种可能的实现方式中的描述以及说明书中相关部分的描述,此处不再一一赘述。
第七方面,本申请实施例提供了一种存储客户端,该存储客户端包括处理器,所述处理器耦合至存储器,所述处理器执行所述存储器上的程序时实现如上述第二方面或基于该第二方面的各种可能的实现方式中的任一方法。
以上存储客户端的各实施方式的效果可以参考上面第二方面及其各种可能的实现方式中的描述以及说明书中相关部分的描述,此处不再一一赘述。
第八方面,本申请实施例提供了一种对象存储服务系统,该对象存储服务系统包括如所述第三方面或基于该第三方面的各种可能的实现方式中的任一对象存储服务器、数据持久化组件、元数据数据库、以及多个硬盘;数据持久化组件用于保存所述多个Bucket;所述元数据数据库用于保存所述对象迁移策略;所述多个硬盘用于提供物理存储空间。
第九方面,本申请实施例提供了一种存储系统,该存储系统包括如上述第三方面或基于该第三方面的各种可能的实现方式中的任一对象存储服务器以及如第四方面或基于该第四方面的各种可能的实现方式中的任一存储客户端。
第十方面,本申请实施例提供了一种计算机可读存储介质,该计算机存储介质中存储有程序代码,该程序代码用于指示执行上述第一方面。
第十一方面,本申请实施例提供了一种计算机可读存储介质,该计算机存储介质中存储有程序代码,该程序代码用于指示执行上述第二方面的任意可能的实现方式中的方法。
以上计算机可读存储介质的各实施方式的效果可以参考上面相应地方的描述以及说明书中相关部分的描述,此处不再一一赘述。
附图说明
图1为本发明实施例提供的一种存储系统架构示意图;
图2为本发明实施例提供的一种对象迁移方法流程图;
图3为本发明实施例提供的另一种对象迁移策略的配置方法流程图;
图4为本发明实施例提供的一种对象迁移方法的实际应用场景流程图;
图5为本发明实施例提供的另一种对象迁移方法的实际应用场景流程图;
图6为本发明实施例提供的一种对象迁移方法流程流程图;
图7为本发明实施例提供的又一对象迁移方法程流程图;
图8为本发明实施例提供的再一种对象迁移方法程流程图;
图9为本发明实施例提供的一种对象存储服务器的结构示意图;
图10为本发明实施例提供的一种存储客户端的结构示意图;
图11为本发明实施例提供的另一种对象存储服务器的结构示意图;
图12为本发明实施例提供的一种存储客户端的结构示意图。
具体实施方式
下面将结合附图对本发明作进一步地详细描述,显然,所描述的实施例仅仅是本发明一部分实施例,而不是全部的实施例。基于本发明中的实施例,本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其它实施例,都属于本发明保护的范围。
本发明实施例描述的网络架构以及业务场景是为了更加清楚的说明本发明实施例的技术方案,并不构成对于本发明实施例提供的技术方案的限定,本领域普通技术人员可知,随着网络架构的演变和新业务场景的出现,本发明实施例提供的技术方案对于类似的技术问题,同样适用。
本发明实施例可应用于如下图1所示的存储系统100中,如图1所示,该存储系统100包括对象存储服务系统110以及存储客户端112。其中所述对象存储服务系统110包括对象存储服务器102,数据持久化组件104,元数据数据库106,以及硬盘服务器108。所述对象存储服务系统110向存储客户端户提供对象存储服务。用户通过存储客户端112使用所述的对象存储服务,如用户通过存储客户端112对存储在云上的数据或文件进行各种操作,例如创建、修改、维护或删除Bucket,以及通过上传、下载、删除Bucket中的对象等操作来管理自己数据。
其中,对象存储服务器102,用于提供对象存储接口以及接口功能的实现,为存储客户端112与对象存储服务系统110之间的交互提供接口,管理多个Bucket,接受存储客户端的各类操作请求,例如创建、修改、维护或删除Bucket,或者上传、下载、删除Bucket中的对象等操作请求,根据所述操作请求进行处理,并向存储客户端返回处理结果。所述对象存储服务器102,还用于接收存储客户端112的发送的携带迁移策略参数的迁移策略配置请求,根据所述的迁移策略参数为所述源Bucket配置对象迁移策略,并将所述对象迁移策略保存在所述元数据数据库中,以及根据所述的对象迁移策略对所述Bucket中的对象进行迁移处理。具体参考下述图2-8所示各实施例中的详细描述。
数据持久化组件104,用于将硬盘空间虚拟化,提供Bucket形态的逻辑空间,向所述对象存储服务器102提供数据存储接口,存储用户以及存储客户端的数据;
元数据数据库106,用于向所述对象存储服务器102提供元数据存储接口,存储所述对象存储服务系统的元数据以及控制数据。如以下实施例中所描述的迁移策略参数,具体参考下述图3所示实施例中步骤306中的详细描述。
硬盘服务器108,用于挂载所述对象存储服务系统的所有硬盘,对上层组件,如上述对象存储服务器102、数据持久化组件104、以及及元数据数据库106等,提供物理存储空间。
可以理解的是,上述数据持久化组件104、元数据数据库106以及硬盘服务器108三者中的一个或多个可合并设置到所述的对象存储服务器102中,当它们全部都合并设置入所述对象存储服务器102中时,所述对象存储服务器102与所述对象存储服务系统110等同。
图2示出了本发明实施例中提供的一种对象迁移方法。该方法可应用于上述图1所示的对象存储服务系统100,所述对象存储服务系统100包括对象存储服务器和存储客户端,所述对 象存储服务器管理多个桶Bucket,每个所述Bucket包括一个或多个对象,所述存储客户端使用所述多个Bucket中的至少两个Bucket。如图2所示,所述的对象迁移方法包括如下处理:
202:对象存储服务器获得所述源Bucket对应的对象迁移策略。
一种具体的实施例中,所述对象存储服务器从所述对象存储服务系统中的元数据数据库中获得之前针对该源Bucket配置的对象迁移策略,所述对象迁移策略表示对象从所述多个Bucket中的源Bucket迁移至目的Bucket的条件。所述对象迁移策略具体的配置过程请参考以下图3所示的实施例中的的详细描述。
该具体实施例中,该对象迁移策略可以由至少以下迁移策略参数构成:源Bucket的标识、对象的迁移属性、所述迁移属性的属性值、以及所述目的Bucket的Bucket标识。该具体实施例中,所述的对象的迁移属性包括对象的重要程度或对象被访问的频率,或者在多个Bucket分布在多个不同地区Region的情况下,所述对象的迁移属性包括对象被某一Region(例如Region 1)内的访问请求者访问的频度或频率,所述某一Region不同于所述目的Bucket所归属的Region(例如Region 2)。
需要说明的是,本申请实施例中所用的频度表示的是针对两个不同频率的比值,如上述实施例中,对象被某一Region内的访问请求者访问的频度指的是该对象被某一Region内的访问请求者访问的次数与该对象被被某一Region内的访问请求者访问的次数的比值。此外,上述Bucket的标识并不必然表示一个具体的Bucket,在有的实施例中,该Bucket的标识可以表示由多个具有相同特征的目的Bucket构成的一类Bucket,例如,Region 2中的Bucket。
此外,可以理解,对象的迁移属性可以根据业务场景的不同而进行不同的设置,以上仅仅是举例说明,不表示本发明仅仅限于上述三种列举的对象的属性。
另外,需要说明的是,不同的对象的迁移属性、该迁移属性的属性值以及这两个参数关联的源Bucket的Bucket标识和目的Bucket的Bucket标识构成一条对象迁移策略,可以称为第一对象迁移策略,例如,第一对象迁移策略可以包括:源Bucket的Bucket标识、对象的重要程度、重要程度相关的属性值、以及目的Bucket的Bucket标识。相应的,第二对象迁移策略包括:源Bucket的Bucket标识、对象被访问的频率、频率阈值、以及目的Bucket的Bucket标识;第三对象迁移策略包括:源Bucket的Bucket标识、对象的迁移属性包括对象被某一Region内的访问请求者访问的频度或频率、频度阈值或频率阈值、以及目的Bucket的Bucket标识。
另一的实施例中,所述迁移策略参数还可以进一步包括优先级参数,所述的优先级参数与所述对象的迁移属性相关联,这样每个对象前一策略就有一个对应的优先级,该优先级参数用于在所述源Bucket配置有多个对象迁移策略且在某个对象满足多个所述对象迁移策略的场景下,根据该优先级参数选择所述源Bucket的对象迁移策略。具体可以参考以下步骤206以及下述图3所示实施例中的步骤306的详细描述。
204:对象存储服务器判断所述源Bucket中的第一对象是否满足所述对象迁移策略。
一种具体实施例中,对象存储服务器判断该源Bucket中的第一对象是否满足为该源Bucket配置的对象迁移策略。具体的判断过程会随着不同的对象迁移策略而有不同的判断处理,具体可与参考下述图6-8所示的实施例中的详细描述。图6-8分别举例说明三种不同的对象迁移策略的处理流程,分别为:图6描述了以对象被访问的频率为迁移属性的第一对象迁移策略的对象迁移处理流程;图7描述以对象的重要程度为迁移属性的第二对象迁移策略对象迁移处理流程,图8描述了以对象被某一Region内的访问请求者访问的频度为移属性的第三对象迁移策略的对象迁移处理流程。
206:如果步骤204的判断结果为所述源Bucket中的第一对象满足所述对象迁移策略,对 象存储服务器将所述第一对象迁移至所述目标Bucket。
一种具体的实施例中,对象存储服务器可以在判断出该第一对象满足其中的一条对象迁移策略后,就可以将该第一对象迁移至该条策略对应的目的Bucket迁移。例如,对象存储服务器判断该第一对象满足设置的对象的重要程度以及重要程度相关的属性值,则将该第一对象迁移至条对象迁移策略对应的目的Bucket标识所对应的目的Bucket中,不再进行是否满足其他迁移属性即对应的属性值的判断。
另一种具体的实施例中,对象存储服务器可以在判断出该第一对象满足其中的一条对象迁移策略后,继续判断是否满足其他对象迁移策略,并在满足其他对象迁移策略,且目的Bucket不存在冲突的情况下,将该第一对象迁移至各符合条件的对象迁移策略中的目的Bucket的标识所表示的目的Bucket中。
再一种具体的实施例中,对象存储服务器可以在判断出该第一对象满足其中的一条对象迁移策略后,继续判断是否满足其他对象迁移策略,在满足其他对象迁移策略,但目的Bucket会存在冲突的情况下,可以针对的迁移策略设置优先级,并根据优先级高的对象迁移策略进行对象的迁移。优先级参数的设置可以设置在迁移策略参数中,具体可以参考上述步骤202的描述以及下述图3所示实施例中的步骤306的详细描述。可以理解的是,也可以以其他的方式设置对象迁移策略的优先级。
208:所述对象存储服务器通知所述存储客户端所述对象迁移的结果。
一种具体实施例中,所述对象存储服务器在该第一对象迁移成功后,通知所述所述存储客户端所述对象迁移的结果。当对象迁移策略中的目的Bucket标识表示一个具体的目的Bucket时,该通知消息中携带目的Bucket的Bucket标识,以及该第一对象的对象标识。当对象迁移策略中的目的Bucket标识表示一类具有相同特征的Bucket时,该通知中除了携带目的Bucket的Bucket标识,该第一对象的标识外,还可以包括迁移至的具体目的Bucket的编号、名称或地址之类的标识信息。
另一种具体的实施例中,所述对象存储服务器在该第一对象迁移成功后,并不立即主动通知所述所述存储客户端所述对象迁移的结果。在这种情况下,所述的存储客户端很难感知到存储对象被迁移,所以可能仍然访问迁移前的源Bucket中的对象。当所述对象存储服务器接收到所述存储客户端的对象访问请求之后,检查对象访问请求中携带的对象标识所对应的对象是否已经迁移至其他Bucket,如果已经迁移到其他Bucket中,则通知所述所述存储客户端所述对象的迁移目的Bucket。该通知消息中可以携带目的Bucket的Bucket标识,以及所述对象的对象标识。
再一种具体的实施例中,所述对象存储服务器在该第一对象迁移成功后,并不立即主动通知所述所述存储客户端所述对象迁移的结果。而是等到所述存储客户端主动查询对象的位置信息时,才告知所述所述所述存储客户端所述对象迁移的结果。在这种情况下,所述存储客户端可以在任何时候向所述对象存储服务器发送对象查询消息,所述的对象查询消息中包括被查询对象的对象标识。当所述对象存储服务器接收到所述存储客户端的对象查询请求之后,检查对象查询请求中携带的对象标识所对应的对象是否已经迁移至其他Bucket,如果已经迁移到其他Bucket中,则通知所述所述存储客户端所述对象的迁移目的Bucket。该通知消息中可以携带目的Bucket的Bucket标识,以及所述对象的对象标识。该存储客户端提前主动查询,可以提前获知该存储客户端所创建和使用的对象的存储位置,从而可以在后续访问对象时,直接根据查询到的位置进行访问,提高了访问效率。
210:如果步骤210的判断结果为所述源Bucket中的第一对象不满足所述对象迁移策略, 则不对所述第一对象进行迁移操作。
一种具体的实施例中,对象存储服务器可以在判断出该第一对象不满足其中的一条对象迁移策略后,继续判断是否满足其他对象迁移策略,只有在都不满足的情况下,才确定所述所述源Bucket中的第一对象不满足所述源Bucket的对象迁移策略,则不对该第一对象进行迁移处理。
另一种具体的实施例中,虽然该第一对象满足一条以上的对象迁移策略,但如果根据这些对象迁移策略的迁移会导致冲突,例如,在没有为各对象迁移策略设置优先权的情况下或者即使设置了优先权但还是存在冲突的情况下,根据这些对象迁移策略的迁移会导致冲突。则这种情况视为所述源Bucket中的第一对象不满足所述源Bucket的对象迁移策略。
通过本实施中的处理,对象存储服务器可以根据预先配置的对象迁移策略针对不同的对象判断是否需要进行对象迁移处理,而不需要存储客户端根据用户的手动操作指示进行对象迁移处理,从而可以提高对象迁移的效率,另外还可以解决用户无法自动管理对象归属Bucket的问题。
此外,通过实施例,还可以将需要重点管理的对象(例如,被访问频率大或者重要程度较高的对象)都放置在配置了较多的增值功能或服务的目的Bucket中,而将其他对象(例如,被访问频率小或者重要程度较低的对象)集中放置到配置了较少的增值功能或服务的所述的源Bucket中,从而可以优化资源配置和利用,并相应达到节约资源的效果。因为配置了较多的增值功能或服务的Bucket的需要更多的资源的支持,如网络资源、处理资源,和/或存储资源等。
实际应用中,在图2所示实施例的步骤202之前,对象存储服务器需要根据存储客户端的迁移策略的配置对象迁移策略,然后根据之前配置的对象迁移策略,获得所述源Bucket对应的对象迁移策略。具体配置对象迁移策略的处理流程可以参考以下图3。
图3示出了本发明实施例中提供的一种对象迁移策略的配置方法。该方法可应用于上述图1所示的对象存储服务系统100,如图3所示,所述的对象迁移策略的配置方法包括如下处理:
302:存储客户端确定迁移策略参数。
参考上述步骤202的描述,在一种具体的实施例中,该对象迁移策略由至少以下迁移策略参数构成:源Bucket的标识、对象的迁移属性、所述迁移属性的属性值、以及所述目的Bucket的标识。所述对象的迁移属性包括:对象的重要程度或对象被访问的频率;或者在多个Bucket分布在多个不同地区Region的情况下,所述对象的迁移属性包括对象被某一Region(例如Region 1)内的访问请求者访问的频度或频率,所述某一Region不同于所述目的Bucket所归属的Region(例如Region 2)。
具体实施例中,存储客户端可以根据该存储客户端所创建或使用的各Bucket的配置情况确定各Bucket的迁移策略参数,包括确定各Bucket的对象的迁移属性以及对应的属性值。其中所述Bucket的配置情况包括该各Bucket所配置的增值功能或服务。例如,Bucket所配置的增值功能或服务包括至少下述之一:网络加速、数据高可靠性、存储的Qos、数据的就近访问或数据的多版本控制。
以下分别就不同场景下,针对不同的Bucket配置情况确定不同的迁移策略参数举例说明如下:
场景1:例如,存储客户端建立或使用了2个Bucket,分别为第一Bucket和第二Bucket,其中第一Bucket配置了较高水平的的存储Qos能力,例如该第一Bucket配置的每秒钟处理的外部请求数量(Transactions Per Second,TPS)以及并发数指标都很高,而第二Bucket配置的每秒 钟处理的外部请求数量(Transactions Per Second,TPS)以及并发数指标都没有特别的要求。
在上述这种配置情况下,存储客户端可能需要尽量将被访问频率大的对象都放置在所述第一Bucket中,而将被访问频率小的对象集中放置到所述的第二Bucket中,从而可以优化资源配置和利用,并相应达到节约资源的效果。因为配置了较多的增值功能或服务的Bucket的需要更多的资源的支持,如网络资源、处理资源,和/或存储资源等,从而需要支付更高的费用,因此也相应的可以节约更多的成本。
相应地,存储客户端会将对象的被访问频率确定为迁移属性,对于该迁移属性的属性值,即本场景中的对象被访问的频率大小,可以由存储客户端根据用户的指示,例如通过命令行或者用户界面输入的命令确定,也可以由存储客户端根据具体场景确定。
据此,可以将迁移策略参数确定为:源Bucket标识为第二Bucket的标识、对象被访问的频率、对象被访问频率高于100次/天、以及目的Bucket标识为第一Bucket的Bucket标识。或者将迁移策略参数确定为:源Bucket标识为第一Bucket的标识、对象被访问的频率、对象被访问频率低于100次/天、以及目的Bucket标识为第二Bucket的Bucket标识。
场景2:例如,存储客户端建立或使用了2个Bucket,分别为第一Bucket和第二Bucket,其中第一Bucket配置了较高性能的固态存储来作为Bucket的底层介质,而第二Bucket配置了较低性能的一般存储介质来作为Bucket的底层介质。
则在上述这种配置情况下,存储客户端可能需要尽量将被访问频率大的对象都放置在所述第一Bucket中,而将被访问频率小的对象放置到所述的第二Bucket中,从而可以优化资源配置和利用,并相应达到节约资源的效果。
相应地,存储客户端会将对象的被访问频率确定为迁移属性,对于该迁移属性的属性值,即本场景中的对象被访问的频率大小,可以由存储客户端根据用户的指示,例如通过命令行或者用户界面输入的命令确定,也可以由存储客户端根据具体场景确定。
据此,可以将迁移策略参数确定为:源Bucket标识为第二Bucket的标识、对象被访问的频率、对象被访问频率高于100次/天、以及目的Bucket标识为第一Bucket的Bucket标识。或者将迁移策略参数确定为:源Bucket标识为第一Bucket的标识、对象被访问的频率、对象被访问频率低于100次/天、以及目的Bucket标识为第二Bucket的Bucket标识。
场景3:例如,存储客户端建立或使用了2个Bucket,分别为第一Bucket和第二Bucket,其中第一Bucket对数据可靠性有较高的配置,例如对数据的容灾要求比较高,要求异地容灾;而第二Bucket则对数据可靠性有没有特殊的要求,仅满足一般性容灾条件即可。
则在上述这种配置情况下,存储客户端可能需要尽量将重要程度比较高的对象都放置在所述第一Bucket中,而将重要程度较低的对象放置到所述的第二Bucket中,从而可以优化资源配置和利用,并相应达到节约资源的效果。
相应地,存储客户端会将对象重要程度确定为迁移属性,对于该迁移属性的属性值,可以由存储客户端根据用户的指示,例如通过命令行或者用户界面输入的命令确定,也可以由存储客户端根据具体场景确定。一种具体的实施例中,该属性值可以设置为对象的存储时长,超过一定存储时长的对象即可以认为重要程度降低了,从而变得不重要了。当然也可以设置超过一定存储时长的对象即被认为重要程度升高了,从而变得更加重要了。
据此,可以将迁移策略参数确定为:源Bucket标识为第二Bucket的标识、对象的重要程度、对象的存储时长(例如大于或等于30天)、以及目的Bucket标识为第一Bucket的Bucket标识。或者将迁移策略参数确定为:源Bucket标识为第一Bucket的标识、对象的重要程度、对象的存储时长(例如大于或等于30天)、以及目的Bucket标识为第二Bucket的Bucket标识。
场景4:例如,为了解决就近访问的问题,云存储厂商一般都会在不同的城市搭建多个地区Region级数据中心,每个Region可以包括多个Bucket。例如华为公有云在Beijing部署有第一Region、在Hongkong部署有第二Region。同一个存储客户端可以在多个Region中创建或使用Bucket。例如,存储客户端创建了2个Bucket,分别为第一Bucket和第二Bucket,其中,第一Bucket归属于第一Region,第二Bucket归属于第二Region。如果第一Bucket中的对象经常被第二Region内的访问请求者访问。为了更好地解决就近访问的问题,需要将经常被第二Region内访问请求者访问的归属于所述第一Region的第一Bucket中的对象迁移至该第二Region中的第二Bucket。从而可以节省网络资源以及提高访问效力。
在这种场景下,存储客户端会将对象被第二Region内的访问请求者的访问频度或频率确定为迁移属性,对于该迁移属性的属性值,可以由存储客户端根据用户的指示,例如通过命令行或者用户界面输入的命令确定,也可以由存储客户端根据具体场景确定。该属性值可以设置为:对象被来自该第二Region内的访问请求者的访问次数与对象被来自所有Region内的访问请求者访问的次数的比值(针对频率情况),或者100次/天(针对频率的情况)。
据此,可以将迁移策略参数确定为:源Bucket标识为第一Bucket的标识、对象被第二Region内的访问请求者的访问频度、对象被来自该第二Region内的访问请求者的访问次数与对象被来自所有Region内的访问请求者访问的次数的比值(例如大于或等于20%)、以及目的Bucket标识为第二Bucket的Bucket标识。或者将迁移策略参数确定为:源Bucket标识为第一Bucket的标识、对象被第二Region内的访问请求者的访问频率、对象被来自该第二Region内的访问请求者访问的频率(例如大于或等于100次/天)、以及目的Bucket标识为第二Bucket的Bucket标识。
同理,当归属于第二Region内的第二Bucket中的对象经常被第一Region内的访问请求者访问时,为了更好地解决就近访问的问题,需要将经常被第一Region内访问请求者访问的归属于所述第二Region的第二Bucket中的对象迁移至该第一Region中的第一Bucket。从而可以节省网络资源以及提高访问效力。具体处理过程参考上面的描述,在此不再赘述。
上述4种场景中,是仅考虑Bucket的某种或某几种配置情况,有时需要综合考虑存储客户端建立或使用各Bucket上的全面配置情况后,确定迁移策略参数。以下场景4是综合考虑各中配置情况后确定迁移策略参数的举例说明:
场景5:例如,存储客户端建立或使用了2个Bucket,分别为第一Bucket和第二Bucket,其中第一Bucket配置了较多的增值功能或服务,如网络加速、数据高可靠性以及高水平的存储Qos,数据的就近访问等配置。而第二Bucket配置了较少的增值功能或服务甚至仅配置了基本的功能和服务。在这种情况下,存储客户端可能需要综合考虑对象的被访问频率和对象的重要程度甚至对象被所述某一Region内的访问请求者访问频度或频率,从而确定将分别满足被访问频率的设定属性值、对象的重要程度的设定属性值、以及对象被所述某一Region内的访问请求者访问频度或频率的设定属性值的对象放置在所述第一Bucket中,而将不满足上述条件的对象放置到所述的第二Bucket中,从而可以更加优化资源的配置和利用,达到更好地节约资源的效果。
需要说明的是,以上场景仅是为了便于理解本申请的技术方案而做的举例说明,具体实施中,存储客户端可以根据其他场景,例如网络加速,存储性能、数据的就近访问或数据的多版本控制等,进行如上述方法类似的确定迁移策略参数。在此不一一列举和说明,本领域技术人员在阅读本申请实施例的基础上很容易想到其他场景下的相似处理,这些都在本申请的保护范围内。
可以理解,存储客户端可以根据创建和使用的Bucket的变化或者这些Bucket的配置更新相应的修改或更新迁移策略参数。修改或更新迁移策略参数可以参考以上确定迁移策略参数的处理过程,在此不予以赘述。
304:存储客户端向对象存储服务器请求迁移策略的配置。
一种具体的实施例中,存储客户端在确定迁移策略参数后向对象存储服务器请求迁移策略的配置。具体可以通过在存储客户端和对象存储服务器新增加的Bucket配置接口向对象存储服务器发送迁移策略配置请求消息。该迁移策略配置请求消息可以采用OBS协议。该迁移策略配置请求消息包括如下表1所示例的迁移策略参数:
Figure PCTCN2017119464-appb-000001
表1:迁移策略配置请求消息携带的参数
可以理解,存储客户端可以根据创建或使用的Bucket的变化或者这些Bucket的配置更新相应的修改或更新迁移策略参数,进而向所述对象存储服务器发送请求以通知所述对象存储服务器修改或更新之前的对象迁移策略的配置。
以上仅是存储客户端向对象存储服务器请求迁移策略配置的一种实现方式的举例说明,实际应用中,可以通过其他方式在对象存储服务器或对象存储服务系统配置对象迁移策略,本领域技术人员在阅读上述实施例后可以容易想到其他配置方式,这些其他方式都在本申请的保护范围之内。
306:对象存储服务器对所述源Bucket配置对象迁移策略;
在一个具体实施例中,对象存储服务器根据收到的所述迁移策略配置请求消息中包括的迁移策略参数进行对象迁移策略的配置。具体而言,对象存储服务器在元数据数据库创建(如果原来没有针对某个Bucket的对象迁移策略)或者修改(如果某一Bucket已经存在对象迁移策略)相应的数据,具体示例如下表2和3所示。
根据上述步骤302中的5种不同的场景为例,相应的,对象存储服务器针对Bucket1配置对象迁移策略,在元数据数据库创建或修改后的记录如下表2所示例:
Figure PCTCN2017119464-appb-000002
Figure PCTCN2017119464-appb-000003
表2:对象存储服务器针对Bucket 1配置如的对象迁移策略
以上述步骤302中的5种不同的场景为例,对象存储服务器针对Bucket 2配置对象迁移策略,在元数据数据库创建或修改后的记录如下表3所示例:
Figure PCTCN2017119464-appb-000004
表3:对象存储服务器针对Bucket 2配置如的对象迁移策略
308:对象存储服务器通知所述存储客户端对象迁移策略配置成功。
一种具体实施例中,对象存储服务器通可以通过前述步骤304中的Bucket配置接口通知所述存储客户端对象迁移策略配置成功,也可以通过OBS协议中的其他消息通知所述存储客户端对象迁移策略配置成功。
实际应用中,在图2所示的实施例中,对象存储服务器可以在以下两种情况或场景下启动执行上述图2所示的实施例中的步骤,分别是前台业务启动场景以及后台定时启动场景,以下图4和图5分别介绍了这两种启动场景。
图4示出了本申请实施例中提供的一种对象迁移方法的实际应用场景。该方法可应用于上述图1所示的对象存储服务系统100,以及应用于图2和3所示的方法流程中,如图4所示,所述的一种对象迁移方法包括如下处理:
402:存储客户端访问对象。
一种具体实施例中,存储客户端以OBS协议访向对象存储服务器发送对象访问请求,请求访问该存储客户端创建的第一Bucket的第一对象,该对象访问请求中可以包括该第一Bucket的Bucket标识以及所述第一对象的标识。
404:对象存储服务器检查相应的Bucket是否配置了对象迁移策略。
一种具体实施例中,对象存储服务器在收到所述存储客户端发送的所述对象访问请求后,根据所述存储客户端中携带的Bucket标识检查该Bucket标识对应的Bucket是否配置了对象迁移策略。具体检查过程,例如,可以为对象存储服务器到所述元数据数据库中查询各Bucket相对应的对象迁移策略,如查询所述元数据数据库中是否记录了如上述表2或表3所示的针对不同的Bucket的对象迁移策略配置记录。
需要说明的是,在另一种实施例中,当所述对象访问请求中携带的不是所述第一对象,而是所述Bucket中的其他对象,该对象存储服务器在根据该对象的对象标识确定该对象归属于所述Bucket后,也会检查所述Bucket是否配置了对象迁移策略。
406:如果对象存储服务器确定所述Bucket没有配置对象迁移策略,则不执行对象迁移流程。
一种具体实施例中,如果通过查询后确认所述元数据数据库中没有记录所述Bucket标识所对应的Bucket相对应的对象迁移策略配置,则不执行针对所述Bucket中的对象的对象迁移处理流程,例如,不执行图2所对应实施例中的对象迁移处理流程。
408:如果对象存储服务器确定相应的Bucket配置了对象迁移策略,则进一步获得所述Bucket对应的对象迁移策略。
一种具体实施例中,如果通过查询后确认所述元数据数据库中记录有所述Bucket标识所对应的Bucket相对应的对象迁移策略配置,则根据所述的记录获取所述Bucket相对应的对象迁移策略,具体获取过程可以参考上述图2所示实施例中的步骤202,此处不再予以赘述。
如果获取的对象迁移策略为第一对象迁移策略,则根据第一对象迁移策略进行后续对象迁移处理流程的操作,具体可以参考上述上述图2所示实施例中的处理步骤204-210,此处不再赘述。
一个具体的例子中,所述第一对象迁移策略可以如上述表2中所示的policy 1,包括如下迁移策略参数:
源Bucket标识为:第一Bucket的标识,即Bucket 1;
对象的迁移属性为:对象被访问的频率;
属性值为:低于100次/天;
目的Bucket的Bucket标识为:第二Bucket的标识,即Bucket 2。
进一步该迁移策略参数的还可以包括优先级参数为1,表示第一优先级。
根据第一对象迁移策略进行后续对象迁移处理流程的操作,具体可以包括根据上述第一对象的标识以及上述第一对象迁移策略的迁移策路参数判断所述第一Bucket中的第一对象是否满足所述第一对象迁移策略。具体参考上述上述图2所示实施例中的处理步骤204,此处不再赘述。
如果获取的对象迁移策略为第二对象迁移策略,则根据第一对象迁移策略进行后续对象迁移处理流程的操作,具体可以参考上述上述图2所示实施例中的处理步骤204-210,此处不再赘述。
一个具体的例子中,所述第二对象迁移策略可以如上述表2中所示的policy 2,包括如下 迁移策略参数:
源Bucket标识为:第一Bucket的标识,即Bucket 1;
对象的迁移属性为:对象的重要程度;
属性值为:对象的存储时长大于或等于30天;
目的Bucket的Bucket标识为:第二Bucket的标识,即Bucket 2。
进一步该迁移策略参数的还可以包括优先级参数为1,表示第一优先级。
根据第二对象迁移策略进行后续对象迁移处理流程的操作,具体可以包括根据上述第一对象的标识以及上述第二对象迁移策略的迁移策路参数判断所述第一Bucket中的第一对象是否满足所述第二对象迁移策略。具体参考上述上述图2所示实施例中的处理步骤204,此处不再赘述。
同理,如果获取的对象迁移策略第三对象迁移策略、第四对象迁移策略、或者第N对象迁移策路,它们的处理过程参照上述处理,此处不予以一一赘述。
图4所示的实施例是一种前台业务启动场景下的对象迁移处理流程,也就是根据存储客户端的对象访问业务启动了对特定对象(如本实施例中的第一对象)的对象迁移处理流程。可以理解,在这种启动场景下,对象存储服务器还可以进一步扩大判断范围,即除了判断上述第一对象是否满足所述第一Bucket相对应的对象迁移策略外,还可以进一步判断所述第一Bucket中的其他对象是否满足所述第一Bucket相对应的对象迁移策略,并根据判断结果做出迁移与否的处理,具体的流程同上,这里不予以一一赘述。
因为前台业务启动场景主要是针对特定的对象或者特定的Bucket中的对象进行对象迁移处理流程,而不会广泛地检查该对象存储服务器所管理的所有Bucket中的对象,并对所有满足对象迁移策略的对象进行迁移处理,处理效率会相对较高,持续时间相对较短,对其他业务的影响较少,而且根据业务启动可以根据特定对象的活跃程度启动对象迁移处理流程,对于活跃的对象,则该对象的对象迁移处理流程的启动也频繁些,而对于不活跃的对象,则该对象的对象迁移处理流程的启动次数就会相应减少,从而可以有针对性的进行对象迁移流程的处理,进而可以节省处理资源。
此外,需要说明的是,在上述通过前台业务启动对象迁移处理流程的场景下,如果存储客户端要求访问的第一对象满足所述第一Bucket的对象迁移策略,则在进行所述第一对象迁移至目的Bucket前,根据该存储客户端的访问请求,向该存储客户端返回所述第一对象。
图5示出了本申请实施例中提供的一种对象迁移方法的实际应用场景。该方法可应用于上述图1所示的对象存储服务系统100,以及应用于图2和3所示的方法流程中。与上述图4所示前台业务启动场景实施例不同在于,本实施例中是后台定时启动场景,一旦后台任务启动,则会检查该对象存储服务器所管理的Bucket中的对象,具体处理过程如图5所示,包括如下处理:
502:任务启动。
一种具体实施例中,对象存储服务器可以在后天设定启动任务,按照一定的周期或者设置的期限定时启动任务,扫描检查所述象存储服务器所管理或维护的Bucket中的对象。
504:对象存储服务器逐Bucket检查各Bucket是否配置了对象迁移策略。
一种具体实施例中,对象存储服务器逐Bucket检查该对象存储服务器所管理的各Bucket,确定各Bucket是否配置了对象迁移策略。具体检查过程,例如,可以为对象存储服务器到所述元数据数据库中查询各Bucket相对应的对象迁移策略,如查询所述元数据数据库中是否记录了如上述表2或表3所示的针对不同的Bucket的对象迁移策略配置记录。
506:如果对象存储服务器确定某一Bucket没有配置对象迁移策略,则对该Bucket中的对 象不执行对象迁移处理。
一种具体实施例中,如果通过查询后确认所述元数据数据库中没有记录该Bucket相对应的对象迁移策略配置,则不执行针对该Bucket中的对象的对象迁移处理流程,例如,不执行图2所对应实施例中的对象迁移处理流程。
508:如果对象存储服务器确定某一Bucket配置了对象迁移策略,则进一步获得该Bucket对应的对象迁移策略。
一种具体实施例中,如果通过查询后确认所述元数据数据库中记录有该Bucket标识所对应的Bucket相对应的对象迁移策略配置,则根据所述的记录获取所述Bucket相对应的对象迁移策略,具体获取过程可以参考上述图2所示实施例中的步骤202,此处不再予以赘述。
如果获取的对象迁移策略为第一对象迁移策略,则根据第一对象迁移策略针对该Bucket中的各个对象分别进行后续对象迁移处理流程的操作,具体可以参考上述上述图2所示实施例中的处理步骤204-210,此处不予以一一赘述。
针对该Bucket不同的对象迁移策略的处理,具体可以参考上述步骤408中的处理,此处不予以一一赘述。
当对该Bucket中的所有对象分别执行完对象迁移处理流程的操作后,所述对象存储服务器进一步处理其他配置了对象迁移策略的Bucket中的对象,具体过程同上,此处不予以一一赘述,直到所述对象存储服务器所管理的所有Bucket的所有对象执行完毕为止。
由于后台定时启动场景是针对该对象存储服务器所管理的Bucket中的对象进行对象迁移处理流程,可以比较及时和完整的完成对象的迁移处理,从而在存储客户端访问对象时,可以及时地访问迁移后的对象,而不用等到用户访问之后再进行对象迁移处理,进一步优化了资源和利用。
可以理解,由于后台定时启动场景是在一个特定时间段内针对该对象存储服务器所管理的所有Bucket中的对象进行扫描检查和对象迁移处理,如果涉及到的Bucket很多,则可能会对资源的占用稍微大些。为了不对资源的利用造成较大影响或对其他业务造成较大影响,对象存储服务器可以设定在不同时间段内针对一定数量而不是全部的Bucket进行扫描检查和对象迁移处理,而在另一时间段内读其他的Bucket进行扫描检查和对象迁移处理,从而可以减轻对资源的利用造成的影响或对其他业务造成的影响。
实际应用中,在图2所示实施例的步骤204中,对于不同的对象迁移策略的判断过程会有些不同的处理,以下图6-8分别举例说明三种不同的对象迁移策略的处理流程,分别为:以对象被访问的频率为迁移属性的第一对象迁移策略(图6),对象的重要程度为迁移属性的第二对象迁移策略(图7),以及以对象被某一Region内的访问请求者访问的频度为迁移属性的第三对象迁移策略(图8)。需要说明的是,以下图6-8所示的实施例都以应用在图4所示的前台业务启动场景为例说明的,可以理解,图6-8所示的实施例也可以应用在图5所示的后台任务定时启动场景,具体过程可以参考上述图5,本申请对此不在一一赘述。
图6示出了本发明实施例中提供的一种对象迁移方法。该方法根据对象的被访问频率确定是否进行对象迁移处理。该方法可应用于上述图1所示的对象存储服务系统100,以及应用于图2、3和4所示的方法流程中,如图6所示,所述的对象迁移策略的配置方法包括如下处理:
602:存储客户端访问对象。
一种具体实施例中,存储客户端以OBS协议访向对象存储服务器发送对象访问请求,请求访问该存储客户端创建的第一Bucket的第一对象,该对象访问请求中可以包括该第一Bucket的Bucket标识以及所述第一对象的标识。具体参考上述402步骤,这里不再赘述。
604:判断是否切换到新的统计周期。
一种具体实施例中,对象存储服务器在收到所述存储客户端发送的所述对象访问请求后,获取所述第一Bucket对应的对象迁移策略,并在确定所述对象迁移策略为以对象的被访问频率为对象的迁移属性的第一对象迁移策略(具体过程可参考上述步骤202、步骤306以及步骤408),进一步判断是否切换到新的统计周期。其中所述的统计周期指的是统计对象被访问频率的单位,例如是以天统计还是以小时统计所述对象被访问的频率。当确定需要切换到新的统计周期后,执行下述步骤606然后再执行步骤608;当不确定需要切换到新的统计周期后,直接执行步骤608。
606:重置当前统计值。
一种具体实施例中,对象存储服务器在确定需要切换到新的统计周期后,以新的统计周期作为当前的统计周期,例如由之前的以1小时为单位的统计切换到以1天为单位的统计,然后根据该新的统计周期统计所述第一对象在本次访问之前被访问的次数,然后执行下述步骤608。可以理解,本申请中所说的统计可以是基于一定期间内的统计,也可以是针对所述对象整个生命周期的统计。
608:对象在当前统计周期内访问次数加1。
一种具体实施例中,对象存储服务器在当前的统计周期内进行所述第一对象的被访问频率统计。根据上述例子,如果对象存储服务器确定不需要切换到新的统计周期,则直接以天为单位统计第一对象的被访问频率,并根据所述存储客户端的本次访问请求将访问次数加1。如果对象存储服务器确定需要切换到新的统计周期,则由于步骤606已经重置了统计周期,本步骤中,对象存储服务器直接在上述步骤606的统计基础上(基于切换后的统计周期),根据所述存储客户端的本次访问请求将访问次数加1得到更新后的所述第一对象的被访问次数。
可以理解,在另一具体实施例中,上述步骤604和606也可以不用执行,而直接从步骤602跳转至步骤608。
610:判断当前对象的被访问次数是否低于属性值。
一种具体实施例中,所述对象存储服务器根据所述第一对象迁移策略获得所述对象的迁移属性的属性值,例如为所述对象被访问的频率阈值。所述对象存储服务器判断所述更新后的第一对象的被访问次数是否低于所述第一对象迁移策略中包括的属性值,即上述例子中的对象被访问的频率阈值,如低于100天/次。
612:在所述第一对象的被访问次数低于所述第一对象迁移策略中的属性值的情况下,进行对象迁移处理。
一种具体实施例中,在所述第一对象的被访问次数低于所述第一对象迁移策略中的属性值的情况下,所述对象存储服务器将所述第一对象迁移至所述第一对象迁移策略中包括的目的Bucket标识所对应的目的Bucket中。
614:在所述第一对象的被访问次数不低于所述第一对象迁移策略中的属性值的情况下,不进行对象迁移处理。
可以理解,上述步骤610-614中,在其他的实施例中,所述对象存储服务器还可以判断所述第一对象的被访问次数是否高于某一对象迁移策略中包括的属性值,如低于100天/次,并在所述第一对象的被访问次数高于所述某一对象迁移策略中包括的属性值的情况下,进行进行对象迁移处理,否则,不进行迁移处理,具体可以参考上述步骤306的描述。此处不一一进行赘述。
通过本实施中的处理,可以将被访问频率大的对象都放置在配置了较多的增值功能或服 务的目的Bucket中,而将被访问频率小的对象集中放置到配置了较少的增值功能或服务的所述的源Bucket中,从而可以优化资源配置和利用,并相应达到节约资源的效果。因为配置了较多的增值功能或服务的Bucket的需要更多的资源的支持,如网络资源、处理资源,和/或存储资源等。
图7示出了本发明实施例中提供的一种对象迁移方法。该方法根据对象的重要程度确定是否进行对象迁移处理。该方法可应用于上述图1所示的对象存储服务系统100,以及应用于图2、3和4所示的方法流程中,如图7所示,所述的对象迁移策略的配置方法包括如下处理:
702:存储客户端访问对象。
一种具体实施例中,存储客户端以OBS协议访向对象存储服务器发送对象访问请求,请求访问该存储客户端创建的第一Bucket的第一对象,该对象访问请求中可以包括该第一Bucket的Bucket标识以及所述第一对象的标识。具体参考上述402步骤,这里不再赘述。
704:获取系统当前时间。
一种具体实施例中,对象存储服务器在收到所述存储客户端发送的所述对象访问请求后,获取所述第一Bucket对应的对象迁移策略,并在确定所述对象迁移策略为以对象的重要程度为对象的迁移属性的第二对象迁移策略(具体过程可参考上述步骤202、步骤306以及步骤408)后获取系统当前时间,对象存储服务系统的当前时间,例如T1。
706:判断当前时间与对象上传时间的差值是否大于或等于所述属性值。
一种具体实施例中,对象存储服务器在确定所述对象迁移策略为以对象的重要程度为对象的迁移属性的第二对象迁移策略后,根据对象访问请求中包括的所述第一对象的标识,确定所述第一对象上传至所述对象存储服务系统的时间,例如T2。然后,所述对象存储服务器将所述T1减去所述T2获得系统当前时间与所述第一对象上传时间的差值,并根据所述第二对象迁移策略中包括的属性值判断所述差值是否大于或等于所述属性值,所述的属性值,例如为所述对象存储在所述源Bucket中的时长阈值。
708:在当前时间与对象上传时间的差值大于或等于属性值的情况下,进行对象迁移处理。
一种具体实施例中,在当前时间与对象上传时间的差值大于或等于属性值的情况下,所述对象存储服务器将所述第一对象迁移至所述第二对象迁移策略中包括的目的Bucket标识所对应的目的Bucket中。
710:在当前时间与对象上传时间的差值小于所述属性值的情况下,不进行对象迁移处理。
通过本实施中的处理,可以将根据对象的生命周期确定对象的重要程度,例如,一份重要程度高的数据,经过一定的时间后可能会变成重要程度低的数据,最终可能会变成归档数据,然后将重要程度比较高的对象都放置在配置了较多的增值功能或服务的源Bucket中,而将重要程度较低的对象放置到配置了较少的增值功能或服务的目的Bucket中,从而可以优化资源配置和利用,并相应达到节约资源的效果。因为配置了较多的增值功能或服务的Bucket的需要更多的资源的支持,如网络资源、处理资源,和/或存储资源等。
可以理解,以上仅是对对象的重要程度的一种举例说明,在另外的场合,一份重要程度低的数据经过一定的时间后可能会变成重要程度高的数据。或者,还可以根据时间以外的其他标准,例如访问该对象的请求者的位置,确定该数据重要与否。
图8示出了本发明实施例中提供的一种对象迁移方法。该方法根据对象的被访问频率确定是否进行对象迁移处理。该方法可应用于上述图1所示的对象存储服务系统100,以及应用于图2、3和4所示的方法流程中,如图8所示,所述的对象迁移策略的配置方法包括如下处理:
802:存储客户端访问对象。
一种具体实施例中,存储客户端以OBS协议访向对象存储服务器发送对象访问请求,请求访问该存储客户端创建的第一Bucket的第一对象,该对象访问请求中可以包括该第一Bucket的Bucket标识以及所述第一对象的标识。具体参考上述402步骤,这里不再赘述。
804:判断是否切换到新的统计周期。
一种具体实施例中,对象存储服务器在收到所述存储客户端发送的所述对象访问请求后,获取所述第一Bucket对应的对象迁移策略,并在确定所述对象迁移策略为第三对象迁移策略,第三对象迁移策略包括的对象的迁移属性为对象被某一Region内的访问请求者访问的频度(具体过程可参考上述步骤202、步骤306以及步骤408),进一步判断是否切换到新的统计周期。在本实施例中,所述某一Region为所述存储客户端所归属的Region,且所述所述存储客户端所归属的Region与所述第一Bucket所归属的Region不同。所述的统计周期指的是统计对象被访问频率的单位,例如是以天统计还是以小时统计所述对象被访问的频率。当确定需要切换到新的统计周期后,执行下述步骤806然后再执行步骤808;当不确定需要切换到新的统计周期后,直接执行步骤808。
806:重置当前统计值。
一种具体实施例中,对象存储服务器在确定需要切换到新的统计周期后,以新的统计周期作为当前的统计周期,例如由之前的以1小时为单位的统计切换到以1天为单位的统计,然后根据该新的统计周期统计所述第一对象在所述存储客户端本次访问之前被所述存储客户端所归属的Region内的访问请求者访问的次数,然后执行下述步骤808。此外,在本步骤中,所述对象存储服务器还需要在当前统计周期内,统计所述对象被所有Region内的访问请求者访问的次数。一种具体实施例中,对象存储服务器在当前的统计周期内统计统计所述第一对象被来自于所有Region内的访问请求者访问的次数。根据上述例子,如果对象存储服务器确定不需要切换到新的统计周期,则直接以天为单位统计第一对象被来自于所有Region内的访问请求者访问的次数。可以理解,本实施例中的所有Region指的是已经对所述对象进行访问了的各访问请求者所归属的各Region,而不包括一个或多个Region,该一个或多个Region内的访问请求者没有访问过所述的对象。另外,可以理解的是,本申请中所说的统计可以是基于一定期间内的统计,也可以是针对所述对象整个生命周期的统计。
808:在当前统计周期内,将所述对象被所述存储客户端所归属的Region内的访问请求者的访问次数加1获得更新后的所述对象被所述存储客户端所归属的Region内的访问请求者的访问次数。
一种具体实施例中,对象存储服务器在当前的统计周期内统计所述第一对象被来自于所述存储客户端所归属的Region内的访问请求者访问的次数。根据上述例子,如果对象存储服务器确定不需要切换到新的统计周期,则直接以天为单位统计第一对象被来自于所述存储客户端所归属的Region内的访问请求者访问的次数,并根据所述存储客户端的本次访问请求将访问次数加1。如果对象存储服务器确定需要切换到新的统计周期,则由于步骤806已经重置了统计周期,本步骤中,对象存储服务器直接在上述步骤806的统计基础上(基于切换后的统计周期),根据所述存储客户端的本次访问请求将访问次数加1得到更新后的所述第一对象被来自于所述存储客户端所归属的Region内的访问请求者访问的次数。
可以理解,在另一具体实施例中,上述步骤804和806也可以不用执行,而直接从步骤802跳转至步骤808。
需要说明的是,在直接从步骤802跳转至步骤808的情场景下,在本步骤中,所述对象存储服务器还需要在当前统计周期内,统计所述对象被所有Region内的访问请求者访问的次数。
810:判断更新后的所述对象被来自于所述存储客户端所归属的Region内的访问请求者访问的次数占所述对象被所有Region内的访问请求者访问的次数的比例是否超过设定的属性值。
一种具体实施例中,判断更新后的所述第一对象被来自于所述存储客户端所归属的Region内的访问请求者访问的次数占所述第一对象被所有Region内的访问请求者访问的次数的比例是否超过设定的属性值,所述第三对象迁移策略的属性值为所述对象被所述某一Region(本实施例中为所述存储客户端所归属的Region)内的访问请求者访问的频度阈值,例如20%。
812:在所述比例超过设定的属性值的情况下,进行对象迁移处理。
一种具体实施例中,当所述对象存储服务器确定更新后的所述第一对象被来自于所述存储客户端所归属的Region内的访问请求者访问的次数占所述第一对象被所有Region内的访问请求者访问的次数的比例超过设定的属性值(例如20%)后,所述对象存储服务器将所述第一对象迁移至所述第三对象迁移策略中包括的目的Bucket标识所对应的目的Bucket中。
814:在所述比例没有超过设定的属性值的情况下,进行对象迁移处理。
可以理解,本实施例也可以根据对象被某一Region内访问请求者的访问频率作为确定是否迁移的标准,具体可以参考上述图3所示实施例中的步骤302中的场景4以及步骤306中的表2和表3中的Policy 4,这里不予以一一赘述。
本实施中,根据对象被某一Region内访问请求者的访问频度,将被某一Region内的访问请求者的频繁访问的对象迁移至该Region,从更好地解决就近访问的问题,进而可以节省网络资源以及提高访问效力。
可以理解,以上仅示例性的描述了三种不同的对象迁移策略的处理流程,如上述图2以及图3所示的实施例所说明的,在实际应用中,还包括其他对象迁移策略,如上述步骤306的第四对象迁移策略、第五对象迁移策略、甚至第N对象迁移策路,它们的处理过程参照上述处理,此处不予以一一赘述。
下面结合附图对上述图1-图8所示实施例中所涉及的对象存储服务器以及存储客户端的具体实现进行详细说明。
如图9所示,图9为本申请实施例提供的一种对象存储服务器的结构示意图。需要说明的是,本申请本实施例中的对象存储服务器所述可以通过软件实现,相应的,该对象存储服务器为软件模块或程序,;也可以通过硬件实现,相应的,该对象存储服务器为硬件形态的对象存储服务器,例如各种物理服务器;还可以通过软件和硬件结合实现,相应的,该对象存储服务器为安装有能够实现上述图1-8中所述实施例中由该对象存储服务器执行的操作的软件模块或计算机程序的硬件形态的对象存储服务器,例如内置软件模块或计算机程序的各种带有处理器的对象存储服务器。所述对象存储服务器可以应用在上述图1-8所示的实施例中,所述对象存储服务器管理多个桶Bucket,每个所述Bucket包括一个或多个对象,所述对象存储服务器包括:接收模块902、策略配置模块904、策略获取模块906、策略判断模块908、迁移模块910、发送模块912。
其中,所述接收模块902,用于接收存储客户端发送的迁移策略配置请求,所述迁移策略配置请求包括所述对象迁移策略所对应的所述源Bucket的Bucket标识、对象的迁移属性、所述迁移属性的属性值、以及所述目的Bucket的Bucket标识;其中所述对象的迁移属性包括所述对象的重要程度或所述对象被访问的频率,或者在所述多个Bucket分布在多个不同地区Region的情况下,所述对象的迁移属性包括对象被某一Region内的访问请求者访问的频度,所述某一Region不同于所述目的Bucket所归属的Region。该接收模块902具体详细的处理功能或者可以执行的步骤可以参考上述图3所示实施例中的步骤304的详细描述。
所述策略配置模块904,用于根据所述的迁移策略配置请求对所述源Bucket配置对象迁移策略。该策略配置模块904具体详细的处理功能或者可以执行的步骤可以参考上述图3所示实施例中的步骤306的详细描述。
所述策略获取模块906,用于获取对象迁移策略,所述对象迁移策略表示对象从所述多个Bucket中的源Bucket迁移至目的Bucket的条件,所述对象迁移策略对应所述源Bucket;该策略获取模块具体详细的处理功能或者可以执行的步骤可以参考上述图2所示实施例中的步骤202的详细描述。
所述策略判断模块908,用于确定所述源Bucket中的第一对象是否满足所述对象迁移策略;该策略判断模块908具体详细的处理功能或者可以执行的步骤可以参考上述图2所述实施例中的步骤204、206以及210的详细描述、图6所示实施例中的步骤604-610的详细描述、图7所示实施例中的步骤704-706的详细描述、以及图8所示的实施例中的步骤804-810的详细描述。
所述迁移模块910,用于在所述第一对象满足所述对象迁移策略的情况下,将所述第一对象迁移至所述目的Bucket中;该迁移模块910具体详细的处理功能或者可以执行的步骤可以参考上述图2所示实施例中的步骤206的详细描述、图6所示实施例中的步骤612-614中的详细描述、图7所示实施例中的步骤708-710的详细描述、以及图8所示的实施例中的步骤812-814的详细描述。
发送模块912,用于在对象迁移成功后,通知所述所述存储客户端所述对象迁移的结果;该策发送模块912具体详细的处理功能或者可以执行的步骤可以参考上述图2所示实施例中的步骤208以及上述图3所示实施例中的步骤308的详细描述。
需要说明的是,图9所示的对象存储服务器中的模块划分以及各模块间的交互关系只是一种举例,在具体实现时,可以基于图1-图8所示的方法实施例对所述对象存储服务器内部的模块进行合并或者按相关功能进行重新划分。
如图10所示,图10为本申请实施例提供的一种存储客户端的结构示意图。需要说明的是,本申请实施例中的存储客户端可以通过软件实现,相应的,该存储客户端为软件模块,例如软件开发工具包(software development kit,SDK);也可以通过硬件实现,相应的,该存储客户端为硬件形态的存储客户端,例如个人PC机、移动终端、固定终端设备、智能卡、芯片、电路装置、或者瘦终端;还可以通过软件件结合实现,相应的,该存储客户端为安装有能够实现上述图1-8中所述的实施例中由该存储客户端执行的操作的软件模块或计算机程序的物理硬件形态的存储客户端,例如内置软件模块或计算机程序的各种带有处理器的存储客户端,例如内置软件模块或计算机程序的个人PC机、移动终端、固定终端设备、智能卡、芯片、电路装置、或者瘦终端。
所述存储客户端可以应用在上述图1-8所示的实施例中,所述存储客户端使用对象存储服务器管理多个桶Bucket中的至少另个Bucket,每个所述Bucket包括一个或多个对象,所述存储客户端包括:迁移策略确定模块1002、发送模块1004以及接收模块1006。
其中,所述迁移策略确定模块1002,用于确定迁移策略参数,所述策略参数包括对象迁移策略所对应的源Bucket的Bucket标识、对象的迁移属性、所述迁移属性的属性值、以及目的Bucket的Bucket标识。该迁移策略确定模块1002具体详细的处理功能或者可以执行的步骤可以参考上述图3所述实施例中的步骤302的详细描述。
所述发送模块1004,用于向所述对象存储服务器发送迁移策略配置请求以请求所述对象存储服务器对所述源Bucket进行对象迁移策略的配置,所述迁移策略配置请求包括所述迁移策略参数,所述对象迁移策略表示对象从所述源Bucket迁移至目的Bucket的条件;所述对象的迁移属性包括所述对象的重要程度或所述对象被访问的频率,或者在所述多个Bucket分 布在多个不同地区Region的情况下,所述对象的迁移属性包括对象被某一Region内的访问请求者访问的频度,所述某一Region不同于所述目的Bucket所归属的Region。该发送模块1004具体详细的处理功能或者可以执行的步骤可以参考上述图3所述实施例中的步骤304的详细描述。
所述接收模块1006,用于接收所述对象存储服务器返回的对象迁移策略配置成功的响应消息。该接收模块1006具体详细的处理功能或者可以执行的步骤可以参考上述图3所述实施例中的步骤308的详细描述。另外的实施例中,所述接收模块还可以用于接收所述对象存储服务器发送的通知消息,以通知所述存储客户端所述对象迁移的结果,具体可以参考上述图2所示实施例中的步骤208中的详细描述。
另外的具体实施例中,所述迁移策略确定模块1002可以根据所述存储客户端所使用的各Bucket的配置情况,针对所述源Bucket确定所述迁移策略参数,其中所述Bucket的配置情况包括所述各Bucket的增值功能或服务;所述Bucket的增值功能或服务包括至少下述之一:网络加速、数据高可靠性、存储的Qos、存储的性能、数据的就近访问或数据的多版本控制。
在另外的具体实施例中,所述存储客户端还包括Bucket配置接口1008,所述发送模块1006通过所述Bucket配置接口1008向所述对象存储服务器发送所述迁移策略配置请求。
应该理解到,以上所描述的装置实施例仅仅是示意性的,例如,所述模块的划分,仅仅为一种逻辑功能划分,实际实现时可以有另外的划分方式,例如多个模块、单元或组件可以结合或者其中的某个模块可以进一步划分成不同的功能模块。例如上述实施例中的对象存储服务器中的策略配置模块的功能也可能合并到所述策略获取模块,或者将所述策略配置模块、策略获取模块和策略判断模块合并在一个模块中。此外,需要说明的是,图中显示或描述的模块或设备之间的耦合或通信连接可以是通过一些接口、装置或单元而构成的间接耦合或通信连接,也可以是电性,机械或其它的形式的耦合或连接。
对于其中作为分离部件说明的模块可以是物理上分开的,也可以是物理上在同一个物理部件中。以模块命名的部件可以是硬件单元、也可以是软件模块或者逻辑单元、或者是硬件和软件的结合,该模块既可以位于一个网元内,或者也可以分布到多个网元上。这些都可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。
图11为本申请实施例中所涉及的对象存储服务器1100的一种可能的结构示意图。该对象存储服务器1100可以应用在上述图1-8所示的实施例中。在本实施例中,对象存储服务器的功能或操作步骤由一个通用的计算机或服务器中的一个或多个处理器通过执行存储器中的程序代码来实施。在这种实施方式下,该对象存储服务器1100包括:收发器1110、处理器1120、随机存取存储器1140、只读存储器1150以及总线1160。
其中,处理器1120通过总线1160分别耦接收发器1110、随机存取存储器1140以及只读存储器1150。
处理器1120可以是一个通用中央处理器(CPU),微处理器,特定应用集成电路(application-specific integrated circuit,ASIC),或一个或多个用于控制本发明方案程序执行的集成电路。
总线1160可包括一通路,在上述组件之间传送信息。
收发器1110,用于与其他设备或通信网络通信,如以太网,无线接入网(RAN),无线局域网(Wireless Local Area Networks,WLAN)等,在本发明实施例中,收发器1110可用于与存储客户端进行通信。
随机存取存储器1140可加载实现图2-图8所示实施例中的对象存储服务器的应用程序代 码,并由处理器1120来控制执行。
当需要运行对象存储服务器1100时,通过固化在只读存储器1150中的基本输入输出系统或者嵌入式系统中的boot loader引导系统进行启动,引导对象存储服务器1100进入正常运行状态。在对象存储服务器1100进入正常运行状态后,处理器1120在随机存取存储器1140中运行应用程序和操作系统,使得对象存储服务器1100可执行图2-图8所示实施例中的功能和操作。
其中,与存储客户端备的交互由收发器1110在处理器1120的控制下完成,对象存储服务器1100的内部处理(例如,对象迁移策略的配置、对象迁移策略的获取、对象迁移策略的判断、以及对象的迁移处理等)由处理器1120完成。
需要说明的是,上述实施方式中,除了上述几种通过处理器执行存储器上的程序代码指令方式等常规方式之外,本实施方式也可以基于物理服务器结合网络功能虚拟化NFV技术实现的虚拟对象存储服务器,所述虚拟对象存储服务器可以为虚拟网络设备或者服务器。本领域技术人员通过阅读本申请即可结合NFV技术在物理服务器上虚拟出具有上述功能的多个对象存储服务器。此处不再赘述。
图12为本申请实施例中所涉及的存储客户端1200的一种可能的结构示意图。该存储客户端1200可以应用在上述图1-8所示的实施例中。在本实施例中,存储客户端1200的功能或操作步骤由一个通用的计算机或服务器中的一个或多个处理器通过执行存储器中的程序代码来实施。在这种实施方式下,该对存储客户端1200可以包括:收发器1210、处理器1220、随机存取存储器1240、只读存储器1250以及总线1260。
其中,处理器1220通过总线1260分别耦接收发器1210、随机存取存储器1240以及只读存储器1250。
处理器1220可以是一个通用中央处理器(CPU),微处理器,特定应用集成电路(application-specific integrated circuit,ASIC),或一个或多个用于控制本发明方案程序执行的集成电路。
总线1260可包括一通路,在上述组件之间传送信息。
收发器1210,用于与其他设备或通信网络通信,如以太网,无线接入网(RAN),无线局域网(Wireless Local Area Networks,WLAN)等,在本发明实施例中,收发器1210可用于与对象存储服务器进行通信。
随机存取存储器1240可加载实现图2-图8所示实施例中的存储客户端的应用程序代码,并由处理器1220来控制执行。
当需要运行存储客户端1200时,通过固化在只读存储器1250中的基本输入输出系统或者嵌入式系统中的boot loader引导系统进行启动,引导存储客户端1200进入正常运行状态。在存储客户端1200入正常运行状态后,处理器1220在随机存取存储器1240中运行应用程序和操作系统,使得存储客户端1200可执行图2-图8所示实施例中的功能和操作。
其中,与对象存储服务器的交互由收发器1210在处理器1220的控制下完成,存储客户端1200的内部处理(例如,确定迁移策略参数、生成迁移策略配置请求消息等)由处理器1220完成。
需要说明的是,上述实施方式中,除了上述几种通过处理器执行存储器上的程序代码指令方式等常规方式之外,本实施方式也可以基于物理服务器结合网络功能虚拟化NFV技术实现的虚拟存储客户端,所述虚拟存储客户端可以为虚拟网络设备或者计算机。本领域技术人员通过阅读本申请即可结合NFV技术在物理服务器上虚拟出具有上述功能的多个存储客户端。此 处不再赘述。
本发明实施例还提供了一种计算机存储介质,用于储存为上述对象存储服务器所用的计算机软件指令,其包含用于执行上述图2-8所示的实施例中的对象存储服务器的功能所涉及的程序。
本发明实施例还提供了一种计算机存储介质,用于储存为上述存储客户端所用的计算机软件指令,其包含用于执行上述图2-8所示的实施例中的存储客户端的功能所涉及的程序。
尽管在此结合各实施例对本发明进行了描述,然而,在实施所要求保护的本发明过程中,本领域技术人员通过查看所述附图、公开内容、以及所附权利要求书,可理解并实现所述公开实施例的其他变化。在权利要求中,“包括”(comprising)一词不排除其他组成部分或步骤,“一”或“一个”不排除多个的情况。单个处理器或其他单元可以实现权利要求中列举的若干项功能。相互不同的从属权利要求中记载了某些措施,但这并不表示这些措施不能组合起来产生良好的效果。
本领域技术人员应明白,本申请的实施例可提供为方法、装置(设备)、或计算机程序产品。因此,本申请实施例可采用硬件实施例、软件实施例、或软件和硬件相结合的实施例的形式。而且,本申请可采用在一个或多个其中包含有计算机可用程序代码的计算机可用存储介质(包括但不限于磁盘存储器、CD-ROM、光学存储器等)上实施的计算机程序产品的形式。计算机程序存储/分布在合适的介质中,与其它硬件一起提供或作为硬件的一部分,也可以采用其他分布形式,如通过Internet或其它有线或无线电信系统。
本发明是参照本发明实施例的方法、装置(设备)和计算机程序产品的流程图和/或方框图来描述的。应理解可由计算机程序指令实现流程图和/或方框图中的每一流程和/或方框、以及流程图和/或方框图中的流程和/或方框的结合。可提供这些计算机程序指令到通用计算机、专用计算机、嵌入式处理机或其他可编程数据处理设备的处理器以产生一个机器,使得通过计算机或其他可编程数据处理设备的处理器执行的指令产生用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的装置。
这些计算机程序指令也可存储在能引导计算机或其他可编程数据处理设备以特定方式工作的计算机可读存储器中,使得存储在该计算机可读存储器中的指令产生包括指令装置的制造品,该指令装置实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能。这些计算机程序指令也可装载到计算机或其他可编程数据处理设备上,使得在计算机或其他可编程设备上执行一系列操作步骤以产生计算机实现的处理,从而在计算机或其他可编程设备上执行的指令提供用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的步骤。
尽管结合具体特征及其实施例对本发明进行了描述,显而易见的,在不脱离本发明的精神和范围的情况下,可对其进行各种修改和组合。相应地,本说明书和附图仅仅是所附权利要求所界定的本发明的示例性说明,且视为已覆盖本发明范围内的任意和所有修改、变化、组合或等同物。显然,本领域的技术人员可以对本发明进行各种改动和变型而不脱离本发明的精神和范围。这样,倘若本发明的这些修改和变型属于本发明权利要求及其等同技术的范围之内,则本发明也意图包含这些改动和变型在内。

Claims (30)

  1. 一种对象迁移方法,应用于存储系统,所述存储系统包括对象存储服务器和存储客户端,所述对象存储服务器管理多个桶Bucket,每个所述Bucket包括一个或多个对象,所述存储客户端使用所述多个Bucket中的至少两个Bucket,其特征在于,所述方法包括:
    所述对象存储服务器获取对象迁移策略,所述对象迁移策略表示对象从所述多个Bucket中的源Bucket迁移至目的Bucket的条件,所述对象迁移策略对应所述源Bucket;
    所述对象存储服务器确定所述源Bucket中的第一对象是否满足所述对象迁移策略;
    所述对象存储服务器在所述第一对象满足所述对象迁移策略的情况下,将所述第一对象迁移至所述目的Bucket中。
  2. 如权利要求1所述的方法,其特征在于,所述方法还包括:
    所述对象存储服务器接收存储客户端发送的迁移策略配置请求,所述迁移策略配置请求包括所述源Bucket的标识、对象的迁移属性、所述迁移属性的属性值、以及所述目的Bucket的标识;其中所述对象的迁移属性包括所述对象的重要程度或所述对象被访问的频率,或者在所述多个Bucket分布在多个不同地区Region的情况下,所述对象的迁移属性包括对象被某一Region内的访问请求者访问的频度,所述某一Region不同于所述目的Bucket所归属的Region;
    所述对象存储服务器根据所述的迁移策略配置请求对所述源Bucket配置对象迁移策略。
  3. 如权利要求1或2所述的方法,其特征在于,在所述对象存储服务器获取对象迁移策略之前,所述方法还包括:
    所述对象存储服务器接收所述存储客户端的对象访问请求,所述对象访问请求请求访问所述源Bucket中的所述对象;
    所述对象存储服务器根据所述的对象访问请求检查所述源Bucket是否配置了所述对象迁移策略;
    所述对象存储服务器获得对象迁移策略包括:
    所述对象存储服务器在确定所述源Bucket配置了所述对象迁移策略后,读取所述对象迁移策略。
  4. 如权利要求3所述的方法,其特征在于,当所述对象访问请求请求访问所述源Bucket中的所述对象为所述第一对象时,所述对象存储服务器在将所述第一对象迁移至所述目的Bucket之前向所述存储客户端返回所述第一对象。
  5. 如权利要求1或2所述的方法,其特征在于,在所述对象存储服务器获取对象迁移策略之前,所述方法还包括:
    所述对象存储服务器根据预设的时间定期检查所述对象存储服务器管理的各所述Bucket是否分别配置了所述对象迁移策略;
    所述对象存储服务器获得对象迁移策略包括:
    所述对象存储服务器在确定所述源Bucket配置了所述对象迁移策略后,读取所述的对象迁移策略。
  6. 如权利要求1-4所述的任一方法,其特征在于,当所述对象的迁移属性为所述对象的重要程度时,所述迁移属性的属性值为所述对象存储在所述源Bucket中的时长阈值;
    所述对象存储服务器确定所述源Bucket中的第一对象是否满足所述对象迁移策略包括:
    所述对象存储服务器获取所述对象存储服务器的当前时间;
    所述对象存储服务器比较所述对象存储服务器的当前时间与所述第一对象上传至所述源Bucket的时间的时间差值;
    所述对象存储服务器确定所述时间差值是否大于或等于所述迁移属性的属性值;
    所述对象存储服务器在所述第一对象满足所述对象迁移策略的情况下,将所述第一对象迁移至所述目的Bucket中包括:
    所述对象存储服务器在确定所述时间差值大于或等于所述迁移属性的属性值后将所述第一对象迁移至所述目的Bucket中。
  7. 如权利要求1-4所述的任一方法,其特征在于,当所述对象的迁移属性为所述对象被访问的频率时,所述迁移属性的属性值为所述对象被访问的频率阈值;
    所述对象存储服务器确定所述源Bucket中的第一对象是否满足所述对象迁移策略包括:
    所述对象存储服务器统计所述源Bucket中的所述第一对象的被访问频率;
    所述对象存储服务器确定所述所述第一对象的被访问频率是否大于或等于所述迁移属性的属性值;
    所述对象存储服务器在所述第一对象满足所述对象迁移策略的情况下,将所述第一对象迁移至所述目的Bucket中包括:
    所述对象存储服务器在确定所述第一对象的被访问频率大于或等于所述迁移属性的属性值后将所述第一对象迁移至所述目的Bucket中。
  8. 如权利要求1-4所述的任一方法,其特征在于,当所述对象的迁移属性为所述对象被访问的频率时,所述迁移属性的属性值为所述对象被访问的频率阈值;
    所述对象存储服务器确定所述源Bucket中的第一对象是否满足所述对象迁移策略包括:
    所述对象存储服务器统计所述源Bucket中的所述第一对象的被访问频率;
    所述对象存储服务器确定所述第一对象的被访问频率是否小于或等于所述迁移属性的属性值;
    所述对象存储服务器在所述第一对象满足所述对象迁移策略的情况下,将所述第一对象迁移至所述目的Bucket中包括:
    所述对象存储服务器在确定所述第一对象的被访问频率小于或等于所述迁移属性的属性值后将所述第一对象迁移至所述目的Bucket中。
  9. 如权利要求7或8所述的方法,其特征在于,所述对象存储服务器在统计所述源Bucket中的所述第一对象的被访问频率之前,所述方法还包括:
    所述对象存储服务器确定是否切换统计周期,并在确定需要切换成新的统计周期之后,以所述新的统计周期统计所述源Bucket中的所述第一对象的被访问频率。
  10. 如权利要求1-4所述的任一方法,其特征在于,当所述对象的迁移属性为对象被某一Region内的访问请求者访问的频度,所述迁移属性的属性值为所述对象被所述某一Region内的访问请求者访问的频度阈值;
    所述对象存储服务器确定所述源Bucket中的第一对象是否满足所述对象迁移策略包括:
    所述对象存储服务器统计所述源Bucket中的所述第一对象的被所述某一Region内的访问请求者访问的次数;
    所述对象存储服务器统计所述源Bucket中的所述第一对象的被多个不同Region内的访问请求者访问的次数;
    所述对象存储服务器比较所述源Bucket中的所述第一对象的被所述某一Region内的访问请求者访问的次数与所述源Bucket中的所述第一对象的被多个不同Region内的访问请求 者访问的次数获得比值;
    所述对象存储服务器确定所述比值是否大于或等于所述迁移属性的属性值;
    所述对象存储服务器在所述第一对象满足所述对象迁移策略的情况下,将所述第一对象迁移至所述目的Bucket中包括:
    所述对象存储服务器在确定所述比值大于或等于所述迁移属性的属性值后将所述第一对象迁移至所述目的Bucket中,所述目的Bucket所归属的Region为所述某一Region。
  11. 如权利要求10所述的方法,其特征在于,所述对象存储服务器在统计所述源Bucket中的所述第一对象的被所述某一Region内的访问请求者访问的次数之前,所述方法还包括:
    所述对象存储服务器确定是否切换统计周期,并在确定需要切换成新的统计周期之后,以所述新的统计周期统计所述源Bucket中的所述第一对象的被所述某一Region内的访问请求者访问的次数以及所述源Bucket中的所述第一对象的被多个不同Region内的访问请求者访问的次数。
  12. 一种对象存储服务方法,所述方法应用于存储系统,所述存储系统包括对象存储服务器和存储客户端,所述对象存储服务器管理多个桶Bucket,每个所述Bucket包括一个或多个对象,所述存储客户端使用所述多个Bucket中的至少两个Bucket,其特征在于,所述方法包括:
    所述存储客户端确定迁移策略参数,所述迁移策略参数包括对象迁移策略所对应的源Bucket的标识、对象的迁移属性、所述迁移属性的属性值、以及目的Bucket的标识;
    所述存储客户端向所述对象存储服务器发送迁移策略配置请求以请求所述对象存储服务器对所述源Bucket进行对象迁移策略的配置,所述迁移策略配置请求包括所述迁移策略参数,所述对象迁移策略表示对象从所述源Bucket迁移至目的Bucket的条件;
    所述存储客户端接收所述对象存储服务器返回的对象迁移策略配置成功的消息。
  13. 如权利要求12所述的方法,其特征在于,所述对象的迁移属性包括所述对象的重要程度或所述对象被访问的频率,或者在所述多个Bucket分布在多个不同地区Region的情况下,所述对象的迁移属性包括对象被某一Region内的访问请求者访问的频度,所述某一Region不同于所述目的Bucket所归属的Region。
  14. 如权利要求12或13所述的方法,其特征在于,所述方法进一步包括:
    所述存储客户端根据所述存储客户端所使用的各Bucket的配置情况,针对所述源Bucket确定所述迁移策略参数,其中所述Bucket的配置情况包括所述各Bucket的增值功能或服务。
  15. 如权利要求14所述的方法,其特征在于,所述Bucket的增值功能或服务包括至少下述之一:网络加速、数据高可靠性、存储的Qos、存储性能、数据的就近访问或数据的多版本控制。
  16. 如权利要求14或15所述的方法,其特征在于,所述方法进一步包括:
    所述存储客户端根据所述Bucket的增值功能或服务的变化更新所述迁移策略参数,并向所述对象存储服务器发送迁移策略配置更新请求以请求所述对象存储服务器对所述源Bucket配置的对象迁移策略进行更新。
  17. 如权利要求13-16所述的任一方法,其特征在于,所述存储客户端通过所述存储客户端与所述对象存储服务器之间的Bucket配置接口向所述对象存储服务器发送所述迁移策略配置请求。
  18. 一种硬件形态的对象存储服务器,所述对象存储服务器管理多个桶Bucket,每个所 述Bucket包括一个或多个对象,其特征在于:
    所述对象存储服务器包括策略获取模块、策略判断模块以及迁移模块;
    所述策略获取模块,用于获取对象迁移策略,所述对象迁移策略表示对象从所述多个Bucket中的源Bucket迁移至目的Bucket的条件,所述对象迁移策略对应所述源Bucket;
    所述策略判断模块,用于确定所述源Bucket中的第一对象是否满足所述对象迁移策略;
    所述迁移模块,用于在所述第一对象满足所述对象迁移策略的情况下,将所述第一对象迁移至所述目的Bucket中。
  19. 如权利要求18所述的对象存储服务器,其特征在于,所述对象存储服务器还包括接收模块和策略配置模块;
    所述接收模块,用于接收存储客户端发送的迁移策略配置请求,所述迁移策略配置请求包括所述源Bucket的标识、对象的迁移属性、所述迁移属性的属性值、以及所述目的Bucket的标识;其中所述对象的迁移属性包括所述对象的重要程度或所述对象被访问的频率,或者在所述多个Bucket分布在多个不同地区Region的情况下,所述对象的迁移属性包括对象被某一Region内的访问请求者访问的频度,所述某一Region不同于所述目的Bucket所归属的Region;
    所述策略配置模块,用于根据所述的迁移策略配置请求对所述源Bucket配置对象迁移策略。
  20. 如权利要求19所述的对象存储服务器,其特征在于,所述接收模块进一步用于接收所述存储客户端的对象访问请求,所述对象访问请求请求访问所述源Bucket中的所述对象;
    所述策略获取模块,进一步用于根据所述的对象访问请求检查所述源Bucket是否配置了所述对象迁移策略,并在确定所述源Bucket配置了所述对象迁移策略后,读取所述的对象迁移策略。
  21. 如权利要求18-20所述的任一对象存储服务器,其特征在于,当所述对象的迁移属性为所述对象的重要程度时,所述迁移属性的属性值为所述对象存储在所述源Bucket中的时长阈值;
    所述策略判断模块确定所述源Bucket中的第一对象是否满足所述对象迁移策略包括:
    所述策略判断模块确定所述第一对象为重要对象;
    所述策略判断模块获取所述对象存储服务器的当前时间;
    所述策略判断模块比较所述对象存储服务器的当前时间与所述第一对象上传至所述源Bucket的时间的时间差值;
    所述策略判断模块确定所述时间差值是否大于或等于所述迁移属性的属性值;
    所述迁移模块在所述第一对象满足所述对象迁移策略的情况下,将所述第一对象迁移至所述目的Bucket中包括:
    所述迁移模块在确定所述时间差值大于或等于所述迁移属性的属性值后将所述第一对象迁移至所述目的Bucket中。
  22. 如权利要求18-20所述的任一对象存储服务器,其特征在于,当所述对象的迁移属性为所述对象被访问的频率时,所述迁移属性的属性值为所述对象被访问的频率阈值;
    所述策略判断模块确定所述源Bucket中的第一对象是否满足所述对象迁移策略包括:
    所述策略判断模块统计所述源Bucket中的所述第一对象的被访问频率;
    所述策略判断模块确定所述所述第一对象的被访问频率是否大于或等于所述迁移属性的属性值;
    所述迁移模块在所述第一对象满足所述对象迁移策略的情况下,将所述第一对象迁移至所述目的Bucket中包括:
    所述迁移模块在确定所述第一对象的被访问频率大于或等于所述迁移属性的属性值后将所述第一对象迁移至所述目的Bucket中。
  23. 如权利要求18-20所述的任一对象存储服务器,其特征在于,当所述对象的迁移属性为对象被某一Region内的访问请求者访问的频度,所述迁移属性的属性值为所述对象被所述某一Region内的访问请求者访问的频度阈值;
    所述策略判断模块确定所述源Bucket中的第一对象是否满足所述对象迁移策略包括:
    所述策略判断模块统计所述源Bucket中的所述第一对象的被所述某一Region内的访问请求者访问的次数;
    所述策略判断模块统计所述源Bucket中的所述第一对象的被多个不同Region内的访问请求者访问的次数;
    所述策略判断模块比较所述源Bucket中的所述第一对象的被所述某一Region内的访问请求者访问的次数与所述源Bucket中的所述第一对象的被多个不同Region内的访问请求者访问的次数获得比值;
    所述策略判断模块确定所述比值是否大于或等于所述迁移属性的属性值;
    所述迁移模块在所述第一对象满足所述对象迁移策略的情况下,将所述第一对象迁移至所述目的Bucket中包括:
    所述迁移模块在在确定所述比值大于或等于所述迁移属性的属性值后将所述第一对象迁移至所述目的Bucket中,所述目的Bucket所归属的Region为所述某一Region。
  24. 一种硬件形态的存储客户端,所述存储客户端使用对象存储服务器管理多个桶Bucket中的至少另个Bucket,每个所述Bucket包括一个或多个对象,其特征在于,所述存储客户端包括:
    迁移策略确定模块、发送模块以及接收模块;
    所述迁移策略确定模块,用于确定迁移策略参数,所述迁移策略参数包括对象迁移策略所对应的源Bucket的标识、对象的迁移属性、所述迁移属性的属性值、以及目的Bucket的Bucket标识;
    所述发送模块,用于向所述对象存储服务器发送迁移策略配置请求以请求所述对象存储服务器对所述源Bucket进行对象迁移策略的配置,所述迁移策略配置请求包括所述迁移策略参数,所述对象迁移策略表示对象从所述源Bucket迁移至目的Bucket的条件;
    所述接收模块,用于接收所述对象存储服务器返回的对象迁移策略配置成功的响应消息。
  25. 如权利要求24所述的存储客户端,其特征在于,所述对象的迁移属性包括所述对象的重要程度或所述对象被访问的频率,或者在所述多个Bucket分布在多个不同地区Region的情况下,所述对象的迁移属性包括对象被某一Region内的访问请求者访问的频度,所述某一Region不同于所述目的Bucket所归属的Region。
  26. 如权利要求24或25所述的存储客户端,其特征在于,所述迁移策略确定模块根据所述存储客户端所使用的各Bucket的配置情况,针对所述源Bucket确定所述迁移策略参数,其中所述Bucket的配置情况包括所述各Bucket的增值功能或服务;所述Bucket的增值功能或服务包括至少下述之一:网络加速、数据高可靠性、存储的Qos、数据的就近访问或数据的多版本控制。
  27. 如权利要求23-26所述的任一存储客户端,其特征在于,所述存储客户端还包括Bucket 配置接口,所述发送模块通过所述Bucket配置接口向所述对象存储服务器发送所述迁移策略配置请求。
  28. 一种对象存储服务系统,其特征在于,所述对象存储服务系统包括对象存储服务器、数据持久化组件、元数据数据库、以及多个硬盘;其中,所述对象存储服务器,用于管理多个桶Bucket,每个所述Bucket包括一个或多个对象,接收存储客户端的发送的携带迁移策略参数的迁移策略配置请求,根据所述的迁移策略参数为源Bucket配置对象迁移策略,并将所述对象迁移策略保存在所述元数据数据库中,以及根据所述对象迁移策略对所述源Bucket中的满足所述对象迁移策略的对象进行迁移处理;所述迁移策略参数包括对象迁移策略所对应的源Bucket的标识、对象的迁移属性、所述迁移属性的属性值、以及目的Bucket的标识;数据持久化组件用于存储所述多个Bucket;所述元数据数据库用于保存所述对象迁移策略;所述多个硬盘用于提供物理存储空间。
  29. 一种对象存储服务系统,其特征在于,所述对象存储服务系统包括如权利要求18-23所述的任一对象存储服务器、数据持久化组件、元数据数据库、以及多个硬盘;数据持久化组件用于保存所述多个Bucket;所述元数据数据库用于保存所述对象迁移策略;所述多个硬盘用于提供物理存储空间。
  30. 一种存储系统,其特征在于,所述存储系统包括如权利要求18-23所述的任一对象存储服务器以及如权利要求24-27所述的任一存储客户端。
PCT/CN2017/119464 2017-12-28 2017-12-28 一种对象迁移的方法、设备和系统 WO2019127234A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
PCT/CN2017/119464 WO2019127234A1 (zh) 2017-12-28 2017-12-28 一种对象迁移的方法、设备和系统
US16/911,974 US11573725B2 (en) 2017-12-28 2020-06-25 Object migration method, device, and system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2017/119464 WO2019127234A1 (zh) 2017-12-28 2017-12-28 一种对象迁移的方法、设备和系统

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US16/911,974 Continuation US11573725B2 (en) 2017-12-28 2020-06-25 Object migration method, device, and system

Publications (1)

Publication Number Publication Date
WO2019127234A1 true WO2019127234A1 (zh) 2019-07-04

Family

ID=67064337

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2017/119464 WO2019127234A1 (zh) 2017-12-28 2017-12-28 一种对象迁移的方法、设备和系统

Country Status (2)

Country Link
US (1) US11573725B2 (zh)
WO (1) WO2019127234A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111475117A (zh) * 2020-04-09 2020-07-31 南京信息职业技术学院 一种聚合云存储QoS控制方法及系统
WO2024008024A1 (zh) * 2022-07-04 2024-01-11 华为云计算技术有限公司 一种基于对象存储服务的跨区域转储方法及相关装置

Families Citing this family (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11231876B2 (en) * 2019-09-24 2022-01-25 Western Digital Technologies, Inc. Initial data placement in hybrid object storage system
US11461123B1 (en) * 2019-11-21 2022-10-04 Amazon Technologies, Inc. Dynamic pre-copy and post-copy determination for live migration between cloud regions and edge locations
US11734038B1 (en) 2019-11-21 2023-08-22 Amazon Technologies, Inc Multiple simultaneous volume attachments for live migration between cloud regions and edge locations
US11573839B1 (en) 2019-11-21 2023-02-07 Amazon Technologies, Inc. Dynamic scheduling for live migration between cloud regions and edge locations
US11403134B2 (en) * 2020-01-31 2022-08-02 Hewlett Packard Enterprise Development Lp Prioritizing migration of data associated with a stateful application based on data access patterns
CN112187738A (zh) * 2020-09-11 2021-01-05 中国银联股份有限公司 业务数据访问控制方法、装置及计算机可读存储介质
CN112328573A (zh) * 2020-11-03 2021-02-05 杭州朗澈科技有限公司 kubernetes集群中数据库迁移方法和系统
US20230161505A1 (en) * 2021-11-22 2023-05-25 Nutanix, Inc. System and method for shallow copy
CN116266839A (zh) * 2021-12-16 2023-06-20 华为云计算技术有限公司 一种对象存储桶的数据访问方法以及云管理平台

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101251797A (zh) * 2008-03-31 2008-08-27 中国船舶重工集团公司第七〇九研究所 基于域模型的构件实时主动迁移方法
US20140108723A1 (en) * 2012-10-17 2014-04-17 Datadirect Networks, Inc. Reducing metadata in a write-anywhere storage system
CN104123359A (zh) * 2014-07-17 2014-10-29 江苏省邮电规划设计院有限责任公司 一种分布式对象存储系统的资源管理方法
US20150286701A1 (en) * 2014-04-04 2015-10-08 Quantum Corporation Data Classification Aware Object Storage
CN105843554A (zh) * 2016-03-11 2016-08-10 中国联合网络通信集团有限公司 基于对象存储的数据迁移的方法及其系统

Family Cites Families (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2008112291A (ja) * 2006-10-30 2008-05-15 Hitachi Ltd 記憶制御装置及び記憶制御装置のデータ移動方法
JP5331555B2 (ja) * 2009-04-23 2013-10-30 株式会社日立製作所 データ移行システムおよびデータ移行方法
WO2011055412A1 (en) * 2009-11-06 2011-05-12 Hitachi, Ltd. File management sub-system and file migration control method in hierarchical file system
WO2012004837A1 (en) * 2010-07-09 2012-01-12 Hitachi, Ltd. Storage apparatus and storage management method
US9116914B1 (en) * 2011-04-18 2015-08-25 American Megatrends, Inc. Data migration between multiple tiers in a storage system using policy based ILM for QOS
CN102521138B (zh) * 2011-11-28 2015-11-25 华为技术有限公司 数据迁移触发方法和装置
US20160041996A1 (en) * 2014-08-11 2016-02-11 Netapp, Inc. System and method for developing and implementing a migration plan for migrating a file system
WO2015186248A1 (ja) * 2014-06-06 2015-12-10 株式会社日立製作所 ストレージシステム、計算機システム及びデータ移行方法
JP2016066220A (ja) * 2014-09-24 2016-04-28 富士通株式会社 ストレージ装置、ストレージ装置制御方法及びストレージ装置制御プログラム
US20180246659A1 (en) * 2017-02-28 2018-08-30 Hewlett Packard Enterprise Development Lp Data blocks migration
US10642809B2 (en) * 2017-06-26 2020-05-05 International Business Machines Corporation Import, export, and copy management for tiered object storage

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101251797A (zh) * 2008-03-31 2008-08-27 中国船舶重工集团公司第七〇九研究所 基于域模型的构件实时主动迁移方法
US20140108723A1 (en) * 2012-10-17 2014-04-17 Datadirect Networks, Inc. Reducing metadata in a write-anywhere storage system
US20150286701A1 (en) * 2014-04-04 2015-10-08 Quantum Corporation Data Classification Aware Object Storage
CN104123359A (zh) * 2014-07-17 2014-10-29 江苏省邮电规划设计院有限责任公司 一种分布式对象存储系统的资源管理方法
CN105843554A (zh) * 2016-03-11 2016-08-10 中国联合网络通信集团有限公司 基于对象存储的数据迁移的方法及其系统

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111475117A (zh) * 2020-04-09 2020-07-31 南京信息职业技术学院 一种聚合云存储QoS控制方法及系统
CN111475117B (zh) * 2020-04-09 2024-01-23 南京信息职业技术学院 一种聚合云存储QoS控制方法及系统
WO2024008024A1 (zh) * 2022-07-04 2024-01-11 华为云计算技术有限公司 一种基于对象存储服务的跨区域转储方法及相关装置

Also Published As

Publication number Publication date
US11573725B2 (en) 2023-02-07
US20200326876A1 (en) 2020-10-15

Similar Documents

Publication Publication Date Title
WO2019127234A1 (zh) 一种对象迁移的方法、设备和系统
US9971823B2 (en) Dynamic replica failure detection and healing
CN111522636B (zh) 应用容器的调整方法、调整系统、计算机可读介质及终端设备
US20170220564A1 (en) On-demand mailbox synchronization and migration system
US10771533B2 (en) Adaptive communication control device
WO2020192598A1 (zh) 一种部署虚拟机和容器的方法及装置
CN106775946B (zh) 一种虚拟机快速创建方法
WO2018121334A1 (zh) 一种提供网页应用服务的方法、装置、电子设备及系统
WO2017054197A1 (zh) 一种扩展联动的方法、装置及系统
US9672069B2 (en) Dynamic zone-based cloud management
CN110677462B (zh) 多区块链网络的访问处理方法、系统、装置及存储介质
WO2020042612A1 (zh) 消息存储、读取方法及装置、服务器、存储介质
CN110289993B (zh) 一种连接池管理方法、装置、设备及转接系统
CN108667744B (zh) 流量控制方法及装置
US9875137B2 (en) Intelligent application back stack management
WO2012053393A1 (ja) 仮想計算機を配置する方法及び装置
JP2015191523A (ja) 構成管理装置、構成管理システム、および構成管理プログラム
CN111582824B (zh) 云资源同步方法、装置、设备及存储介质
CN106911741B (zh) 一种虚拟化网管文件下载负载均衡的方法及网管服务器
WO2018170732A1 (zh) 一种边缘云架构下服务部署的方法和装置
WO2017070963A1 (zh) 一种虚拟资源的部署方法、装置及系统
WO2016179803A1 (zh) 建立vnfm与vim之间的连接的方法、装置及系统
WO2024113847A1 (zh) 共享资源分配方法、装置及计算设备集群
EP2975516B1 (en) Intelligent application back stack management
CN117081931B (zh) 一种异构分布式存储系统在线扩容方法及装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17936149

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 17936149

Country of ref document: EP

Kind code of ref document: A1