WO2017041650A1 - 用于扩展分布式一致性服务的方法和设备 - Google Patents
用于扩展分布式一致性服务的方法和设备 Download PDFInfo
- Publication number
- WO2017041650A1 WO2017041650A1 PCT/CN2016/097494 CN2016097494W WO2017041650A1 WO 2017041650 A1 WO2017041650 A1 WO 2017041650A1 CN 2016097494 W CN2016097494 W CN 2016097494W WO 2017041650 A1 WO2017041650 A1 WO 2017041650A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- migration
- service
- service partition
- data resource
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Ceased
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/40—Support for services or applications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
Definitions
- the present application relates to the field of computers, and in particular, to a method and apparatus for extending a distributed consistency service.
- Each update operation requires more than half of the service components to agree that an update operation can be considered successful.
- the update operation is very frequent, there is no inherent way to simply and effectively improve the service capabilities of the system.
- a simple method of adding a service component does not improve the processing power of the service, but also affects the response time of the update operation.
- distributed locks have extremely demanding requirements for the failure time of their own services.
- the data size of the service is gradually increased, more and more access requests are generated, and the stored data resources are getting larger and larger, and the failure time will follow. With the simultaneous growth, the availability of services can be very challenging.
- the technical problem to be solved by the present application is to provide a method and device for extending a distributed consistency service based on a service partition.
- a method for extending a distributed consistency service includes:
- the partial data resources stored in the service partition that are over-loaded are migrated to the migration service partition, and after the migration is completed, the access request corresponding to the migrated data resource is processed by the migration service partition.
- Partitions include:
- the pre-migration data resources are migrated to the migration service partition in a replication manner to form a post-migration data resource.
- migrating part of the data resources stored in the service partition that is overloaded to the migration service partition further includes:
- the migrated data resource is set to an invalid state during the migration process
- the pre-migration data resource is set to a read-only state when the migration is about to be completed
- the pre-migration data resource is set to an unreadable and non-writable state, and the migrated data is set to a read-only state.
- processing, by the migration service partition, the access request corresponding to the migrated data resource after the migration is completed includes:
- the state of the pre-migration data resource and the post-migration data resource is reset to a readable and writable state, and the corresponding identification file in the migration data that has not obtained the change operation request of the user is deleted.
- the identification file includes a distributed lock file.
- the service component of the service partition maintains consistency using a majority mechanism.
- the migration service partition is an existing idle service partition or a newly configured service partition.
- a device for extending a distributed consistency service includes:
- a configuration device configured to configure a plurality of service partitions, and process, according to a data resource distribution rule of the service partition, a corresponding access request by the service partition, wherein each of the service partitions has a plurality of consistent service components;
- a migration device configured to migrate part of the data resources stored by the service partition that is excessively processed to the migration service partition, and after the migration is completed, processing, by the migration service partition, the access corresponding to the migrated data resource request.
- the migration device is used to:
- the pre-migration data resources are migrated to the migration service partition in a replication manner to form a post-migration data resource.
- the migration device further includes:
- the migrated data resource is set to an invalid state during the migration process
- the pre-migration data resource is set to a read-only state when the migration is about to be completed
- the pre-migration data resource is set to an unreadable and non-writable state, and the migrated data is set to a read-only state.
- the migration device is configured to: after the migration is completed,
- the state of the pre-migration data resource and the post-migration data resource is reset to a readable and writable state, and the corresponding identification file in the migration data that has not obtained the change operation request of the user is deleted.
- the identification file includes a distributed lock file.
- the migration device is used to:
- the pre-migration data resource is deleted when the state of the pre-migration data resource and the post-migration data resource is reset to a readable and writable state and the timeout period is exceeded.
- the service component of the service partition maintains consistency using a majority mechanism.
- the migration service partition is an existing idle service partition or a newly configured service partition.
- the method for extending a distributed consistency service and the device provided by another aspect provided by one aspect of the present application, by deploying multiple sets of service partitions including distributed lock service components, each group
- the service partition stores part of the data resources and provides corresponding access requests, and dynamically adjusts the data resources of each partition service at runtime to balance the processing load pressure between each group of service partitions, thereby improving the extension of the distributed consistency service. ability.
- FIG. 1 shows a flow chart of a method for extending a distributed consistency service, provided in accordance with an aspect of the present application
- FIG. 2 to FIG. 4 are schematic diagrams showing a migration process for extending a distributed consistency service according to a preferred embodiment of the present application
- FIG. 5 shows a schematic diagram of an apparatus for extending a distributed consistency service, according to an aspect of the present application.
- distributed conformance services are generally implemented based on a C/S architecture (customer/service architecture), and services are generally served by multiple nodes formed by multiple service components to ensure consistency and availability; Provides an interface to the user or application to access the distributed consistency service.
- Distributed consistency services include distributed locks, which are a way to control the simultaneous access of shared resources between distributed systems. If different systems or different hosts of the same system share the data resources of one or a group of service components, then when requesting access to these data resources, mutual exclusion is often required to prevent mutual interference to ensure consistency. In this case, you need to use a distributed lock.
- the present application provides a method for a distributed consistency service based on a service partition, by configuring a service partition with a consistent service, and setting a distribution rule of data resources served by each service partition.
- distributing the data resources into multiple service partitions each service partition processing a part of the access request and storing some corresponding data resources; meanwhile, when the access request increases or the data resources increase, causing one or several service partitions
- the migration service partition is dynamically configured.
- the migration service partition may be an existing idle service partition or a newly configured service partition, and then migrate part of the data resources stored in the service partition that is overburdened to the load to The service partition is migrated, and after the migration is completed, the access request corresponding to the migrated data resource is processed by the migration service partition, thereby balancing the load pressure between the multiple service partitions and ensuring service stability.
- FIG. 1 shows a flow chart of a method for extending a distributed consistency service, according to an aspect of the present application, wherein the method includes:
- Step S11 Configuring a plurality of service partitions, and processing, according to a data resource distribution rule of the service partition, a corresponding access request by the service partition, where each of the service partitions has a plurality of consistent service components;
- Step S12 The partial data resource stored in the service partition with excessive processing load is migrated to the migration service partition, and after the migration is completed, the access request corresponding to the migrated data resource is processed by the migration service partition.
- the service component of the service partition maintains consistency by a majority mechanism, and the service components in the same service partition store the same data resource.
- the access request is routed to the corresponding service partition capable of processing the access request according to the content of the access request and according to the data resource distribution rule, by the service.
- the partition completes the application for access to the user Manage, release distributed locks, set the owner information of distributed locks, and so on.
- step S12 after the distributed consistency service works for a period of time, there may be too many access requests or excessive storage data resources on one or some service partitions, resulting in excessive processing load.
- the running condition dynamically migrates part of the data resources stored in the service partition that is overloaded to the migration service partition, and after the migration is completed, processes the access corresponding to the migrated data resource by the migration service partition. request.
- migrating part of the data resources stored in the service partition with excessive processing load to the migration service partition includes:
- the part of the data resource stored in the service partition that is over-loaded is selected as the pre-migration data resource; specifically, the selected mode is not limited, and the service may be serviced according to the data amount of the data resource and the corresponding access request.
- the partition is specifically divided; the pre-migration data resource is migrated to the migration service partition in a replication manner to form a migrated data resource. By copying, it is guaranteed that users who have owned the identification file (distributed lock file) before migration will continue to provide data services during the migration process.
- the migration of the partial data resources stored in the service partition that is excessively processed to the migration service partition further includes:
- the post-migration data resource is set to an invalid state; the invalid state is set to prevent the system from deleting the post-migration data resource if the system does not obtain the access request within the data resource timeout period after the migration.
- the pre-migration data resource is set to a read-only state; at this time, the identifier file that the user already holds is not affected, that is, the pre-migration data resource can be read by using the identifier file.
- the pre-migration data resource is set to an unreadable and non-writable state
- the post-migration data is set to a read-only state
- the data resource resource-x is prepared to start providing services.
- the access request corresponding to the migrated data resource is processed by the migration service partition after the migration is completed, including:
- the state of the pre-migration data resource and the post-migration data resource is reset to a readable and writable state, and the corresponding identification file in the migration data that has not obtained the change operation request of the user is deleted.
- the identification file comprises a distributed lock file.
- the pre-migration data resource is deleted when the timeout period is reset after the state of the pre-migration data resource and the post-migration data resource is reset to a readable and writable state. , finally complete the data migration.
- FIG. 2 to FIG. 4 are schematic diagrams showing a migration process of an extended distributed consistency service according to a preferred embodiment of the present application.
- the service partition quorum-n is taken as an example to illustrate extended distribution.
- the service partition providing the service includes a service partition quorum-1 to a service partition quorum-n, wherein the service partition quorum-n has a data resource resource-n and a data resource resouce- x, all served by the service partition quorum-n service, the quorum-n service allocates a lock file lock://resource-n/foo about the data resource resource-n to the user, and assigns a lock file about the data resource resource-x to the user.
- Lock as the owner of the user can obtain the data resource resource-n related services with the lock file lock://resource-n/foo, holding the lock file lock://resource -x/foo Get the related services of the data resource resource-x.
- a migration operation (migration operation) is started, and the data related to the data resource resource-x on the service partition quorum-n is dynamically copied to the migration service partition quorum-x in real time.
- the Migration operation utilizes the distributed lock native synchronization protocol to synchronize all data and update operations to new partitions in real time. The entire process does not have any impact on the original services.
- the migrated data resource resource-x' migrated to the migration service partition quorum-x is set to an invalid state (invalid), and is set to an invalid state to prevent the system from considering that the data resource resource-x' timeout period after migration is not
- the migrated data resource resource-x' is deleted by obtaining an access request.
- the data of the data resource resource-x of the service partition quorum-n and the data of the resource resource-x' of the service partition quorum-x are basically the same, for example, the data resources to be migrated are 1024M, and The remaining 8M data to be migrated needs to be transmitted.
- the read-only status of the data resource resource-x is set on the service partition quorum-n (read- Only).
- the lock file lock://resource-x/foo that the user already holds is not affected, that is, the data resource resource-x can be continuously read by using the lock file.
- the data resource resource-x of the service partition quorum-n and the data of the resource resource resource-x' on the service partition quorum-x are completely identical, then all pairs of data are prohibited on the service partition quorum-n.
- the resource resource-x operation, and the data resource resource-x' is set to read-only on the service partition quorum-x, that is, the data resource resource-x' starts to provide services.
- Update all service partition routing rules the data resource resource-x' is served by the service partition quorum-x, and the data resource resource-x' is identical to the data resource resource-x.
- All users are notified of the owner change operation (chown operation) through the system front end.
- the user who is the original owner will initiate a chown operation to the service partition quorum-x after receiving the notification.
- the service partition quorum-x receives the chown operation from the user of the original owner, the corresponding lock file (lock://resource-x/foo) is set to be valid, and the owner information is changed to the original owner.
- the user if the user who is the original owner does not initiate a chown operation to the service partition quorum-x after receiving the notification, deletes the lock file lock that is not chowned on the service partition quorum-x after waiting for more than the preset time: //resource-x/foo, can re-release the distributed lock according to the access request of other users, and at the same time, cancel the state limit of the data resource resource-x and the data resource resource-x' after the preset time is exceeded, that is, both Read and write status. If the data resource resource-x does not receive the heartbeat message of the access request within the timeout period, the system will clear the data resource resource-x and finally complete the data migration, as shown in Figure 4.
- FIG. 5 is a schematic diagram of a device for extending a distributed consistency service according to an aspect of the present disclosure, wherein the device 1 includes:
- the configuration device 11 configures a plurality of service partitions, and processes, according to a data resource distribution rule of the service partition, a corresponding access request by the service partition, wherein each of the service partitions has a plurality of consistent service components;
- the migration device 12 migrates part of the data resources stored in the service partition with excessive processing load to the migration service partition, and after the migration is completed, processes the access request corresponding to the migrated data resource by the migration service partition. .
- the device 1 includes, but is not limited to, a user equipment, or a device formed by integrating a user equipment and a network device through a network.
- the user equipment includes, but is not limited to, any mobile electronic product that can interact with a user through a touchpad, such as a smart phone, a PDA, etc., and the mobile electronic product can adopt any operating system.
- the network device includes an electronic device capable of automatically performing numerical calculation and information processing according to an instruction set or stored in advance, and the hardware includes but is not limited to a microprocessor, an application specific integrated circuit (ASIC), and a programmable gate. Arrays (FPGAs), digital processors (DSPs), embedded devices, and more.
- the network includes, but is not limited to, the Internet, a wide area network, a metropolitan area network, a local area network, a VPN network, a wireless ad hoc network (Ad Hoc network), and the like.
- the device 1 may also be a script program running on the user equipment, or a user equipment and a network device, a touch terminal, or a device formed by integrating a network device and a touch terminal through a network.
- the above-mentioned device 1 is only an example, and other existing or future devices 1 may be applicable to the present application, and are also included in the protection scope of the present application, and are hereby incorporated by reference. Included here.
- the configuration device 11 maintains consistency with the service component of the service partition using a majority mechanism, and the service components in the same service partition store the same data resource. After the configuration is completed, when the access request reaches the front end of the distributed consistency service, the access request is routed to the corresponding service partition capable of processing the access request according to the content of the access request and according to the data resource distribution rule.
- the service partition completes the processing of the user's access request, releases the distributed lock, and sets the owner information of the distributed lock.
- the migration device 12 when the distributed consistency service is working for a period of time, there may be too many access requests or excessive storage data resources on one or some service partitions, resulting in excessive processing load. According to the running condition of the system, a part of the data resources stored in the service partition with excessive processing load are dynamically migrated to the migration service partition, and after the migration is completed, the migration service partition is processed by the migration service partition. Said access request.
- migrating part of the data resources stored in the service partition that is excessively processed to the migration service partition includes:
- the part of the data resource stored in the service partition that is over-loaded is selected as the pre-migration data resource; specifically, the selected mode is not limited, and the service may be serviced according to the data amount of the data resource and the corresponding access request.
- the partition is specifically divided; the pre-migration data resource is migrated to the migration service partition in a replication manner to form a migrated data resource. By copying, it is guaranteed that users who have owned the identification file (distributed lock file) before migration will continue to provide data services during the migration process.
- migrating part of the data resources stored in the service partition that is excessively processed to the migration service partition further includes:
- the post-migration data resource is set to an invalid state; the invalid state is set to prevent the system from deleting the post-migration data resource if the system does not obtain the access request within the data resource timeout period after the migration.
- the pre-migration data resource is set to a read-only state; at this time, the identifier file that the user already holds is not affected, that is, the pre-migration data resource can be read by using the identifier file.
- the pre-migration data resource is set to an unreadable and non-writable state
- the post-migration data is set to a read-only state
- the data resource resource-x is prepared to start providing services.
- processing, by the migration service partition, the access request corresponding to the migrated data resource after the migration is completed includes:
- the state of the pre-migration data resource and the post-migration data resource is reset to a readable and writable state, and the corresponding identification file in the migration data that has not obtained the change operation request of the user is deleted.
- the identification file comprises a distributed lock file.
- FIG. 2 to FIG. 4 are schematic diagrams showing a migration process of an extended distributed consistency service according to a preferred embodiment of the present application.
- the service partition quorum-n is taken as an example to illustrate extended distribution.
- the data resource resource-n lock file lock://resource-n/foo allocates a lock file lock://resource-x/foo for the data resource resource-x to the user, and the user as the owner can hold the lock.
- the file lock://resource-n/foo obtains the related service of the data resource resource-n, and acquires the related service of the data resource resource-x by holding the lock file lock://resource-x/foo.
- a migration operation (migration operation) is started, and the data related to the data resource resource-x on the service partition quorum-n is dynamically copied to the migration service partition quorum-x in real time.
- the Migration operation utilizes the distributed lock native synchronization protocol to synchronize all data and update operations to new partitions in real time. The entire process does not have any impact on the original services.
- the migrated data resource resource-x' migrated to the migration service partition quorum-x is set to an invalid state (invalid), and is set to an invalid state to prevent the system from considering that the data resource resource-x' timeout period after migration is not
- the migrated data resource resource-x' is deleted by obtaining an access request.
- the data of the data resource resource-x of the service partition quorum-n and the data of the resource resource-x' of the service partition quorum-x are basically the same, for example, the data resources to be migrated are 1024M, and The remaining 8M data to be migrated needs to be transmitted.
- the read-only status of the data resource resource-x is set on the service partition quorum-n (read- Only).
- the lock file lock://resource-x/foo that the user already holds is not affected, that is, the data resource resource-x can be continuously read by using the lock file.
- the data resource resource-x of the service partition quorum-n and the data of the resource resource resource-x' on the service partition quorum-x are completely identical, then all pairs of data are prohibited on the service partition quorum-n.
- the resource resource-x operation, and the data resource resource-x' is set to read-only on the service partition quorum-x, that is, the data resource resource-x' starts to provide services.
- Update all service partition routing rules the data resource resource-x' is served by the service partition quorum-x, and the data resource resource-x' is identical to the data resource resource-x.
- All users are notified of the owner change operation (chown operation) through the system front end.
- the user who is the original owner will initiate a chown operation to the service partition quorum-x after receiving the notification.
- the service partition quorum-x receives the chown operation from the user of the original owner, the corresponding lock file (lock://resource-x/foo) is set to be valid, and the owner information is changed to the original owner.
- the user if the user who is the original owner does not initiate a chown operation to the service partition quorum-x after receiving the notification, deletes the lock file lock that is not chowned on the service partition quorum-x after waiting for more than the preset time: //resource-x/foo, can re-release the distributed lock according to the access request of other users, and at the same time, cancel the state limit of the data resource resource-x and the data resource resource-x' after the preset time is exceeded, that is, both Read and write status. If the data resource resource-x does not receive the heartbeat message of the access request within the timeout period, the system will clear the data resource resource-x and finally complete the data migration, as shown in Figure 4.
- the service partition-based extended distributed consistency service device configures a service partition with a consistent service by configuring a device, and sets a service for each service partition.
- Data resource distribution rules which distribute data resources into multiple service partitions.
- Each service partition processes a part of access requests and stores some corresponding data resources.
- the migration service partition is dynamically configured by the migration device, and then part of the data resources stored in the service partition with excessive processing load are migrated to the migration service partition, and the migration is completed. Then, the access request corresponding to the migrated data resource is processed by the migration service partition, thereby balancing the load pressure between the multiple service partitions to ensure service stability.
- the present application can be implemented in software and/or a combination of software and hardware, for example, using an application specific integrated circuit (ASIC), a general purpose computer, or any other similar hardware device.
- the software program of the present application can be executed by a processor to implement the steps or functions described above.
- the software programs (including related data structures) of the present application can be stored in a computer readable recording medium such as a RAM memory, a magnetic or optical drive or a floppy disk and the like.
- some of the steps or functions of the present application may be implemented in hardware, for example, as a circuit that cooperates with a processor to perform various steps or functions.
- a portion of the present application can be applied as a computer program product, such as computer program instructions, which, when executed by a computer, can invoke or provide a method and/or technical solution in accordance with the present application.
- the program instructions for invoking the method of the present application may be stored in a fixed or removable recording medium, and/or transmitted by a data stream in a broadcast or other signal bearing medium, and/or stored in a The working memory of the computer device in which the program instructions are run.
- an embodiment in accordance with the present application includes a device including a memory for storing computer program instructions and a processor for executing program instructions, wherein when the computer program instructions are executed by the processor, triggering
- the apparatus operates based on the aforementioned methods and/or technical solutions in accordance with various embodiments of the present application.
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Multimedia (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
本申请提供了用于扩展分布式一致性服务的方法和设备,通过配置装置配置若干具有一致性服务的服务分区,并设定每一服务分区所服务的数据资源的分布规则,从而将数据资源分散到多个服务分区中,每个服务分区处理一部分访问请求并存储部分相应数据资源;同时,当访问请求增多或数据资源增大时,造成某个或某几个服务分区的处理负载过大时,则通过迁移装置动态配置迁移服务分区,然后将处理负载过大的所述服务分区所存储的部分数据资源迁移至迁移服务分区,并在迁移完成后,由所述迁移服务分区处理所迁移的数据资源对应的所述访问请求,从而平衡多个服务分区之间的负载压力,保证服务稳定性。
Description
本申请要求2015年09月09日递交的申请号为201510570429.9发明名称为“用于扩展分布式一致性服务的方法和设备”的中国专利申请的优先权,其全部内容通过引用结合在本申请中。
本申请涉及计算机领域,尤其涉及一种用于扩展分布式一致性服务的方法和设备。
在分布式系统中,通常由多个服务组件共同接受访问请求并提供服务,以保持稳定性,同时使用分布式一致性服务(例如分布式锁)来协调多个服务组件之间的关系和操作。当前主流分布式锁都会基于quorum(多数派机制,例如chubby、zookeeper)来保证一致性和可用性。现有基于quorum实现的分布式锁,一般会面临下面几个严重的问题:
每次更新操作,需要经过半数以上服务组件同意,才可以认为一次更新操作成功。当更新操作非常频繁时,没有固有的办法能够简单有效地提高系统的服务能力。典型的,简单的添加服务组件的方法,一方面不能提高服务的处理能力,同时还会影响更新操作的响应时间。此外,分布式锁对自身服务的失效时间(Failover)有着异常苛刻的要求,当服务的数据规模逐渐增大时,访问请求越来越多,存储的数据资源越来越大,失效时间会随之同步增长,服务的可用性会受到非常大的挑战。
发明内容
本申请要解决的技术问题是:提供一种基于服务分区的扩展分布式一致性服务的方法和设备。
根据本申请一方面提供的一种用于扩展分布式一致性服务的方法,其中,所述方法包括:
配置若干服务分区,并基于所述服务分区的数据资源分布规则,由所述服务分区处理相应访问请求,其中,每一所述服务分区具有若干一致的服务组件;
将处理负载过大的所述服务分区所存储的部分数据资源迁移至迁移服务分区,并在迁移完成后,由所述迁移服务分区处理所迁移的数据资源对应的所述访问请求。
进一步地,将处理负载过大的所述服务分区所存储的部分数据资源迁移至迁移服务
分区包括:
将处理负载过大的所述服务分区所存储的部分数据资源选定为迁移前数据资源;
将所述迁移前数据资源以复制方式迁移到所述迁移服务分区,形成迁移后数据资源。
进一步地,将处理负载过大的所述服务分区所存储的部分数据资源迁移至迁移服务分区还包括:
在迁移过程中,将所述迁移后数据资源设置为无效状态;
在即将完成迁移时,将所述迁移前数据资源设置为只读状态;
在完成迁移后,将所述迁移前数据资源设置为不可读不可写状态,并将所述迁移后数据设置为只读状态。
进一步地,在迁移完成后由所述迁移服务分区处理与所迁移的数据资源相应的所述访问请求包括:
更新所述数据资源分布规则;
向所有用户发起服务分区变更通知,并基于所述用户的拥有者变更操作请求,将迁移后数据资源中的标识文件设置为有效,且拥有者信息设置为相应的请求用户;
在完成迁移超出预设时间后,重置所述迁移前数据资源和所述迁移后数据资源的状态为可读可写状态,并删除迁移数据中未获得更改用户者操作请求的相应标识文件。
进一步地,所述标识文件包括分布式锁文件。
进一步地,所述服务分区的服务组件采用多数派机制维持一致性。
进一步地,所述迁移服务分区为已有空闲的服务分区或新配置的服务分区。
根据本申请一方面提供的一种用于扩展分布式一致性服务的设备,其中,所述设备包括:
配置装置,用于配置若干服务分区,并基于所述服务分区的数据资源分布规则,由所述服务分区处理相应访问请求,其中,每一所述服务分区具有若干一致的服务组件;
迁移装置,用于将处理负载过大的所述服务分区所存储的部分数据资源迁移至迁移服务分区,并在迁移完成后,由所述迁移服务分区处理所迁移的数据资源对应的所述访问请求。
进一步地,所述迁移装置用于:
将处理负载过大的所述服务分区所存储的部分数据资源选定为迁移前数据资源;
将所述迁移前数据资源以复制方式迁移到所述迁移服务分区,形成迁移后数据资源。
进一步地,所述迁移装置还包括:
在迁移过程中,将所述迁移后数据资源设置为无效状态;
在即将完成迁移时,将所述迁移前数据资源设置为只读状态;
在完成迁移后,将所述迁移前数据资源设置为不可读不可写状态,并将所述迁移后数据设置为只读状态。
进一步地,所述迁移装置用于:在迁移完成后,
更新所述数据资源分布规则;
向所有用户发起服务分区变更通知,并基于所述用户的拥有者变更操作请求,将迁移后数据资源中的标识文件设置为有效,且拥有者信息设置为相应的请求用户;
在完成迁移超出预设时间后,重置所述迁移前数据资源和所述迁移后数据资源的状态为可读可写状态,并删除迁移数据中未获得更改用户者操作请求的相应标识文件。
进一步地,所述标识文件包括分布式锁文件。
进一步地,所述迁移装置用于:
重置所述迁移前数据资源和所述迁移后数据资源的状态为可读可写状态后超出超时时间时,删除所述迁移前数据资源。
进一步地,所述服务分区的服务组件采用多数派机制维持一致性。
进一步地,所述迁移服务分区为已有空闲的服务分区或新配置的服务分区。
与现有技术相比,在本申请一方面提供的所述用于扩展分布式一致性服务的方法及另一方面提供的设备,通过部署多组包括分布式锁服务组件的服务分区、每组服务分区存储部分数据资源,并提供相应访问请求,并在运行时动态调整每个分区服务的数据资源来平衡各组服务分区之间的处理负载压力,以此提高了分布式一致性服务的扩展能力。
通过阅读参照以下附图所作的对非限制性实施例所作的详细描述,本申请的其它特征、目的和优点将会变得更明显:
图1示出根据本申请一个方面的提供的一种用于扩展分布式一致性服务的方法流程图;
图2~图4示出本申请一个优选实施例提供的一种用于扩展分布式一致性服务的迁移过程示意图;
图5示出根据本申请一个方面的提供的一种用于扩展分布式一致性服务的设备示意图。
附图中相同或相似的附图标记代表相同或相似的部件。
下面结合附图对本申请作进一步详细描述。
在分布式系统中,分布式一致性服务一般基于C/S架构(客户/服务架构)来实现,服务一般由多个服务组件形成的多个节点提供服务,以保证一致性和可用性;客户端提供接口给用户或应用方用来访问分布式一致性服务。分布式一致性服务包括分布式锁,分布式锁是控制分布式系统之间同步访问共享资源的一种方式。如果不同的系统或是同一个系统的不同主机之间共享了一个或一组服务组件的数据资源,那么请求访问这些数据资源的时候,往往需要互斥来防止彼此干扰来保证一致性,在这种情况下,便需要使用到分布式锁。
基于以上,本申请一方面提供了一种基于服务分区的分布式一致性服务的方法,通过配置若干具有一致性服务的服务分区,并设定每一服务分区所服务的数据资源的分布规则,从而将数据资源分散到多个服务分区中,每个服务分区处理一部分访问请求并存储部分相应数据资源;同时,当访问请求增多或数据资源增大时,造成某个或某几个服务分区的处理负载过大时,则动态配置迁移服务分区,迁移服务分区可以是已有空闲的服务分区或新配置的服务分区,然后将处理负载过大的所述服务分区所存储的部分数据资源迁移至迁移服务分区,并在迁移完成后,由所述迁移服务分区处理所迁移的数据资源对应的所述访问请求,从而平衡多个服务分区之间的负载压力,保证服务稳定性。
图1示出根据本申请一个方面的提供的一种用于扩展分布式一致性服务的方法流程图,其中,所述方法包括:
步骤S11:配置若干服务分区,并基于所述服务分区的数据资源分布规则,由所述服务分区处理相应访问请求,其中,每一所述服务分区具有若干一致的服务组件;
步骤S12:将处理负载过大的所述服务分区所存储的部分数据资源迁移至迁移服务分区,并在迁移完成后,由所述迁移服务分区处理所迁移的数据资源对应的所述访问请求。
在所述步骤S11中,所述服务分区的服务组件采用多数派机制维持一致性,相同所述服务分区中的服务组件存储相同的数据资源。在配置完成后,当访问请求到达分布式一致性服务前端时,会基于访问请求的内容,并根据数据资源分布规则,将访问请求路由到相应能够处理该访问请求的服务分区中,由该服务分区完成对用户的访问申请的处
理、释放分布式锁,设置分布式锁的拥有者信息等操作。
在所述步骤S12中,在分布式一致性服务工作一段时间后,某个或某些服务分区上会出现访问请求过多或存储数据资源过大,导致处理负载过大,此时可以根据系统的运行情况,动态将处理负载过大的所述服务分区所存储的部分数据资源迁移至迁移服务分区,并在迁移完成后,由所述迁移服务分区处理所迁移的数据资源对应的所述访问请求。
进一步地,在所述步骤S12中,将处理负载过大的所述服务分区所存储的部分数据资源迁移至迁移服务分区包括:
将处理负载过大的所述服务分区所存储的部分数据资源选定为迁移前数据资源;具体地,选定方式并不被限制,可根据数据资源的数据量及所对应的访问请求对服务分区进行具体划分;将所述迁移前数据资源以复制方式迁移到所述迁移服务分区,形成迁移后数据资源。以复制的方式能够保证在迁移过程中,不影响迁移前已拥有标识文件(分布式锁文件)的用户继续提供数据服务。
进一步地,在所述步骤S12中,将处理负载过大的所述服务分区所存储的部分数据资源迁移至迁移服务分区还包括:
在迁移过程中,将所述迁移后数据资源设置为无效状态;设置为无效状态用以防止系统认为迁移后数据资源超时时间内未获得访问请求而删除所述迁移后数据资源。
在即将完成迁移时,将所述迁移前数据资源设置为只读状态;此时,用户已经持有的标识文件并不受影响,即可以继续利用所述标识文件读取迁移前数据资源,
在完成迁移后,将所述迁移前数据资源设置为不可读不可写状态,并将所述迁移后数据设置为只读状态,为数据资源resource-x开始提供服务做准备。
进一步地,在所述步骤S12中,在迁移完成后由所述迁移服务分区处理与所迁移的数据资源相应的所述访问请求包括:
更新所述数据资源分布规则;
向所有用户发起服务分区变更通知,并基于所述用户的拥有者变更操作请求,将迁移后数据资源中的标识文件设置为有效,且拥有者信息设置为相应的请求用户;具体地,若收到来自拥有迁移前数据资源的标识文件的用户的拥有者变更操作请求后,会把对应迁移后数据资源中的标识文件设置为有效、且将拥有者信息更改为该原拥有者用户,若作为拥有迁移前数据资源的标识文件的用户在收到通知后未向服务分区发起拥有者请求操作,在等待超出预设时间后,则删除相应标识文件,可以根据其他用户的访问请求重新释放分布式锁。
在完成迁移超出预设时间后,重置所述迁移前数据资源和所述迁移后数据资源的状态为可读可写状态,并删除迁移数据中未获得更改用户者操作请求的相应标识文件。其中,优选地,所述标识文件包括分布式锁文件。
此外,当重置所述迁移前数据资源和所述迁移后数据资源的状态为可读可写状态后超出超时时间时,删除所述迁移前数据资源。,最终完成数据迁移。
图2~图4示出本申请一个优选实施例提供的一种扩展分布式一致性服务的迁移过程示意图,在具体的场景中,以服务分区quorum-n为例,来说明用于扩展分布式一致性服务的方法中动态迁移数据的过程:
如图2所示,初始配置后,例如,在提供服务的服务分区包括服务分区quorum-1至服务分区quorum-n,其中,服务分区quorum-n上有数据资源resource-n和数据资源resouce-x,都由服务分区quorum-n服务,在quorum-n服务为用户分配关于数据资源resource-n的锁文件lock://resource-n/foo,为用户分配关于数据资源resource-x的锁文件lock://resource-x/foo,作为拥有者的用户可以持所述锁文件lock://resource-n/foo获取数据资源resource-n的相关服务,持所述锁文件lock://resource-x/foo获取数据资源resource-x的相关服务。
如图3所示,当服务分区quorum-n的处理负载压力过大,决定把数据资源resource-x迁移到较空闲的服务分区quorum-x上。
首先,启动一个迁移操作(migration操作),实时地将服务分区quorum-n上和数据资源resource-x相关的数据动态以复制的方式迁移到迁移服务分区quorum-x上。Migration操作利用分布式锁原生的同步协议,可以实时的同步所有数据及更新操作到新的分区,整个过程对原有服务不会产生任何影响。此时,迁移到迁移服务分区quorum-x上的迁移后数据资源resource-x’设置为无效状态(invalid),设置为无效状态用以防止系统认为迁移后数据资源resource-x’超时时间内未获得访问请求而删除所述迁移后数据资源resource-x’。
当迁移即将完成时,服务分区quorum-n的数据资源resource-x和服务分区quorum-x上数据资源resource-x’的相关数据的状态基本一致时,例如需要迁移的数据资源一共有1024M,还剩8M需要迁移的数据需要传送,剩余需要迁移的数据资源所需传递的时间远小于一次访问请求的处理时间时,在服务分区quorum-n上对数据资源resource-x设置只读状态(read-only)。此时,用户已经持有的锁文件lock://resource-x/foo并不受影响,即可以继续利用所述锁文件读取数据资源resource-x。
当迁移完成时,待服务分区quorum-n的数据资源resource-x和服务分区quorum-x上数据资源resource-x’的相关数据的完全一致后,则在服务分区quorum-n上禁止所有对数据资源resource-x的操作,同时在服务分区quorum-x上设置数据资源resource-x’为只读状态(read-only),即为数据资源resource-x’开始提供服务做准备。
更新所有服务分区路由规则:数据资源resource-x’由服务分区quorum-x服务,数据资源resource-x’与数据资源resource-x完全一致。
通过系统前端通知所有用户作拥有者变更操作(chown操作)。作为原拥有者的用户在收到通知后会向服务分区quorum-x发起一个chown操作。则服务分区quorum-x收到来自原拥有者的用户的chown操作后,会把对应的锁文件(lock://resource-x/foo)设置为有效、且拥有者信息更改为该原拥有者用户,若作为原拥有者的用户在收到通知后未向服务分区quorum-x发起chown操作,在等待超出预设时间后,则删除服务分区quorum-x上未被chown操作的锁文件lock://resource-x/foo,可以根据其他用户的访问请求重新释放分布式锁,同时,在超出预设时间后,取消数据资源resource-x和数据资源resource-x’的状态限制,即均为可读可写状态,在超出超时时间内数据资源resource-x未收到访问请求的心跳报文,则系统会清除数据资源resource-x,最终完成数据迁移,如图4所示。
当然,本领域技术人员应能理解上述分布式一致性服务的动态迁移的具体场景及其中锁文件、服务分区、数据资源、chown操作,migration操作的描述仅为举例,其他现有的或今后可能出现的其他描述如可适用于本申请,也应包含在本申请保护范围以内,并在此以引用方式包含于此。
图5示出根据本申请一个方面的提供的一种用于扩展分布式一致性服务的设备示意图,其中,所述设备1包括:
配置装置11,配置若干服务分区,并基于所述服务分区的数据资源分布规则,由所述服务分区处理相应访问请求,其中,每一所述服务分区具有若干一致的服务组件;
迁移装置12,将处理负载过大的所述服务分区所存储的部分数据资源迁移至迁移服务分区,并在迁移完成后,由所述迁移服务分区处理所迁移的数据资源对应的所述访问请求。
在此,所述设备1包括但不限于用户设备、或用户设备与网络设备通过网络相集成所构成的设备。所述用户设备其包括但不限于任何一种可与用户通过触摸板进行人机交互的移动电子产品,例如智能手机、PDA等,所述移动电子产品可以采用任意操作系统,
如android操作系统、iOS操作系统等。其中,所述网络设备包括一种能够按照事先设定或存储的指令,自动进行数值计算和信息处理的电子设备,其硬件包括但不限于微处理器、专用集成电路(ASIC)、可编程门阵列(FPGA)、数字处理器(DSP)、嵌入式设备等。所述网络包括但不限于互联网、广域网、城域网、局域网、VPN网络、无线自组织网络(Ad Hoc网络)等。优选地,设备1还可以是运行于所述用户设备、或用户设备与网络设备、触摸终端或网络设备与触摸终端通过网络相集成所构成的设备上的脚本程序。当然,本领域技术人员应能理解上述设备1仅为举例,其他现有的或今后可能出现的设备1如可适用于本申请,也应包含在本申请保护范围以内,并在此以引用方式包含于此。
上述各装置之间是持续不断工作的,在此,本领域技术人员应理解“持续”是指上述各装置分别实时地或者按照设定的或实时调整的工作模式要求,直至所述设备1停止工作。
所述配置装置11对所述服务分区的服务组件采用多数派机制维持一致性,相同所述服务分区中的服务组件存储相同的数据资源。在配置完成后,当访问请求到达分布式一致性服务的前端时,会基于访问请求的内容,并根据数据资源分布规则,将访问请求路由到相应能够处理该访问请求的服务分区中,由该服务分区完成对用户的访问申请的处理、释放分布式锁,设置分布式锁的拥有者信息等操作。
在所述迁移装置12中,在分布式一致性服务在工作一段时间中,某个或某些服务分区上会出现访问请求过多或存储数据资源过大,导致处理负载过大,此时可以根据系统的运行情况,动态将处理负载过大的所述服务分区所存储的部分数据资源迁移至迁移服务分区,并在迁移完成后,由所述迁移服务分区处理所迁移的数据资源对应的所述访问请求。
进一步地,在所述迁移装置12中,将处理负载过大的所述服务分区所存储的部分数据资源迁移至迁移服务分区包括:
将处理负载过大的所述服务分区所存储的部分数据资源选定为迁移前数据资源;具体地,选定方式并不被限制,可根据数据资源的数据量及所对应的访问请求对服务分区进行具体划分;将所述迁移前数据资源以复制方式迁移到所述迁移服务分区,形成迁移后数据资源。以复制的方式能够保证在迁移过程中,不影响迁移前已拥有标识文件(分布式锁文件)的用户继续提供数据服务。
进一步地,在所述迁移装置12中,将处理负载过大的所述服务分区所存储的部分数据资源迁移至迁移服务分区还包括:
在迁移过程中,将所述迁移后数据资源设置为无效状态;设置为无效状态用以防止系统认为迁移后数据资源超时时间内未获得访问请求而删除所述迁移后数据资源。
在即将完成迁移时,将所述迁移前数据资源设置为只读状态;此时,用户已经持有的标识文件并不受影响,即可以继续利用所述标识文件读取迁移前数据资源,
在完成迁移后,将所述迁移前数据资源设置为不可读不可写状态,并将所述迁移后数据设置为只读状态,为数据资源resource-x开始提供服务做准备。
进一步地,在所述迁移装置12中,在迁移完成后由所述迁移服务分区处理与所迁移的数据资源相应的所述访问请求包括:
更新所述数据资源分布规则;
向所有用户发起服务分区变更通知,并基于所述用户的拥有者变更操作请求,将迁移后数据资源中的标识文件设置为有效,且拥有者信息设置为相应的请求用户;具体地,若收到来自拥有迁移前数据资源的标识文件的用户的拥有者变更操作请求后,会把对应迁移后数据资源中的标识文件设置为有效、且将拥有者信息更改为该原拥有者用户,若作为拥有迁移前数据资源的标识文件的用户在收到通知后未向服务分区发起拥有者请求操作,在等待超出预设时间后,则删除相应标识文件,可以根据其他用户的访问请求重新释放分布式锁。
在完成迁移超出预设时间后,重置所述迁移前数据资源和所述迁移后数据资源的状态为可读可写状态,并删除迁移数据中未获得更改用户者操作请求的相应标识文件。其中,优选地,所述标识文件包括分布式锁文件。
图2~图4示出本申请一个优选实施例提供的一种扩展分布式一致性服务的迁移过程示意图,在具体的场景中,以服务分区quorum-n为例,来说明用于扩展分布式一致性服务的设备1进行动态迁移数据的过程:
如图2所示,初始配置后,例如,在服务分区quorum-n上有数据资源resource-n和数据资源resouce-x,都由服务分区quorum-n服务,在quorum-n服务为用户分配关于数据资源resource-n的锁文件lock://resource-n/foo,为用户分配关于数据资源resource-x的锁文件lock://resource-x/foo,作为拥有者的用户可以持所述锁文件lock://resource-n/foo获取数据资源resource-n的相关服务,持所述锁文件lock://resource-x/foo获取数据资源resource-x的相关服务。
如图3所示,当服务分区quorum-n的处理负载压力过大,决定把数据资源resource-x迁移到较空闲的服务分区quorum-x上。
首先,启动一个迁移操作(migration操作),实时地将服务分区quorum-n上和数据资源resource-x相关的数据动态以复制的方式迁移到迁移服务分区quorum-x上。Migration操作利用分布式锁原生的同步协议,可以实时的同步所有数据及更新操作到新的分区,整个过程对原有服务不会产生任何影响。此时,迁移到迁移服务分区quorum-x上的迁移后数据资源resource-x’设置为无效状态(invalid),设置为无效状态用以防止系统认为迁移后数据资源resource-x’超时时间内未获得访问请求而删除所述迁移后数据资源resource-x’。
当迁移即将完成时,服务分区quorum-n的数据资源resource-x和服务分区quorum-x上数据资源resource-x’的相关数据的状态基本一致时,例如需要迁移的数据资源一共有1024M,还剩8M需要迁移的数据需要传送,剩余需要迁移的数据资源所需传递的时间远小于一次访问请求的处理时间时,在服务分区quorum-n上对数据资源resource-x设置只读状态(read-only)。此时,用户已经持有的锁文件lock://resource-x/foo并不受影响,即可以继续利用所述锁文件读取数据资源resource-x。
当迁移完成时,待服务分区quorum-n的数据资源resource-x和服务分区quorum-x上数据资源resource-x’的相关数据的完全一致后,则在服务分区quorum-n上禁止所有对数据资源resource-x的操作,同时在服务分区quorum-x上设置数据资源resource-x’为只读状态(read-only),即为数据资源resource-x’开始提供服务做准备。
更新所有服务分区路由规则:数据资源resource-x’由服务分区quorum-x服务,数据资源resource-x’与数据资源resource-x完全一致。
通过系统前端通知所有用户作拥有者变更操作(chown操作)。作为原拥有者的用户在收到通知后会向服务分区quorum-x发起一个chown操作。则服务分区quorum-x收到来自原拥有者的用户的chown操作后,会把对应的锁文件(lock://resource-x/foo)设置为有效、且拥有者信息更改为该原拥有者用户,若作为原拥有者的用户在收到通知后未向服务分区quorum-x发起chown操作,在等待超出预设时间后,则删除服务分区quorum-x上未被chown操作的锁文件lock://resource-x/foo,可以根据其他用户的访问请求重新释放分布式锁,同时,在超出预设时间后,取消数据资源resource-x和数据资源resource-x’的状态限制,即均为可读可写状态,在超出超时时间内数据资源resource-x未收到访问请求的心跳报文,则系统会清除数据资源resource-x,最终完成数据迁移,如图4所示。
当然,本领域技术人员应能理解上述扩展分布式一致性服务的动态迁移的具体场景
及其中锁文件、服务分区、数据资源、chown操作,migration操作的描述仅为举例,其他现有的或今后可能出现的其他描述如可适用于本申请,也应包含在本申请保护范围以内,并在此以引用方式包含于此。
与现有技术相比,根据本申请的实施例提供的基于服务分区的扩展分布式一致性服务的设备,通过配置装置配置若干具有一致性服务的服务分区,并设定每一服务分区所服务的数据资源的分布规则,从而将数据资源分散到多个服务分区中,每个服务分区处理一部分访问请求并存储部分相应数据资源;同时,当访问请求增多或数据资源增大时,造成某个或某几个服务分区的处理负载过大时,则通过迁移装置动态配置迁移服务分区,然后将处理负载过大的所述服务分区所存储的部分数据资源迁移至迁移服务分区,并在迁移完成后,由所述迁移服务分区处理所迁移的数据资源对应的所述访问请求,从而平衡多个服务分区之间的负载压力,保证服务稳定性。
需要注意的是,本申请可在软件和/或软件与硬件的组合体中被实施,例如,可采用专用集成电路(ASIC)、通用目的计算机或任何其他类似硬件设备来实现。在一个实施例中,本申请的软件程序可以通过处理器执行以实现上文所述步骤或功能。同样地,本申请的软件程序(包括相关的数据结构)可以被存储到计算机可读记录介质中,例如,RAM存储器,磁或光驱动器或软磁盘及类似设备。另外,本申请的一些步骤或功能可采用硬件来实现,例如,作为与处理器配合从而执行各个步骤或功能的电路。
另外,本申请的一部分可被应用为计算机程序产品,例如计算机程序指令,当其被计算机执行时,通过该计算机的操作,可以调用或提供根据本申请的方法和/或技术方案。而调用本申请的方法的程序指令,可能被存储在固定的或可移动的记录介质中,和/或通过广播或其他信号承载媒体中的数据流而被传输,和/或被存储在根据所述程序指令运行的计算机设备的工作存储器中。在此,根据本申请的一个实施例包括一个装置,该装置包括用于存储计算机程序指令的存储器和用于执行程序指令的处理器,其中,当该计算机程序指令被该处理器执行时,触发该装置运行基于前述根据本申请的多个实施例的方法和/或技术方案。
对于本领域技术人员而言,显然本申请不限于上述示范性实施例的细节,而且在不背离本申请的精神或基本特征的情况下,能够以其他的具体形式实现本申请。因此,无论从哪一点来看,均应将实施例看作是示范性的,而且是非限制性的,本申请的范围由所附权利要求而不是上述说明限定,因此旨在将落在权利要求的等同要件的含义和范围内的所有变化涵括在本申请内。不应将权利要求中的任何附图标记视为限制所涉及的权
利要求。此外,显然“包括”一词不排除其他单元或步骤,单数不排除复数。装置权利要求中陈述的多个单元或装置也可以由一个单元或装置通过软件或者硬件来实现。第一,第二等词语用来表示名称,而并不表示任何特定的顺序。
显然,本领域的技术人员可以对本申请进行各种改动和变型而不脱离本申请的精神和范围。这样,倘若本申请的这些修改和变型属于本申请权利要求及其等同技术的范围之内,则本申请也意图包含这些改动和变型在内。
Claims (14)
- 一种用于扩展分布式一致性服务的方法,其中,所述方法包括:配置若干服务分区,并基于所述服务分区的数据资源分布规则,由所述服务分区处理相应访问请求,其中,每一所述服务分区具有若干一致的服务组件;将处理负载过大的所述服务分区所存储的部分数据资源迁移至迁移服务分区,并在迁移完成后,由所述迁移服务分区处理所迁移的数据资源对应的所述访问请求。
- 根据权利要求1所述的方法,其中,将处理负载过大的所述服务分区所存储的部分数据资源迁移至迁移服务分区包括:将处理负载过大的所述服务分区所存储的部分数据资源选定为迁移前数据资源;将所述迁移前数据资源以复制方式迁移到所述迁移服务分区,形成迁移后数据资源。
- 根据权利要求2所述的方法,其中,将处理负载过大的所述服务分区所存储的部分数据资源迁移至迁移服务分区还包括:在迁移过程中,将所述迁移后数据资源设置为无效状态;在即将完成迁移时,将所述迁移前数据资源设置为只读状态;在完成迁移后,将所述迁移前数据资源设置为不可读不可写状态,并将所述迁移后数据设置为只读状态。
- 根据权利要求3所述的方法,其中,在迁移完成后由所述迁移服务分区处理与所迁移的数据资源相应的所述访问请求包括:更新所述数据资源分布规则;向所有用户发起服务分区变更通知,并基于所述用户的拥有者变更操作请求,将迁移后数据资源中的标识文件设置为有效,且拥有者信息设置为相应的请求用户;在完成迁移超出预设时间后,重置所述迁移前数据资源和所述迁移后数据资源的状态为可读可写状态,并删除迁移数据中未获得更改用户者操作请求的相应标识文件。
- 根据权利要求4所述的方法,其中,所述标识文件包括分布式锁文件。
- 根据权利要求1至5中任一项所述的方法,其中,所述服务分区的服务组件采用多数派机制维持一致性。
- 根据权利要去1至6中任一项所述的方法,其中,所述迁移服务分区为已有空闲的服务分区或新配置的服务分区。
- 一种用于扩展分布式一致性服务的设备,其中,所述设备包括:配置装置,用于配置若干服务分区,并基于所述服务分区的数据资源分布规则,由所述服务分区处理相应访问请求,其中,每一所述服务分区具有若干一致的服务组件;迁移装置,用于将处理负载过大的所述服务分区所存储的部分数据资源迁移至迁移服务分区,并在迁移完成后,由所述迁移服务分区处理所迁移的数据资源对应的所述访问请求。
- 根据权利要求8所述的设备,其中,所述迁移装置用于:将处理负载过大的所述服务分区所存储的部分数据资源选定为迁移前数据资源;将所述迁移前数据资源以复制方式迁移到所述迁移服务分区,形成迁移后数据资源。
- 根据权利要求9所述的设备,其中,所述迁移装置还包括:在迁移过程中,将所述迁移后数据资源设置为无效状态;在即将完成迁移时,将所述迁移前数据资源设置为只读状态;在完成迁移后,将所述迁移前数据资源设置为不可读不可写状态,并将所述迁移后数据设置为只读状态。
- 根据权利要求10所述的设备,其中,所述迁移装置用于:在迁移完成后,更新所述数据资源分布规则;向所有用户发起服务分区变更通知,并基于所述用户的拥有者变更操作请求,将迁移后数据资源中的标识文件设置为有效,且拥有者信息设置为相应的请求用户;在完成迁移超出预设时间后,重置所述迁移前数据资源和所述迁移后数据资源的状态为可读可写状态,并删除迁移数据中未获得更改用户者操作请求的相应标识文件。
- 根据权利要求11所述的设备,其中,所述标识文件包括分布式锁文件。
- 根据权利要求8至12中任一项所述的设备,其中,所述服务分区的服务组件采用多数派机制维持一致性。
- 根据权利要去8至13中任一项所述的设备,其中,所述迁移服务分区为已有空闲的服务分区或新配置的服务分区。
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510570429.9A CN106534227B (zh) | 2015-09-09 | 2015-09-09 | 用于扩展分布式一致性服务的方法和设备 |
| CN201510570429.9 | 2015-09-09 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| WO2017041650A1 true WO2017041650A1 (zh) | 2017-03-16 |
Family
ID=58240667
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| PCT/CN2016/097494 Ceased WO2017041650A1 (zh) | 2015-09-09 | 2016-08-31 | 用于扩展分布式一致性服务的方法和设备 |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN106534227B (zh) |
| WO (1) | WO2017041650A1 (zh) |
Families Citing this family (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109947375B (zh) * | 2019-04-04 | 2021-05-14 | 江南大学 | 一种基于分区处理共识算法的分布式存储系统优化方法 |
| CN113076187B (zh) * | 2020-01-03 | 2024-01-09 | 阿里巴巴集团控股有限公司 | 分布式锁管理方法及装置 |
| CN111858079B (zh) * | 2020-07-27 | 2024-03-12 | 北京达佳互联信息技术有限公司 | 分布式锁迁移方法、装置及电子设备、存储介质 |
| CN115525625A (zh) * | 2022-09-16 | 2022-12-27 | 中国建设银行股份有限公司 | 数据处理方法、装置、设备及介质 |
| CN116382576B (zh) * | 2023-03-17 | 2024-02-27 | 平头哥(上海)半导体技术有限公司 | 存储控制芯片、固态硬盘和相关方法 |
| CN119249753A (zh) * | 2024-10-10 | 2025-01-03 | 中国长江电力股份有限公司 | 水电站多模块协同仿真和管理系统 |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101187949A (zh) * | 2007-12-26 | 2008-05-28 | 腾讯科技(深圳)有限公司 | 一种大容量数据的统计方法及系统 |
| CN101710339A (zh) * | 2009-11-20 | 2010-05-19 | 中国科学院计算技术研究所 | 控制机群文件系统中数据存储方法和系统及创建文件方法 |
| CN101909068A (zh) * | 2009-06-02 | 2010-12-08 | 华为技术有限公司 | 一种文件副本管理方法及装置、系统 |
| CN103036994A (zh) * | 2012-12-18 | 2013-04-10 | 曙光信息产业(北京)有限公司 | 实现负载均衡的云存储系统 |
| CN103327094A (zh) * | 2013-06-19 | 2013-09-25 | 成都市欧冠信息技术有限责任公司 | 数据分布式存储方法与系统 |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10831728B2 (en) * | 2012-05-29 | 2020-11-10 | International Business Machines Corporation | Application-controlled sub-LUN level data migration |
| CN104468674B (zh) * | 2013-09-25 | 2020-01-14 | 南京中兴新软件有限责任公司 | 数据迁移方法及装置 |
-
2015
- 2015-09-09 CN CN201510570429.9A patent/CN106534227B/zh active Active
-
2016
- 2016-08-31 WO PCT/CN2016/097494 patent/WO2017041650A1/zh not_active Ceased
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101187949A (zh) * | 2007-12-26 | 2008-05-28 | 腾讯科技(深圳)有限公司 | 一种大容量数据的统计方法及系统 |
| CN101909068A (zh) * | 2009-06-02 | 2010-12-08 | 华为技术有限公司 | 一种文件副本管理方法及装置、系统 |
| CN101710339A (zh) * | 2009-11-20 | 2010-05-19 | 中国科学院计算技术研究所 | 控制机群文件系统中数据存储方法和系统及创建文件方法 |
| CN103036994A (zh) * | 2012-12-18 | 2013-04-10 | 曙光信息产业(北京)有限公司 | 实现负载均衡的云存储系统 |
| CN103327094A (zh) * | 2013-06-19 | 2013-09-25 | 成都市欧冠信息技术有限责任公司 | 数据分布式存储方法与系统 |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106534227B (zh) | 2020-02-18 |
| CN106534227A (zh) | 2017-03-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN116848528B (zh) | 用于自动配置用于容器应用的最小云服务访问权限的技术 | |
| EP3761170B1 (en) | Virtual machine creation method and apparatus | |
| US10701139B2 (en) | Life cycle management method and apparatus | |
| CN107924383B (zh) | 用于网络功能虚拟化资源管理的系统和方法 | |
| JP5510556B2 (ja) | 仮想マシンのストレージスペースおよび物理ホストを管理するための方法およびシステム | |
| CN105354076B (zh) | 一种应用部署方法及装置 | |
| WO2017041650A1 (zh) | 用于扩展分布式一致性服务的方法和设备 | |
| EP3584998B1 (en) | Method for virtual machine capacity expansion and reduction and virtual management device | |
| CN108984266B (zh) | 一种虚拟机的管理方法、装置及系统 | |
| EP3706368A1 (en) | Method and device for deploying virtualized network element device | |
| US10776322B2 (en) | Transformation processing for objects between storage systems | |
| WO2018006676A1 (zh) | 加速资源处理方法、装置及网络功能虚拟化系统 | |
| US20170024251A1 (en) | Scheduling method and apparatus for distributed computing system | |
| WO2018036103A1 (zh) | 虚拟机创建方法、系统以及主机 | |
| JP2015144020A5 (zh) | ||
| CN106664259B (zh) | 虚拟网络功能扩容的方法和装置 | |
| CN107959582A (zh) | 一种切片实例的管理方法及装置 | |
| WO2015176636A1 (zh) | 分布式数据库服务管理系统 | |
| WO2023274014A1 (zh) | 容器集群的存储资源管理方法、装置及系统 | |
| CN104268089B (zh) | 一种信息更新的方法、装置及系统 | |
| CN105100180A (zh) | 集群节点动态加载的方法、装置及系统 | |
| CN109905258B (zh) | PaaS的管理方法、装置及存储介质 | |
| US9379940B2 (en) | Virtual device profile to configure virtual network interface cards | |
| CN104573496B (zh) | 一种禁止启动项启动的方法和装置 | |
| CN107534678B (zh) | 建立vnfm与vim之间的连接的方法、装置及系统 |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| 121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 16843588 Country of ref document: EP Kind code of ref document: A1 |
|
| NENP | Non-entry into the national phase |
Ref country code: DE |
|
| 122 | Ep: pct application non-entry in european phase |
Ref document number: 16843588 Country of ref document: EP Kind code of ref document: A1 |