WO2021088531A1 - Procédé de redistribution de données, dispositif électronique, et support de stockage - Google Patents

Procédé de redistribution de données, dispositif électronique, et support de stockage Download PDF

Info

Publication number
WO2021088531A1
WO2021088531A1 PCT/CN2020/116284 CN2020116284W WO2021088531A1 WO 2021088531 A1 WO2021088531 A1 WO 2021088531A1 CN 2020116284 W CN2020116284 W CN 2020116284W WO 2021088531 A1 WO2021088531 A1 WO 2021088531A1
Authority
WO
WIPO (PCT)
Prior art keywords
data
bucket number
bucket
target
migrated
Prior art date
Application number
PCT/CN2020/116284
Other languages
English (en)
Chinese (zh)
Inventor
郭龙波
岳新新
丁岩
刘志文
Original Assignee
中兴通讯股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 中兴通讯股份有限公司 filed Critical 中兴通讯股份有限公司
Publication of WO2021088531A1 publication Critical patent/WO2021088531A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
    • G06F16/278—Data partitioning, e.g. horizontal or vertical partitioning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor

Definitions

  • the embodiments of the present application relate to the field of distributed data storage, and in particular, to a data redistribution method, electronic equipment, and storage medium.
  • the current method of storing data is no longer limited to centralized storage.
  • the data is beginning to be split and stored in a distributed manner.
  • the data is stored on different data nodes according to the distribution key to form a logically large database, which improves the comprehensive utilization of storage resources to improve the ability of data storage. If the selection of the distribution key is unreasonable, or hot data and other issues, it may cause some The data volume of data nodes far exceeds other data nodes, becoming a bottleneck in performance and storage capacity, and seriously affecting the services of distributed databases.
  • Redistributing current data is an important basic function of distributed databases.
  • the method of redistribution includes offline redistribution and online data redistribution based on the data pre-sharding count.
  • the offline redistribution process shuts down the business operation and then redistributes the data.
  • Online data distribution based on data pre-sharding By pre-establishing multiple tables at each node, using physical partitioning to split the data, and then fill the data into the corresponding table according to the mapping relationship, the table is stored in the corresponding shard according to the mapping relationship, and when the data is redistributed, the data is divided Perform data changes as a basic unit.
  • offline redistribution cannot meet the requirements of business operation.
  • the existing counting online redistribution method not only has low indexing efficiency, but also reduces system reliability; when redistributing extremely large amounts of data , Scalability is poor, hot issues will still occur, and during the data migration process, data migration may fail due to disk capacity.
  • the purpose of the embodiments of the present application is to provide a data redistribution method, electronic device, and storage medium.
  • the embodiment of the application provides a data redistribution method, including: adding a bucket number identifier to the data record to be stored, and storing the data record in the data record according to the preset correspondence relationship between the bucket number identifier and the fragmented database.
  • the recorded bucket number identifies the corresponding shard database; in response to the data redistribution request, the target bucket number identifier of the data to be migrated is obtained; according to the target bucket number identifier, the data to be migrated is migrated to the target shard database.
  • the embodiments of the present application also provide an electronic device, including: at least one processor; and a memory communicatively connected with the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are at least One processor executes, so that at least one processor can execute the above data redistribution method.
  • the embodiment of the present application also provides a computer-readable storage medium storing a computer program, and the computer program is executed by a processor to implement the above-mentioned data redistribution method.
  • Fig. 1 is a flowchart of a data redistribution method according to the first embodiment of the present application
  • Fig. 2 is a flowchart of a data redistribution method according to the second embodiment of the present application
  • FIG. 3 is a schematic diagram of the system architecture of the data redistribution method according to the second embodiment of the present application.
  • Fig. 4 is a schematic diagram of the structure of an electronic device according to a third embodiment of the present application.
  • the first embodiment of the present application relates to a data redistribution method.
  • the bucket number identifier is added to the data record to be stored, and the data record is stored in the segment database corresponding to the bucket number identifier of the data record according to the preset correspondence relationship between the bucket number identifier and the shard database. ;
  • By logically partitioning data for storage it avoids that poor scalability leads to hot spots when storing data in physical partitions.
  • use a bucket number to identify the corresponding data record as the basic unit for data migration , To avoid the impact on data migration performance, improve the efficiency and quality of data migration, and reduce the impact of data migration on business.
  • Step 101 Add a bucket number identifier to a data record to be stored and store it in a fragmented database corresponding to the bucket number identifier.
  • the metadata server After receiving the data to be stored, the metadata server adds a bucket number identifier to the data record to be stored, and stores the data record in the data record according to the preset correspondence between the bucket number identifier and the shard database.
  • the recorded bucket number identifies the corresponding shard database.
  • the metadata server estimates the number of data records to be stored before storing data, and combines the capacity requirements of the migrated data during the data migration process to preset the maximum number of uses for each bucket number, and then combine Estimate the number of data records to be stored and the preset maximum use times of each bucket number identifier, and calculate the number M of bucket number identifiers to be generated according to the following formula:
  • S is the estimated number of data records to be stored
  • L is the preset maximum usage times for each bucket number.
  • the preset maximum use times of the bucket number identifier can be adjusted according to actual conditions and needs, and there is no limitation on the setting of the preset maximum times in this embodiment.
  • the metadata server uses a random hash algorithm to determine the bucket number identifier to be added to the data record to be stored, and adds the determined bucket number identifier to be added to the extended field of the data record to be stored , And according to the preset corresponding relationship between the bucket number identification and the fragment database, the data record is stored in the fragment database corresponding to the bucket number identification of the data record.
  • the random hash algorithm used by the metadata server is a hash algorithm, and the number of bucket IDs calculated and created is 1000. Since the result of the hash algorithm corresponds to the bucket ID one-to-one, the possible hash value is also 1000.
  • the data to be stored is input into the hash algorithm, the hash value obtained by calculation is 23, then the bucket number is added to the data record expansion field of this piece of data Identify the field of 23, and then query the shard database corresponding to the bucket ID 23 in the list of the corresponding relationship between the bucket number ID and the shard database. If the queried shard database is 2, then this data will be stored in the shard database 2. in.
  • the random hash algorithm used can be adjusted according to actual needs. In this embodiment, There are no restrictions on the selection of random hashing algorithms.
  • the random hash algorithm is used to assign the bucket number identifier to be added to the data record to be stored, thereby ensuring the uniform distribution of data when the data record is stored in the shard database according to the bucket number identifier.
  • Step 102 When receiving the data redistribution request, obtain the target bucket ID of the data to be migrated.
  • the metadata server After receiving the data redistribution request, the metadata server obtains the target bucket ID of the data to be migrated when responding to the data redistribution request, that is, determines which bucket ID corresponds to the data to be migrated.
  • the metadata server after receiving the data redistribution request issued by the upper layer, the metadata server generates a complete data redistribution task plan based on the information of the metadata to be redistributed, and determines the bucket number identification of the data to be migrated.
  • the bucket number identifier to be redistributed is the five bucket number identifiers from 5 to 9, and the five bucket number identifiers from 5 to 9 are used as the target bucket number identifier.
  • Step 103 Migrate the data to be migrated to the target shard database.
  • the metadata server determines the target database identified by the target bucket number after determining the bucket number identifier for data migration, and then migrates the data to be migrated to the target shard database according to the target bucket number identifier.
  • the metadata server determines the bucket ID to be migrated, and at the same time obtains the shard database to which the data corresponding to the target bucket ID is to be migrated.
  • the bucket ID is 5-7.
  • the target shard database marked with 3 bucket numbers is database 4
  • the target shard database with bucket numbers 7 and 8 is database 1.
  • all data with bucket numbers from 5 to 7 is migrated to database 4, and the bucket All data with the number identifier 7 or 8 are migrated to database 1, and after the data migration is completed, the mapping table between the bucket number identifier and the shard database is updated.
  • this embodiment provides a data redistribution method.
  • a sufficient number of bucket IDs are established according to the estimated amount of data to be stored.
  • the calculation result of the random hash algorithm is Add the bucket number identifier to the data to be stored, and according to the corresponding relationship between the bucket number identifier and the shard database, store the data to be stored in the corresponding shard database according to its own bucket number identifier, and store the data in logical partitions.
  • the second embodiment of the present application relates to a data redistribution method.
  • the second embodiment is roughly the same as the first embodiment.
  • the data to be stored is detected. Whether the added bucket number identifier is a prohibited bucket number identifier.
  • the bucket number identifier to be added is reassigned; when the data is redistributed, the target bucket number for data migration is identified Grouping, according to the bucket number identification contained in each group, the data to be migrated is migrated in batches, and when the data corresponding to a group of bucket number identification is migrated, the operation authority of the data to be migrated is locked first, and the migration is completed Then, unlock the operation authority again.
  • Fig. 2 The specific process of a data redistribution method in this embodiment is shown in Fig. 2, and specifically includes the following steps:
  • Step 201 Determine the identifier of the bucket number to be added to the data to be stored.
  • the metadata server determines the bucket number identifier to be added to the data record to be stored by using a random hash algorithm.
  • Step 202 Detect whether the determined bucket number identifier to be added is a bucket number identifier that is marked as prohibited to use; if it is a bucket number identifier that is marked as prohibited to use, go back to step 201, if it is not a bucket number marked as prohibited to use, Then go to step 203.
  • the determined bucket number to be added is detected Whether the number identifier is a bucket number identifier marked as prohibited to use; if it is a bucket number identifier marked as prohibited to use, go back to step 201 and re-determine the bucket number identifier to be added according to the random hash algorithm; if it is not marked as prohibited use If the bucket number is identified, step 203 is entered.
  • Step 203 Add a bucket number identifier to the data record to be stored and store it in the fragment database corresponding to the bucket number identifier.
  • the determined bucket number identifier to be added is added to the extended field of the data record to be stored, which is the data to be stored
  • the data to be stored After recording the added bucket number identifier, check whether the number of uses of the added bucket number identifier has reached the preset maximum number of uses; if the preset maximum number of uses has been reached, the added bucket number identifier will be marked as prohibited from use, and then stored
  • the data stores the data in the shard database according to the corresponding relationship between the bucket number identification and the shard database.
  • the bucket number identifier to be added is not marked as disabled, and the determined bucket number identifier to be added is added to the extension field of the data record to be stored, and the bucket number identifier is set according to the preset.
  • the shard database store the data record in the shard database corresponding to the bucket ID of the data record, and then check the usage count of the added bucket ID, and check whether the usage count of the added bucket ID has been The preset maximum number of uses is reached. If the preset maximum number of uses has been reached, the added bucket number identifier will be marked as prohibited. If the added bucket number identifier has not reached the preset maximum number of uses, the bucket number identifier will be updated. The number of uses.
  • the bucket number identifier By disabling a bucket number identifier when it is used enough times, when the calculated data result corresponds to a bucket number identifier that has been forbidden to use, the bucket number identifier is re-allocated, avoiding a bucket number identifier. There are too many corresponding data, which leads to a decrease in the performance of data migration.
  • Step 204 Obtain and group the target bucket ID of the data to be migrated.
  • the metadata server when the metadata server responds to the data redistribution request, it generates a complete redistribution plan based on the metadata information, and first obtains the target bucket ID of the data to be migrated, that is, determines the bucket to which all corresponding data records are to be migrated. Then divide the target bucket ID of the data to be migrated into N groups, where N is an integer greater than 1.
  • the metadata server After the metadata server receives the data redistribution request, according to the information of the data redistribution request, it is determined that the data records corresponding to the bucket ID 1 to 100 are to be redistributed.
  • the metadata server groups the bucket number identifiers of the hundred to-be-migrated data, for example, each ten bucket number identifiers are grouped according to 1 to 10, 11 to 20, and so on.
  • the number of groups and the number of bucket IDs contained in each group can be set according to actual needs.
  • the method for grouping target bucket IDs is not limited in this embodiment.
  • Step 205 Migrate the data to be migrated to the target database in batches.
  • the data to be migrated corresponding to the target bucket number identifiers of the same group is regarded as the data to be migrated in the same batch, and then the target bucket number identifiers of each group are divided into batches.
  • the metadata server locks the target bucket ID of the current batch of data to be migrated, migrates the current batch of data to be migrated to the target database, and updates the bucket ID and shard database. Corresponding relationship: unlock the target bucket ID of the current batch of data to be migrated after the migration is completed, and continue the next batch of data migration, until all the data to be migrated have completed the data migration.
  • the metadata server groups the bucket number identifiers of the data to be migrated, it randomly selects a group of bucket number identifiers, and performs data migration on the data corresponding to the bucket number identifiers. For example, if it is detected that the target bucket number of the current batch of data to be migrated is identified as 1 to 10, the metadata server stores the identification information of the bucket number to be locked in the metadata, and sends a lock bucket request to each database agent, and each database agent After receiving the request issued by the metadata server, it updates the lock bucket information in the memory, and detects the bucket ID corresponding to each shard database of its own. When detecting the bucket ID that needs to be locked, the database agent stops the bucket ID that needs to be locked.
  • the corresponding data performs DML (Data Manipulation Language) operations.
  • DML Data Manipulation Language
  • the database agent detects that some of the data corresponding to the bucket number identifier 7 is being accessed and modified by the running business, while the data corresponding to the remaining bucket number identifiers are in an unaccessed state.
  • the data under the 9 bucket IDs from 1 to 6 and 8 to 10 are locked for operation authority, and the business is prohibited from accessing and changing the data.
  • the data corresponding to the 9 bucket IDs is regarded as this batch of pending data.
  • For the migrated data after no data access to the data with the bucket ID of 7 is detected, the data with the bucket ID of 7 is migrated.
  • the data migration instruction After locking the bucket number identifier of the data to be migrated in the current batch, send a data migration instruction to the database agent.
  • the data migration instruction carries the target bucket number identifier of the data to be migrated in the current batch and the target shard database for the database agent.
  • the metadata server selects database agent 1 as the database agent to perform the data migration, and the metadata server sends a data migration instruction to the database agent 1.
  • the database agent constructs data migration SQL (Structured Query Language) statements according to the data migration instructions, and extracts the data with bucket numbers 1 to 6 from the source shard database to the storage space of the database agent 1 , And then import the acquired data from the storage space of the database agent to the shard database 11, clear its own data cache, and then migrate the data with bucket IDs 8 to 10 to the shard database 9 in the same way.
  • SQL Structured Query Language
  • database agent 1 After the data migration of this batch is completed, database agent 1 not only clears the data cached by itself, but also clears the data that has been migrated out of the sharded database from which the data has been migrated, and corresponds to each sharded database The bucket number identifier of is updated, and then a message that the data migration is successful is fed back to the metadata server.
  • the schematic diagram of the system architecture of the data migration method is shown in Figure 3.
  • the metadata server receives the redistribution request, the metadata server selects a set of bucket IDs of the data to be migrated, and issues a bucket lock instruction to each database agent for the selected bucket. The data corresponding to the number ID is locked, and then a data migration instruction is issued to the designated database agent.
  • the selected database agent will use the data (Bn) corresponding to a bucket ID in a fragmented database as the basic unit and cache it in its own storage space. Then the cached data is imported into the target shard database of the data to be migrated, and the bucket number identifier of the completed data migration is unlocked, and finally the corresponding relationship between the bucket number identifier and the shard database is updated.
  • the bucket number identification for preferential data migration can be determined by oneself according to the actual situation, and the order of data migration is not limited in this embodiment.
  • the data to be migrated is migrated in batches according to the bucket number identification, avoiding too much data in a single migration, occupying too much storage resources, and affecting the migration performance. At the same time, because the data is migrated in batches, the process of data migration is reduced. , The impact on the business.
  • migrating data corresponding to a data identifier in a group by locking the operation authority of the data according to the data identifier, problems such as data loss caused by accessing and operating the data during data migration are avoided, and the quality of data migration is ensured.
  • By updating the corresponding relationship between the bucket number identification and the shard database data access errors are avoided and the quality of data migration is ensured.
  • the data migration command is issued, and the data to be migrated is read and migrated by the database agent, eliminating the need to export the data to the file system and avoiding additional Export and import improve the performance and efficiency of data migration.
  • the metadata server changes the metadata for the bucket IDs 1 to 4 in the metadata.
  • the lock bucket information is released, and an unlock request is issued to each database agent.
  • the database agent unlocks the data corresponding to the bucket ID 1 to 4, and restores the database agent to access and change the data.
  • feedback the successful unlocking information to the metadata server.
  • the metadata server updates the corresponding relationship between the bucket number identifier and the shard database according to the feedback message after the data migration is completed, and restores the bucket number identifiers 1 to 1 to 4 Change and access the corresponding data.
  • the metadata server selects a new set of bucket IDs from the grouped bucket IDs, and performs data migration on the data corresponding to the bucket IDs until the data redistribution requests the corresponding data All completed the data migration and feedback the results of the data migration.
  • this embodiment provides a data redistribution method.
  • detecting the usage times of the bucket number identifier to be added it avoids too much data corresponding to a bucket number identifier and reduces the data migration performance;
  • when performing data migration By grouping the bucket numbers of the data to be migrated, the data is migrated in batches, which improves the performance of data migration and reduces the impact on the business;
  • by completing the data migration inside the database agent avoiding the data to be migrated in the file Additional import and export in the system improve the performance and efficiency of data migration; by locking the operation authority of the current batch of data before data migration, and unlocking it after the migration is completed, avoiding data loss and business occurrence Error; thereby realizing efficient and accurate online redistribution of data, improving the flexibility and usability of data redistribution, and greatly improving the user experience.
  • the third embodiment of the present application relates to an electronic device, as shown in FIG. 4, including at least one processor; and a memory communicatively connected with the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor , The instructions are executed by at least one processor, so that the at least one processor can execute the data redistribution method as described above.
  • the memory and the processor are connected in a bus manner, and the bus may include any number of interconnected buses and bridges, and the bus connects one or more processors and various circuits of the memory together.
  • the bus can also connect various other circuits such as peripheral devices, voltage regulators, power management circuits, etc., which are all known in the art, and therefore, will not be further described herein.
  • the bus interface provides an interface between the bus and the transceiver.
  • the transceiver may be one element or multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices on the transmission medium.
  • the data processed by the processor is transmitted on the wireless medium through the antenna, and further, the antenna also receives the data and transmits the data to the processor.
  • the processor is responsible for managing the bus and general processing, and can also provide various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions.
  • the memory can be used to store data used by the processor when performing operations.
  • the fourth embodiment of the present application relates to a computer-readable storage medium storing a computer program.
  • the computer program is executed by the processor, the above method embodiment is realized.
  • the program is stored in a storage medium and includes several instructions to enable a device ( It may be a single-chip microcomputer, a chip, etc.) or a processor (processor) that executes all or part of the steps of the method described in each embodiment of the present application.
  • the aforementioned storage media include: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disks or optical disks and other media that can store program codes. .
  • the embodiments of the present application provide a data redistribution method, electronic equipment, and storage medium, which enable efficient online data redistribution, reduce the impact of data migration on services, improve the scalability of data redistribution, and avoid hot issues, Ensure the efficiency and quality of data migration.

Landscapes

  • Engineering & Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Computing Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

L'invention concerne un procédé de redistribution de données, un dispositif électronique, et un support de stockage, le procédé consistant : à ajouter un identifiant de numéro de seau à un enregistrement de données à stocker, et le stocker dans une base de données de fragments correspondant à l'identifiant de numéro de seau (101) ; lors de la réception d'une requête de redistribution de données, à acquérir l'identifiant de numéro de seau cible des données à migrer (102) ; et à faire migrer les données à migrer vers une base de données de fragments cibles (103).
PCT/CN2020/116284 2019-11-05 2020-09-18 Procédé de redistribution de données, dispositif électronique, et support de stockage WO2021088531A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201911070817.5 2019-11-05
CN201911070817.5A CN112765262B (zh) 2019-11-05 2019-11-05 一种数据重分布方法、电子设备及存储介质

Publications (1)

Publication Number Publication Date
WO2021088531A1 true WO2021088531A1 (fr) 2021-05-14

Family

ID=75692567

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/116284 WO2021088531A1 (fr) 2019-11-05 2020-09-18 Procédé de redistribution de données, dispositif électronique, et support de stockage

Country Status (2)

Country Link
CN (1) CN112765262B (fr)
WO (1) WO2021088531A1 (fr)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115460230A (zh) * 2022-09-06 2022-12-09 中国科学技术大学 一种数据迁移方法及统一协调系统
WO2024109415A1 (fr) * 2022-11-23 2024-05-30 华为云计算技术有限公司 Procédé et système de redistribution d'une base de données, grappe de dispositifs et support de stockage

Families Citing this family (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113407491B (zh) * 2021-06-10 2024-10-29 北京猿力未来科技有限公司 数据处理方法及装置
CN113392067A (zh) * 2021-06-11 2021-09-14 北京金山云网络技术有限公司 一种针对分布式数据库的数据处理方法、装置及系统
CN113438645A (zh) * 2021-06-29 2021-09-24 恒安嘉新（北京）科技股份公司 一种5g网络下数据安全保护方法、装置、介质及电子设备
CN113377780B (zh) * 2021-07-07 2024-07-02 杭州网易云音乐科技有限公司 一种数据库分片方法、装置、电子设备和可读存储介质
CN113468148B (zh) * 2021-08-13 2023-02-17 上海浦东发展银行股份有限公司 一种数据库的数据迁移方法、装置、电子设备及其存储介质
CN113791736B (zh) * 2021-09-15 2024-10-18 京东科技信息技术有限公司 数据迁移方法、网卡设备、服务器及数据迁移系统
CN113641686B (zh) * 2021-10-19 2022-02-15 腾讯科技（深圳）有限公司 数据处理方法、装置、电子设备、存储介质及程序产品
CN114697376B (zh) * 2022-03-16 2024-09-24 浪潮云信息技术股份公司 一种超大消息传输方法及装置
CN115118604B (zh) * 2022-07-01 2023-04-11 杭州宇信数字科技有限公司 一种动态扩缩容的数据迁移方法、设备、系统和介质

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102033938A (zh) * 2010-12-10 2011-04-27 天津神舟通用数据技术有限公司 基于二级映射的集群动态扩展方法
US20110191389A1 (en) * 2005-07-28 2011-08-04 Vmware, Inc. Distributed data management system
CN103905503A (zh) * 2012-12-27 2014-07-02 中国移动通信集团公司 数据存取方法、调度方法、设备及系统
CN104615657A (zh) * 2014-12-31 2015-05-13 天津南大通用数据技术股份有限公司 一种节点支持多数据分片的分布式集群的扩容和缩容方法
CN108932256A (zh) * 2017-05-25 2018-12-04 中兴通讯股份有限公司 分布式数据重分布控制方法、装置及数据管理服务器

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7613707B1 (en) * 2006-12-22 2009-11-03 Amazon Technologies, Inc. Traffic migration in a multilayered data service framework
US7774329B1 (en) * 2006-12-22 2010-08-10 Amazon Technologies, Inc. Cross-region data access in partitioned framework
CN102968498B (zh) * 2012-12-05 2016-08-10 华为技术有限公司 数据处理方法及装置
US9489239B2 (en) * 2014-08-08 2016-11-08 PernixData, Inc. Systems and methods to manage tiered cache data storage
CN105808340B (zh) * 2014-12-29 2019-04-23 中移(苏州)软件技术有限公司 负载均衡方法及系统
CN106936899B (zh) * 2017-02-25 2021-02-05 九次方大数据信息集团有限公司 分布式统计分析系统的配置方法及分布式统计分析系统

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110191389A1 (en) * 2005-07-28 2011-08-04 Vmware, Inc. Distributed data management system
CN102033938A (zh) * 2010-12-10 2011-04-27 天津神舟通用数据技术有限公司 基于二级映射的集群动态扩展方法
CN103905503A (zh) * 2012-12-27 2014-07-02 中国移动通信集团公司 数据存取方法、调度方法、设备及系统
CN104615657A (zh) * 2014-12-31 2015-05-13 天津南大通用数据技术股份有限公司 一种节点支持多数据分片的分布式集群的扩容和缩容方法
CN108932256A (zh) * 2017-05-25 2018-12-04 中兴通讯股份有限公司 分布式数据重分布控制方法、装置及数据管理服务器

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115460230A (zh) * 2022-09-06 2022-12-09 中国科学技术大学 一种数据迁移方法及统一协调系统
WO2024109415A1 (fr) * 2022-11-23 2024-05-30 华为云计算技术有限公司 Procédé et système de redistribution d'une base de données, grappe de dispositifs et support de stockage

Also Published As

Publication number Publication date
CN112765262B (zh) 2023-02-28
CN112765262A (zh) 2021-05-07

Similar Documents

Publication Publication Date Title
WO2021088531A1 (fr) Procédé de redistribution de données, dispositif électronique, et support de stockage
US11442961B2 (en) Active transaction list synchronization method and apparatus
US20160026667A1 (en) Memory-aware joins based in a database cluster
US8364677B2 (en) Method and apparatus for generating partitioning keys for a range-partitioned database
US9576012B2 (en) Hierarchical tablespace space management
US10599677B2 (en) Methods and systems of splitting database indexes and digests
US8832113B2 (en) Data management apparatus and system
US11586641B2 (en) Method and mechanism for efficient re-distribution of in-memory columnar units in a clustered RDBMs on topology change
US10417257B2 (en) Non-blocking database table alteration
US20190102408A1 (en) Routing requests in shared-storage database systems
WO2022111188A1 (fr) Procédé de traitement de transaction, système, appareil, dispositif, support d'enregistrement et produit-programme
CN112789606A (zh) 数据重分布方法、装置及系统
US12038879B2 (en) Read and write access to data replicas stored in multiple data centers
EP3376403A1 (fr) Procédé d'accès à une base de données distribuée et dispositif fournissant un service de données distribué
CN112749198A (zh) 一种基于版本号的多级数据缓存方法及装置
US9747323B1 (en) Method for reconstruction of a distributed lock state after a node addition or removal using a consistent hash
CN107480254B (zh) 适用于分布式内存数据库的在线负载均衡方法
KR20140021345A (ko) 샤딩 서비스를 제공하는 방법 및 장치
US11741081B2 (en) Method and system for data handling
US20240045887A1 (en) Systems and methods for controlling replica placement in multi-region databases
CN111459913B (zh) 分布式数据库的容量扩展方法、装置及电子设备
US11138231B2 (en) Method and system for data handling
WO2023066222A1 (fr) Procédé et appareil de traitement de données, dispositif électronique, support de stockage et produit-programme
US12038901B2 (en) Method and system for non-blocking database logging batching
EP3686751A1 (fr) Procédé et système de gestion de données

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20885613

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20885613

Country of ref document: EP

Kind code of ref document: A1

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 22/02/2023)

122 Ep: pct application non-entry in european phase

Ref document number: 20885613

Country of ref document: EP

Kind code of ref document: A1