WO2023070935A1 - Procédé et appareil de stockage de données et dispositif associé - Google Patents

Procédé et appareil de stockage de données et dispositif associé Download PDF

Info

Publication number
WO2023070935A1
WO2023070935A1 PCT/CN2021/142795 CN2021142795W WO2023070935A1 WO 2023070935 A1 WO2023070935 A1 WO 2023070935A1 CN 2021142795 W CN2021142795 W CN 2021142795W WO 2023070935 A1 WO2023070935 A1 WO 2023070935A1
Authority
WO
WIPO (PCT)
Prior art keywords
data
copies
azs
file system
partition
Prior art date
Application number
PCT/CN2021/142795
Other languages
English (en)
Chinese (zh)
Inventor
阿瓦鲁卡纳卡•库马尔
库马尔潘卡吉
钱纳巴斯帕雷努卡普拉萨德
莫凯
Original Assignee
华为云计算技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为云计算技术有限公司 filed Critical 华为云计算技术有限公司
Priority to CN202180006938.2A priority Critical patent/CN116635831A/zh
Publication of WO2023070935A1 publication Critical patent/WO2023070935A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/14—Error detection or correction of the data by redundancy in operation

Definitions

  • the embodiments of the present application relate to the technical field of databases, and in particular, to a data storage method, device, and related equipment.
  • the data processing platform is used to provide users with data reading and writing services, such as data storage, data reading, etc., which may include distributed databases and file systems.
  • a distributed database such as an HBase database
  • a distributed database usually includes a master (master) node and multiple partition server (Region Server, RS) nodes.
  • the master node is used to assign to each RS node the region to which the data that the RS node is responsible for reading and writing belongs to.
  • the number of partitions assigned to each RS node can be one or more; the RS node is used to allocate Write new data to the file system, or feed back the data in the text system requested by the user to the user, so as to realize the data reading and writing service.
  • the embodiment of the present application provides a data storage method, so that when a disaster occurs, the quality of data reading and writing services provided by the data processing platform for users can be maintained at a relatively high level.
  • the present application also provides corresponding apparatuses, computing devices, computer-readable storage media, and computer program products.
  • the embodiment of the present application provides a data storage method, which can be applied to a data processing platform including multiple availability zones AZ, and when storing data, the data processing platform can first obtain multiple copy, and store multiple copies of the data to be stored in different AZs of the data processing platform.
  • the data processing platform Since the data processing platform stores multiple copies of the data to be stored in different AZs, when some AZs are unavailable due to natural disasters or other disasters, the physical distance between different AZs is usually large, therefore, The disaster usually does not affect other AZs, so the data processing platform can continue to provide users with data read and write services based on the copies of the data to be stored in other AZs that are running normally, thereby avoiding disasters and reducing the data processing platform. The quality of data read and write services. Moreover, based on this data storage method, the allowable service interruption time length of the data processing platform can be 0, that is, the recovery time target reaches 0; the maximum data loss that the data processing platform can tolerate can reach 0, that is, the recovery point target Can be 0.
  • the data processing platform stores multiple copies of the data to be stored in different AZs instead of in one AZ. This makes it impossible for all copies of the data to be stored to be lost after some AZs become unavailable. In the event of data loss, the reliability of data stored on the data processing platform can be improved.
  • the data to be stored includes the target data and/or the partition to which the target data belongs, then, when the data processing platform stores multiple copies of the data to be stored on the data processing platform, specifically, the Multiple copies of the target data are stored in different AZs of the data processing platform, and/or multiple partition copies of the partition to which the target data belongs are stored in different AZs of the data processing platform.
  • the data processing platform can obtain target data copies and/or partition copies from other AZs, so as to continue to provide users with data read and write services using the target data copies and/or partition copies in other AZs , to improve the reliability of data reading and writing services provided by the data processing platform.
  • the data to be stored includes target data and the partition to which the target data belongs
  • the data processing platform includes a distributed database and a file system
  • the distributed database includes partition servers RS under multiple AZs node
  • the file system includes data nodes under multiple AZs
  • the data processing platform when the data processing platform stores multiple copies of the data to be stored in different AZs, it may specifically store multiple partition copies in different AZs in the distributed database
  • the RS node under stores multiple data copies to the data nodes under different AZs in the file system. In this way, the reliability of data reading and writing services provided by the data processing platform can be improved.
  • multiple AZs under the distributed database and multiple AZs under the file system may overlap (all or part of the AZs may be the same), for example, the distributed database and the file system may have the same multiple AZs.
  • the multiple AZs under the distributed database do not overlap with the multiple AZs under the file system, for example, the distributed database includes AZ1 to AZ5, and the file system includes AZ6 to AZ10. In this way, the reliability of data reading and writing services provided by the data processing platform can be further improved.
  • the data processing platform when it stores multiple copies of data in data nodes under different AZs in the file system, it can first obtain the physical distance between different AZs in the file system, and according to the file The physical distance between different AZs in the system, determine the multiple first AZs in the file system, and the determined physical distance between the multiple first AZs does not exceed the distance threshold (such as 40 kilometers, etc.), so that the data processing The platform can store multiple data copies to the data nodes under the multiple first AZs. In this way, when some AZs fail and the data processing platform reads data copies from other AZs, since the other AZs are relatively close to the AZ, the data processing platform can quickly read the data copies, so that the data copy can be obtained Latency is kept low.
  • the distance threshold such as 40 kilometers, etc.
  • the data processing platform when the data processing platform stores multiple copies of data in data nodes under different AZs in the file system, it can obtain the availability of each AZ in the file system, for example, through The ratio between the data nodes available in the AZ and all data nodes is determined, so that the data processing platform can store multiple data copies in multiple first AZs in the file system according to the availability of each AZ in the file system data nodes, and the file system further includes at least one second AZ with a lower degree of availability, wherein the lower degree of availability of the second AZ means that the degree of availability of the second AZ is lower than that of the first AZ.
  • the data processing platform can preferentially select the data nodes under the first AZ with higher availability to store the data copy, so as to improve the reliability of the data processing platform for reading and writing data.
  • the data processing platform when the data processing platform stores multiple copies of data in data nodes under different AZs in the file system, it can obtain the availability of each AZ in the file system, for example, through The ratio between the data nodes available in the AZ and all data nodes is determined, so that the data processing platform can store some of the data copies in multiple copies of the data in multiple copies of the file system according to the availability of each AZ in the file system.
  • the file system also includes at least one second AZ, and the availability of the second AZ is lower than that of the first AZ; when the availability of the at least one second AZ rises to
  • the data processing platform may store other data copies in the multiple data copies to the data nodes under the at least one second AZ. In this way, when storing multiple data copies, it is possible to avoid migrating the data storage tasks required by AZs with low availability to other AZs, so as to avoid increasing the load of other AZs.
  • the data processing platform when storing multiple partition copies to RS nodes under different AZs in the distributed database, it may be specifically to obtain allocation indication information for the multiple partition copies, and the allocation indication information is used for Indicates the ratio of the number of copies of multiple partition copies stored in different AZs, so that the data processing platform can store multiple partition copies to RS nodes under different AZs in the distributed database according to the allocation instruction information.
  • the allocation indication information may be pre-configured by technicians or users, or the allocation indication information may be automatically generated by the data processing platform, and the like. In this way, the data processing platform can implement cross-AZ storage of multiple partition copies according to the allocation indication information.
  • the allocation indication information can be determined according to the load of the RS nodes under each AZ in the distributed database. In this way, when the data processing platform stores multiple partition copies according to the allocation indication information, it can balance multiple AZ load.
  • the distributed database and the file system both include multiple target AZs, and the multiple target AZs have already stored multiple partition copies of the partition to which the target data belongs, then when storing multiple data copies, It can track multiple target AZs that store multiple partition copies, and store multiple data copies to data nodes under different target AZs.
  • the distributed database can read the target data from the local (that is, the AZ where the partition copy is located) data node based on the partition copy, thereby reducing the delay in reading data and improving the data quality. The efficiency with which the processing platform feeds back data to users.
  • the data processing platform can perform load balancing on the RS nodes under different AZs in the distributed database to adjust The number of partition replicas stored in different AZs. In this way, the load balancing of different RS nodes is realized, and the excessive load of some ES nodes is prevented from affecting the quality of data reading and writing services provided by the data processing platform as a whole.
  • the embodiment of the present application provides a data storage device.
  • the device has functions corresponding to each implementation manner for realizing the above-mentioned first aspect.
  • This function may be implemented by hardware, or may be implemented by executing corresponding software on the hardware.
  • the hardware or software includes one or more modules corresponding to the above functions.
  • the present application provides a computing device, where the computing device includes a processor, a memory, and a display.
  • the processor and the memory communicate with each other.
  • the processor is configured to execute instructions stored in the memory, so that the computing device executes the data storage method in the first aspect or any implementation manner of the first aspect.
  • the present application provides a computer-readable storage medium, where instructions are stored in the computer-readable storage medium, and when the computer-readable storage medium is run on a computing device, the computing device executes the above-mentioned first aspect or any of the first aspects.
  • the present application provides a computer program product containing instructions, which, when run on a computing device, causes the computing device to execute the data storage method described in the first aspect or any implementation manner of the first aspect .
  • FIG. 1 is a schematic diagram of the architecture of an exemplary data processing platform 100 of the present application
  • FIG. 2 is a schematic structural diagram of a cluster 200 constructed across availability zones
  • FIG. 3 is a schematic flow diagram of a data storage method provided by an embodiment of the present application.
  • FIG. 4 is a schematic diagram of storing multiple data copies provided by the embodiment of the present application.
  • FIG. 5 is a schematic structural diagram of a data storage device provided by an embodiment of the present application.
  • FIG. 6 is a schematic diagram of a hardware structure of a computing device provided by an embodiment of the present application.
  • the data processing platform 100 includes a distributed database 101 and a file system 102 .
  • the file system 102 can be used to persistently store data in the form of files
  • the distributed database 101 can be used to manage the data in the file system 102, including reading, writing, and merging of data.
  • the file system 102 includes multiple data nodes (datanodes), and different data nodes may belong to different availability zones (availability zones, AZ).
  • the file system 102 includes data nodes 1021 to 1024 as an example for illustration, and data nodes 1021 and 1022 belong to AZ1, and data nodes 1023 and 1024 belong to AZ2.
  • AZ usually refers to a collection of one or more physical data centers, with independent wind, fire, water and electricity, and computing, network, storage and other resources can be logically divided into multiple clusters within the AZ.
  • the file system 102 may be, for example, a distributed file system (distributed file system, DFS), a Hadoop distributed file system (hadoop distributed file system, HDFS), etc., which are not limited in this embodiment.
  • the file system 102 may also include a named node (namenode), the named node (not shown in FIG. 1 ) may also be called a master node, and is used to manage multiple data nodes, including namespace, metadata that records the data stored in each data node, etc.
  • the distributed database 101 includes a master node 1011 and multiple partition server (region server, RS) nodes, and different RS nodes belong to different availability zones.
  • the distributed database 101 includes the RS node 1012 and the RS node 1012 is taken as an example for illustration, and the RS node 1012 belongs to AZ1, and the RS node 1012 belongs to AZ2.
  • the master node 1011 is used to divide the data managed by the distributed database 101 (that is, the data stored in the file system 102) to obtain multiple partitions, each partition includes one or more data identifiers, and belongs to Data in different partitions often differs.
  • partitioning when managing each piece of data in the distributed database 101, part of the content in the piece of data can be used as a primary key (primary key) corresponding to the piece of data, and the primary key is used for This piece of data is uniquely identified in the distributed database 101, so that the primary node 1011 can perform interval division according to the possible value range of the primary key, and each divided interval corresponds to a partition.
  • the primary node 101 can divide the value range of the primary key into 100 intervals, which are respectively [0, 10000), [10000, 20000), ..., [980000, 990000), [990000, 1000000], each partition can be used to index 10,000 pieces of data, correspondingly, based on the 100 partitions, the distributed database 101 can manage 1 million Article data.
  • the master node 1011 can also achieve high availability through a distributed application coordination service (such as zookeeper service, etc.) 1014, as shown in FIG. 1 .
  • the master node 1011 is also used to allocate partitions for the RS nodes 1012 and 1013 , and the partitions assigned to each RS node can be maintained through the management table created by the master node 1011 .
  • RS node 1012 and RS node 1013 are respectively used to perform data read and write services belonging to different partitions. As shown in Figure 1, RS node 1012 performs data read and write services belonging to partition 1 to partition N, while RS node 1013 performs data read and write services belonging to partition N+1. Data read and write services to partition M.
  • the master node 1011, the RS node 1012 and the RS node 1013 can all be implemented by hardware or software.
  • both the master node 1011 and multiple RS nodes may be physical servers in the distributed database 101 . That is, during actual deployment, at least one server in the distributed database 101 may be configured as the master node 1011, and other servers in the distributed database 101 may be configured as RS nodes.
  • the master node 1011 and each RS node are implemented by software.
  • the master node 1011 and multiple RS nodes may be processes or virtual machines running on one or more devices (such as servers, etc.).
  • the data processing platform 100 shown in FIG. 1 is only used as an exemplary illustration, and is not used to limit the specific implementation of the data processing platform.
  • the distributed database 101 may include any number of master nodes and RS nodes, or the RS nodes in the distributed database 101 and the data nodes in the file system 102 may belong to different AZ, which is not limited in this application.
  • the distributed database 101 can be respectively connected with the file system 102 and the client 103, for example, it can be connected through a wireless communication protocol such as a hypertext transfer protocol (HyperText Transfer Protocol, HTTP).
  • HTTP HyperText Transfer Protocol
  • the client 103 can send a data write request to the RS node 1012, and the data write request carries the data to be written and the corresponding data processing Operation content (such as write operation, modification operation, etc.).
  • the RS node 1012 can analyze and receive the data write request, generate a corresponding data processing record based on the data to be written and the data processing operation, and write the data processing record into a pre-created write ahead log (write ahead log, WAL) file. After determining that the writing of the WAL file is successful, the RS node 1012 persistently stores the WAL file to the file system 102 . And, the RS node 1012 inserts the data to be written in the data processing record into the internal memory 1 of the RS node 1012 .
  • WAL write ahead log
  • the RS node 1012 can first determine the primary key corresponding to the data processing record, and determine which partition to write the data processing record into according to the partition interval to which the value of the key belongs, so that the RS node 1012 can process the data
  • the data to be written in the record is inserted into the storage area corresponding to the partition in the memory 1; then, the RS node 1012 can feed back to the client 103 that the data writing is successful.
  • the RS node 1012 will write data in the memory 1 for one or more clients 103, so the amount of data temporarily stored in the memory 1 will continue to increase.
  • the RS node 1012 can persistently store the data in the memory 1 to the file system 102, such as persistently storing the data in the memory 1 to the data node 1021 under AZ1.
  • the RS node 1012 is also configured with a region store file (region store files) for each partition, and after persistently storing data in the file system 102, the RS node 1012 can store the data of each partition in the file system 102
  • the files stored in the partition are added to the partition storage file corresponding to the partition.
  • the file name corresponding to each data in the partition can be added in the directory of the partition storage file.
  • the client 103 when it needs to read data, it can send a data read request to the RS node 1012, and the data read request carries the primary key of the data to be read.
  • RS node 1012 After receiving the data write request, RS node 1012 can determine the partition according to the value of the primary key corresponding to the data to be written, so that RS node 1012 can store files from the data nodes under AZ1 according to the partition storage file corresponding to the partition. Find out the data required by the client 102 and feed it back to the client 103 .
  • the data processing platform 100 When the data processing platform 100 is deployed, the data processing platform 100 can be used as a local resource to provide local data reading and writing services to clients accessing the data processing platform 100 through the distributed database 101 and the file system 102 .
  • the data processing platform 100 can also be deployed on the cloud, and at this time, the distributed database 101 and the file system 102 can provide cloud services for reading and writing data to clients connected to the cloud.
  • AZs such as AZ1 or AZ2
  • the disaster causes physical damage to some or all computing devices under the AZ.
  • the data stored in the partitions in the RS nodes under the AZ and/or in the data nodes under the AZ may be lost or unreadable due to the unavailability of the AZ, which will reduce the data processing platform 100.
  • the quality of data reading and writing services that is, it is difficult for users to obtain the data stored in the AZ, which affects the user experience.
  • the embodiment of the present application provides a data storage method, aiming at allocating partitions and/or copies of data to different AZs, so as to improve the reliability of data reading and writing services provided by the data processing platform 100 .
  • the data processing platform 100 may acquire multiple copies of the data to be stored, and store the multiple copies of the data to be stored in different AZs of the data processing platform 100 .
  • the data to be stored may be data newly written by the user and/or the partition to which the data belongs, for example.
  • the data processing platform 100 replicates the data to be stored, and different copies of the data to be stored are stored in different AZs, when AZ1 (or AZ2) is unavailable due to natural disasters or other disasters, due to different The physical distance between the AZs is usually relatively large. Therefore, although the disaster makes AZ1 unavailable, it usually does not affect AZ2, so that the data processing platform 100 can be based on the data to be stored stored in AZ2 (or AZ1) in normal operation. continue to provide data reading and writing services for users, thereby avoiding disasters and reducing the quality of data reading and writing services provided by the data processing platform 100.
  • the allowable service interruption time length of the data processing platform 100 can be 0, that is, the recovery time objective (RTO); the maximum amount of data loss that the data processing platform 100 can tolerate can reach 0, that is, the recovery point objective (recovery point objective, RPO) can be 0.
  • RTO recovery time objective
  • RPO recovery point objective
  • the data processing platform 100 stores multiple copies of the data to be stored in different AZs, rather than storing them in one AZ. This makes it impossible for all copies of the data to be stored to be stored after some AZs become unavailable. All data loss occurs, so that the reliability of data storage on the data processing platform 100 can be improved.
  • the distributed database 101 obtains the partition to which the data belongs during the process of storing multiple copies of the data to be stored by the data storage platform 100 Multiple copies of partitions, such as obtaining multiple copies of partitions by copying partitions. Then, the distributed database 101 stores multiple partition copies to RS nodes under different AZs included in the distributed database 101, such as storing part of the partition copies to the RS node 1012 under AZ1, and storing the remaining partition copies to AZ2 RS node 1013 .
  • the file system 102 also obtains multiple data copies of the data, for example, the distributed database 101 sends multiple data copies obtained by duplicating the data to the file system 102, or the file system 102 sends the distributed database 101 The sent data is copied to obtain multiple copies of the data, etc. Then, the file system 102 stores multiple data copies to data nodes under different AZs included in the file system 102. For example, part of the data copies are stored in the data nodes under AZ1, and the remaining data copies are stored in the data nodes under AZ2. .
  • the data processing platform 100 Since the data processing platform 100 stores multiple copies of data in different AZs rather than in one AZ, this will not cause data loss for all data copies of one piece of data when some AZs become unavailable. Therefore, the reliability of data stored by the data processing platform 100 can be improved. Similarly, the data processing platform 100 stores multiple partition copies in different AZs, so that when some AZs become unavailable, data loss will not occur in all partition copies corresponding to one piece of data, so that the data processing platform can 100 The quality of data read based on partition replicas is maintained at a high level.
  • computing devices including RS nodes, data nodes, etc.
  • the data processing platform 100 can utilize the cluster Provide reliable data read and write services.
  • the deployed cluster can be shown in FIG. 2 , and the cluster 200 can include computing devices under AZ1 and AZ2.
  • the cluster 200 can include RS nodes 1012, data nodes 1021, and data nodes 1022 under AZ1.
  • AZ1 also includes other nodes under the AZ1, such as a named node 1031 for updating the namespace of the cluster 200, recording metadata of data stored in the data node, a log node (journalnode) 1032 for storing and managing logs, A resource management device 1033 for managing resources in AZ1, a node management device 1034 for managing data nodes, a node management device 1035, and a master server 1036 for monitoring and managing RS nodes 1012 (such as Hmaster etc.); Moreover, a distributed application program coordination service may also be configured in the AZ1 to improve the high availability of the AZ1. Similarly, AZ2 and AZ1 also have similar configurations, as shown in Figure 2 for details.
  • the computing device under AZ1 and the computing device under AZ2 can be active and standby each other.
  • the named node 1031 under AZ1 fails, the named node 1041 under AZ2 can update the namespace of the cluster 200, etc.
  • the cluster 200 shown in FIG. 2 is based on the deployment of the data processing platform 100 shown in FIG. For each computing device under AZ3 shown by the dotted line, this embodiment does not limit the specific architecture of the cluster deployed across AZs. Moreover, multiple clusters may be deployed across AZs in the data processing platform 100, and different clusters include computing devices under different AZs. For example, assuming that the data processing platform 100 includes 6 AZs, namely AZ1 to AZ6, one cluster may be deployed based on AZ1 to AZ3, and another cluster may be deployed based on AZ4 to AZ6, which is not limited in this embodiment.
  • a standby cluster 300 may also be deployed for the cluster 200, so as to replicate and retrieve data stored in the cluster 200 (including partitions, data written in the file system 102, etc.) storage, so that the reliability of data read and write services provided by the cluster 200 can be further improved.
  • an asynchronous replication method may be adopted between the cluster 200 and the cluster 300 to replicate data in the cluster 200 to the cluster 300 .
  • FIG. 3 it is a schematic flowchart of a data storage method in an embodiment of the present application.
  • the method can be applied to the data processing platform 100 shown in FIG. 1 above, and specifically can be executed by the distributed database 101 and the file system 102 . Alternatively, the method may also be executed by a device separately configured in the data processing platform 100, which is not limited in this embodiment.
  • the data to be stored includes target data (such as the new data provided by the above-mentioned user) and the partition of the target data, and the distributed database 101 and the file system 102 execute the data storage method as an example.
  • the data storage method shown in Figure 3 may specifically include:
  • the distributed database 101 acquires multiple data copies of the target data, and multiple partition copies of the partition to which the target data belongs.
  • the target data acquired by the distributed database 101 may be, for example, new data provided by the user to the data processing platform 100, or data generated by the data processing platform 100 based on the user's modification operation on the data.
  • the data processing platform 100 can write the primary key contained in the target data into the partition, so that the target data can be managed subsequently based on the primary key recorded in the partition; and, the data processing platform 100 can also write the The target data is persistently stored in the file system 102 .
  • the distributed database 101 can respectively replicate the target data and the partition to which the target data belongs, so that multiple data copies of the target data can be obtained (the target data itself can also be regarded as a data copy), a partition copy (The partition to which the target data belongs can also be regarded as a partition copy).
  • the distributed database 101 is used as an example to replicate partitions.
  • the distributed database 101 executes the replication operation on the partition to which the target data belongs to obtain multiple partition copies, and The target is sent to the file system 102, so that the file system 102 executes the copy operation of the target data to obtain multiple data copies.
  • This embodiment does not limit it.
  • the distributed database 101 stores multiple partition copies to RS nodes under different AZs included in the distributed database 101.
  • the distributed database 101 stores all partition copies of the target data in one AZ, then when the AZ is unavailable due to a disaster, it is difficult for the distributed database 101 to use the partition copies in the AZ to provide users with data for the target data Read and write services. Therefore, in this embodiment, the distributed database 101 stores multiple partition copies in at least two AZs, so that even if one of the AZs is unavailable, the distributed database 101 can also manage the target through the partition copies stored in the remaining AZs. data. In this way, unreadable target data in the data processing platform 100 due to the unavailability of a single AZ can be avoided, and data fault tolerance at the AZ level can be realized.
  • the number of partition copies allocated to different AZs may be the same or different.
  • the distributed database 101 when the distributed database 101 includes 3 AZs and the number of partition copies to which the target data belongs is 3, the distributed database 101 can store the 3 partition copies in the 3 AZs respectively, and each AZ stores a copy of the partition.
  • the distributed database 101 when the distributed database 101 stores three partition copies in AZ1 and AZ2 (not in AZ3), one partition copy can be stored in AZ1, two partition copies can be stored in AZ2, and so on.
  • this embodiment provides the following four implementations for storing multiple partition copies in different AZs:
  • the distributed database 101 may acquire the allocation indication information of the multiple partition copies, the allocation indication information may include the AZ identifier and the proportion of the number of copies, and may be used to indicate that the multiple partition copies The proportion of copies stored in different AZs. In this way, the distributed database 101 can store multiple partition copies to RS nodes under different AZs in the distributed database according to the allocation indication information.
  • the distributed database 101 can store 2 (ie 0.5*4) partition copies in AZ1, store 1 (ie 0.25*4) partition copies in AZ2, and store 1 partition copy in AZ2 Stored in AZ3. It is worth noting that among the multiple partition copies written in multiple AZs in the distributed database 101, one of the partition copies is used as the primary partition copy, that is, the distributed database 101 usually provides data read and write for users based on the primary partition copy. service, while the rest of the partition copies are used as secondary partition copies to continue to provide users with data read and write services based on the secondary partition copies when the primary partition copy is unreadable or data loss occurs.
  • the distributed database 101 may process copies of multiple partitions in batches based on the above allocation indication information. For example, assuming that there are currently 10 partitions, and the number of copies of each partition is 4, then when the allocation indication information is "REP: AZ1[0.5], AZ2[0.25], AZ3[0.25]", the distributed database 101 can Store 20 (that is, 0.5*4*10) copies of the partition in AZ1, store 10 (that is, 0.25*4*10) copies of the partition in AZ2, and store 10 copies of the partition in AZ3. Among them, 3 AZs store a copy of each partition.
  • the allocation indication information may be determined by the distributed database 101 according to the load of the RS nodes in each AZ, for example. For example, when it is assumed that the load of RS node 1012 in AZ1 is 10%, the load of RS node 1013 in AZ2 is 30%, and the load of RS node 1014 in AZ3 is 35%, and the number of copies of each partition is set is 4, the distributed database 101 can determine that the allocation indication information is specifically "REP: AZ1[0.5], AZ2[0.25], AZ3[0.25]", that is, AZ1 is used to store 50% of the partition copies, and AZ2 and AZ3 are both used To store 25% of the partition copies, so as to balance the load among RS nodes under different AZs.
  • the allocation instruction information may also be determined in other ways, such as manual configuration by a technician in advance, which is not limited in this embodiment.
  • the distributed database 101 may include a main server (such as the main server 1036 in FIG. 2 ) and an AZ aware balancer (AZ aware balancer), wherein the main server can perceive effective RS nodes, such as Valid RS nodes can be determined based on heartbeat messages sent by RS nodes, so that the master server can generate a network topology map based on valid RS nodes.
  • the master server can forward the allocation request to the AZ-aware balancer, and the AZ-aware balancer will use the AZ-based network
  • the topology map (AZ based network topology), together with the above allocation instructions, determines the AZ used to store the partition replica.
  • the distributed database 101 can also be provided with a default balancer (default balancer), and, for some partitions that do not need to be replicated and stored (such as the importance of the partition is low, etc.), the distributed database 101 can also use this
  • a default balancer is used to determine the AZ where the partition is stored.
  • the default balancer may, for example, determine the AZ of the storage partition through a random algorithm or a load balancing strategy.
  • the main server, the AZ-aware equalizer, and the default equalizer can all be implemented by software (such as a process) or hardware (such as a separately configured computing device), which is not limited in this embodiment.
  • the distributed database 101 may first create AZ allocates a partition copy, and, for the remaining unassigned partition copies, the distributed database 101 can determine the AZ where the RS node with a relatively small load is located according to the current load of the RS nodes in each AZ, so that the distributed database 101 may allocate the remaining partition copies to the AZ for storage. In this way, the distributed database 101 can flexibly allocate storage locations of partition copies according to the load conditions of RS nodes under multiple AZs in various time periods, so as to realize load balancing of RS nodes.
  • the distributed database 101 can be based on each AZ Availability, which determines the AZ where the replica of the partition is stored. Specifically, during the process of writing partition copies to multiple AZs, the distributed database 101 may first obtain the availability degree of each AZ for each AZ. The ratio between the total number of data nodes in the AZ is determined and can reflect the availability of the AZ. For example, assuming that the AZ includes 10 data nodes, and the number of available nodes is 8, the availability of the AZ may be 80% (ie, 8/10).
  • the available data node refers to the data node with further normal ability to read and write data; correspondingly, when the data node is physically damaged or the data is read and written incorrectly, the data node can be determined to be faulty, that is, unavailable , or, when the amount of data stored in the data node is too large and the data node cannot further store data, the data node may also be determined as an unavailable data node.
  • whether a data node is available can also be defined in other ways, for example, when the data to be stored does not have the permission to be written into the data node, the data node can be determined to be unavailable relative to the data to be stored, etc.
  • the distributed database 101 can determine a plurality of first AZs whose availability is higher than a preset threshold and at least one second AZ whose availability is lower than a preset threshold according to the availability of each AZ, so that the distributed database 101 can A plurality of partition copies are written to the plurality of first AZs. In this way, the number of new partition copies written by the distributed database 101 to the AZ with a low degree of availability can be reduced, thereby avoiding the excessive load of the RS node under the AZ with a low degree of availability, causing the RS node to be responsible for The amount of data read exceeds the maximum amount of data that can be stored in this AZ.
  • the distributed database 101 when the distributed database 101 writes the partition copy to the AZ, some AZs may be less available or become unavailable. Based on this, the distributed database 101 can Suspend writing partition copies to this part of the AZ. Specifically, before the distributed database 101 writes a partition copy to one of the AZs, it may first obtain the availability of the AZ, and if the availability of the AZ is lower than the preset threshold, the distributed database 101 may not write to the AZ. Instead, the master server can create a cache queue for the AZ, such as a region in transaction (RIT) queue, etc., mark the partition copy as unallocated and write it to the cache queue middle. Then, the distributed database 101 continues to write the partition copy to the next AZ.
  • RIT region in transaction
  • the distributed database 101 can switch the identity of the master server in other AZs with a high degree of availability from "standby" to "primary". identity, and create a cache queue for that AZ.
  • the distributed database 101 can monitor the availability of the AZ, such as by configuring the RIT work (chore) node for monitoring, and if the availability of the AZ rises and exceeds the preset threshold, Then the distributed database 101 can write the partition copy in the cache queue corresponding to the AZ into the RS node belonging to the AZ.
  • the distributed database 101 may write the secondary partition copy to the AZ after the availability of the AZ increases. If the partition copy stored in the AZ with low availability is the primary partition copy, at this time, because the availability of the AZ is too low, the distributed database 101 can select a secondary partition copy from other AZs with high availability.
  • the partition replica acts as the primary partition replica to ensure the high availability of the primary partition replica in the distributed database 101, and the partition replica in the AZ acts as the secondary partition replica.
  • the distributed database 101 may also store multiple partition copies in different AZs based on other methods.
  • the distributed database 101 may By combining the above implementation methods, multiple AZs with high availability and close physical distances are selected to store multiple partition copies, etc.
  • the file system 102 stores multiple data copies to data nodes under different AZs in the file system 102 .
  • multiple data copies of the target data may be provided by the distributed database 101 to the file system 102, or the file system 102 may replicate the target data to obtain the multiple data copies. Similar to the storage partition copy, the file system 102 may store the obtained multiple data copies in different AZs, specifically, the data nodes in different AZs. Wherein, each AZ includes at least one data copy. In this way, even if one of the AZs where the data copy is stored is unavailable, the distributed database 101 can read the target data from the remaining AZs, so as to avoid loss of the target data and achieve AZ-level data fault tolerance.
  • this embodiment provides the following four implementations for storing multiple copies of data in different AZs:
  • the file system 102 may also store multiple data copies based on allocation indication information.
  • the allocation indication information may include AZ identification And the proportion of the number of copies, which can be used to indicate the proportion of the number of copies of the data nodes that store the multiple copies of data in different AZs.
  • the allocation indication information can specifically be the allocation expression "REP: AZ1[0.5], AZ2[0.25], AZ3[0.25]", then According to the allocation expression, the distributed database 101 can store 2 (ie 0.5*4) copies of data in AZ1, store 1 (ie 0.25*4) copies of data in AZ2, and store 1 copy of data in AZ2 Stored in AZ3. In this way, the file system 102 can store multiple data copies to data nodes under different AZs in the file system 102 according to the allocation indication information.
  • the allocation instruction information may be determined according to the load of the data nodes under each AZ, or manually configured by a technician, which is not limited in this embodiment.
  • a named node remote procedure call server (namenode remote procedure call server) and an availability zones block placement policy (availability zones block placement policy, AZ BPP) node may be set in the file system 102.
  • the named node remote procedure call server can instruct the AZ BPP node to execute the replication process of the target data; the AZ BPP node can determine multiple AZs and each The number of copies of the target data stored in the AZ, so that the corresponding data storage process can be executed.
  • a default block placement policy (default block placement policy) node may also be set in the file system 102, and, for some data that does not need to be copied and stored (such as the importance of the data is low, etc.), the file system 102 may also The default block placement strategy node is used to determine the AZ for storing the data.
  • the default block placement strategy node can be, for example, the AZ for storing data determined by a random algorithm or a load balancing strategy.
  • the file system 102 can generate an AZ based network topology based on the data nodes under each AZ, so that the default block placement policy node can determine the AZ for storing data copies according to the network topology.
  • the file system 102 may determine an AZ for storing data copies according to physical distances between different AZs. Specifically, the file system 102 can obtain the physical distance between different AZs in the file system 102, and store multiple copies of data in multiple copies of the data in the file system 102 that are physically closer to each other according to the physical distance between different AZs.
  • the data nodes under the first AZ that is, the physical distance between each first AZ and at least one of the first AZs in the plurality of first AZs does not exceed the distance threshold.
  • the file system 102 also includes at least one data node under the second AZ, and the physical distance between the second AZ and each first AZ exceeds a distance threshold (such as 40 kilometers, etc.).
  • a distance threshold such as 40 kilometers, etc.
  • the file system 102 can be based on each AZ Availability status, determine the AZ where the data copy is stored. Specifically, the file system 102 can first obtain the availability of each AZ, which can be determined by, for example, calculating the ratio between the number of data nodes available in the AZ and the total number of data nodes in the AZ, and can reflect AZ AVAILABILITY.
  • the file system 102 can store multiple copies of data in the data nodes under the multiple first AZs with relatively high availability in the file system 102 according to the availability of each AZ, each At least one data copy is stored in the first AZ.
  • the file system 102 also includes at least one data node under the second AZ, and the availability of the second AZ is lower than that of the first AZ. In this way, the file system 102 can preferentially allocate the data copy to an AZ with a high degree of availability for storage, thereby improving the reliability of data reading and writing of the data processing platform 100 .
  • the file system 102 when the file system 102 writes data copies to the AZ, some AZs may be less available or become unavailable. Based on this, the file system 102 can suspend Write a data copy to this part of AZ. Specifically, before writing multiple data copies to multiple AZs, the file system 102 can first obtain the availability of each AZ, and the file system 102 can write part of the multiple data copies according to the availability of each AZ The copy is stored in the data nodes under the first AZ with higher availability (higher than the preset threshold) in the file system 102, and at least one data copy is stored in each first AZ.
  • the file system 102 may temporarily not write a data copy to the second AZ. Then, when the availability of the second AZ rises to a preset threshold, the file system 102 stores the rest of the unstored data copies in the second AZ. In this way, when the file system 102 stores multiple data copies, it can avoid migrating the data storage task of the AZ with low availability to other AZs, so as to avoid increasing the load of other AZs.
  • the file system 102 may sequentially write three data copies of the data A into data node 1 under AZ1, data node 2 under AZ2, and data node 3 under AZ3; and
  • the file system 102 can suspend writing the copy of data B to AZ2.
  • a copy of data B is written, while a copy of data C is first written to AZ3.
  • the file system 102 writes a copy of data B into AZ2 according to the data copy stored in AZ1 or AZ3.
  • the file system 102 can also store multiple copies of data in different AZs based on other methods.
  • the file system 102 can be combined In the above implementation manner, multiple AZs with high availability and close physical distances are selected to store multiple copies of data.
  • the distributed database 101 and the file system 102 may include multiple identical AZs, or may not include the same AZ, which is not limited in this embodiment.
  • the file system 102 can track multiple target AZs that store multiple partition copies during the process of storing data copies (The file system 102 also includes the multiple target AZs), so the file system 102 can also store multiple data copies in data nodes under different target AZs.
  • the file system 102 and the distributed database 102 may store data copies and partition copies in the same AZ based on the same allocation indication information.
  • the distributed database 101 can read the target data from the local (that is, the AZ where the partition copy is located) data node based on the partition copy, thereby reducing the time delay for reading data and improving
  • the data processing platform 100 feeds back the efficiency of the data to the user.
  • the distributed database 101 and the file system 102 do not include the same AZ, the distributed database 101 and the file system 102 can independently execute the stored procedure of the copy.
  • N is a positive integer greater than 1
  • the data processing platform 100 can also be based on the partitions stored in the Nth AZ Replicas and data replicas continue to provide users with data read and write services.
  • the data processing platform 100 can also automatically restore the data copies and partition copies stored in the remaining N-1 AZs based on the partition copies and data copies stored in the Nth AZ. Data, so that there is no need for management personnel to intervene, and automatic processing after fault recovery is realized.
  • the target data and the partition to which the target data belongs are copied and stored across AZs as an example.
  • the data processing platform 100 can store them in a similar manner as described above to improve data
  • the processing platform 100 provides reliability of data reading and writing services. In practical applications, there may be differences in the importance of different data based on users.
  • the data processing platform 100 can store the user's data A in the above-mentioned manner, and when storing the user's data B, there is no need to copy and store the data B, or multiple data of the data B Replicas are stored in one AZ.
  • the data processing platform 100 may also periodically balance the loads of the RS nodes under each AZ. For example, the data processing platform 100 can periodically obtain the number of partition copies stored by the RS nodes under each AZ, so that the RS nodes under different AZs in the distributed database 101 can be loaded according to the number of partition copies in each RS node. Balance, specifically, may be migrating partition copies on some RS nodes to other RS nodes, so as to adjust the number of partition copies stored in different AZs, thereby reducing the difference in the number of partition copies stored in different AZs.
  • the data processing platform 100 A partition copy on the RS node under AZ3 can be migrated to the RS node under AZ1, so that the number of partition copies stored in each AZ is 2.
  • the data to be stored includes the target data and the partition to which the target data belongs as an example. In other possible embodiments, the data to be stored may only be the target data.
  • the data processing platform 100 can store multiple copies of the target data in different AZs through the file system 102, and the partition to which the target data belongs can be stored in a single AZ (one partition data can be saved in this AZ, It is also possible to store multiple copies of the partition at the same time), or it can be stored in multiple AZs. Alternatively, the data to be stored can also be only the partition to which the target data belongs.
  • the data processing platform 100 can store the data of multiple partitions in different AZs through the distributed data set 101, and the target data can be stored in the file system In a single AZ under the file system 102 (one copy of the target data can be stored in the AZ, or multiple copies of the target data can be stored at the same time), or can be stored in multiple AZs under the file system 102.
  • FIG. 5 is a schematic structural diagram of a data storage device provided by the present application.
  • the data storage device 500 can be applied to a data processing platform (such as the above-mentioned data processing platform 100 etc.), and the data processing platform includes multiple availability zones AZ.
  • the data recovery device 500 includes:
  • An acquisition module 501 configured to acquire multiple copies of the data to be stored
  • the storage module 502 is configured to store multiple copies of the data to be stored in different AZs of the data processing platform.
  • the data to be stored includes target data and/or the partition to which the target data belongs, and the storage module 502 is configured to:
  • the data to be stored includes target data and the partition to which the target data belongs
  • the data processing platform includes a distributed database and a file system
  • the distributed database includes multiple availability zones AZ
  • the RS node of the partition server, the file system includes data nodes under multiple AZs, and the storage module 502 is used for:
  • the multiple data copies are stored in data nodes under different AZs in the file system.
  • the storage module 502 is configured to:
  • the storage module 502 is configured to:
  • the multiple data copies are stored in the data nodes under the multiple first AZs in the file system, and the file system also includes at least one data node under the second AZ For a data node, the availability of the second AZ is lower than the availability of the first AZ.
  • the storage module 502 is configured to:
  • each AZ in the file system part of the data copies in the multiple data copies are stored in the data nodes under the first AZ in the file system, and the file system also includes at least one second For data nodes under the AZ, the availability of the second AZ is lower than the availability of the first AZ;
  • the storage module 502 is configured to:
  • Acquire allocation indication information for the multiple partition copies where the allocation indication information is used to indicate the proportions of the number of copies stored in different AZs for the multiple partition copies;
  • the allocation indication information store the multiple partition copies to RS nodes under different AZs in the distributed database.
  • the allocation indication information is determined according to the load of the RS nodes under each AZ in the distributed database.
  • both the distributed database and the file system include multiple target AZs, and the storage module 502 is configured to:
  • the multiple data copies are stored in data nodes under different target AZs.
  • the apparatus 500 further includes:
  • the load balancing module 503 is configured to perform load balancing on RS nodes under different AZs in the distributed database, so as to adjust the number of partition copies stored in different AZs.
  • the data storage device 500 may correspond to the implementation of the method described in the embodiment of the present application, and the above-mentioned and other operations and/or functions of the various modules of the data storage device 500 are respectively in order to realize the method embodiment shown in FIG. 3
  • the corresponding process in for the sake of brevity, will not be repeated here.
  • Figure 6 provides a computing device.
  • the computing device 600 may be, for example, the device used to implement the functions of the data processing platform 100 in the foregoing embodiments, and the computer device 600 may specifically be used to implement the data storage device 500 in the embodiment shown in Figure 5 above function.
  • the computing device 600 includes a bus 601 , a processor 602 and a memory 603 .
  • the processor 602 and the memory 603 communicate through the bus 601 .
  • the bus 601 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus or the like.
  • PCI peripheral component interconnect
  • EISA extended industry standard architecture
  • the bus can be divided into address bus, data bus, control bus and so on. For ease of representation, only one thick line is used in FIG. 6 , but it does not mean that there is only one bus or one type of bus.
  • the processor 602 may be a central processing unit (central processing unit, CPU), a graphics processing unit (graphics processing unit, GPU), a microprocessor (micro processor, MP) or a digital signal processor (digital signal processor, DSP) etc. Any one or more of them.
  • CPU central processing unit
  • GPU graphics processing unit
  • MP microprocessor
  • DSP digital signal processor
  • the memory 603 may include a volatile memory (volatile memory), such as a random access memory (random access memory, RAM).
  • volatile memory such as a random access memory (random access memory, RAM).
  • Memory 603 can also include non-volatile memory (non-volatile memory), such as read-only memory (read-only memory, ROM), flash memory, mechanical hard disk (hard drive drive, HDD) or solid state hard disk (solid state drive) , SSD).
  • Executable program codes are stored in the memory 603 , and the processor 602 executes the executable program codes to implement the data storage method performed by the aforementioned data processing platform 100 .
  • the embodiment of the present application also provides a computer-readable storage medium.
  • the computer-readable storage medium may be any available medium that a computing device can store, or a data storage device such as a data center that includes one or more available media.
  • the available media may be magnetic media (eg, floppy disk, hard disk, magnetic tape), optical media (eg, DVD), or semiconductor media (eg, solid state hard disk), etc.
  • the computer-readable storage medium includes instructions for instructing a computing device to execute the above data recovery method.
  • the embodiment of the present application also provides a computer program product.
  • the computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on the computing device, the processes or functions according to the embodiments of the present application will be generated in whole or in part.
  • the computer instructions may be stored in or transmitted from one computer-readable storage medium to another computer-readable storage medium, e.g. (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wirelessly (such as infrared, wireless, microwave, etc.) to another website site, computer or data center.
  • another computer-readable storage medium e.g. (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wirelessly (such as infrared, wireless, microwave, etc.) to another website site, computer or data center.
  • the computer program product may be a software installation package which can be downloaded and executed on a computing device if any of the aforementioned object recognition methods are required.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Quality & Reliability (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

Selon des modes de réalisation, la présente demande concerne un procédé de stockage de données. Le procédé peut être appliqué à une plateforme de traitement de données comprenant une pluralité de zones de disponibilité (AZs), et la plateforme de traitement de données obtient une pluralité de copies de données à stocker, et stocke la pluralité de copies des données dans différentes zones de disponibilité de la plateforme de traitement de données. De cette manière, lorsque certaines des zones de disponibilité ne sont pas disponibles en raison de catastrophes naturelles ou d'autres catastrophes, la catastrophe ne touche généralement pas d'autres zones de disponibilité parce qu'une distance physique entre différentes zones de disponibilité est généralement importante, de telle sorte que la plateforme de traitement de données peut continuer à fournir un service de lecture-écriture de données pour un utilisateur sur la base des copies des données stockées dans d'autres zones de disponibilité fonctionnant normalement, ainsi, la condition selon laquelle la qualité du service de lecture-écriture de données fournie par la plateforme de traitement de données est réduite en raison de la survenue de la catastrophe est évitée, et la fiabilité de stockage de données de la plateforme de traitement de données est améliorée. De plus, selon les modes de réalisation, la présente demande concerne en outre un appareil correspondant et un dispositif associé.
PCT/CN2021/142795 2021-10-28 2021-12-30 Procédé et appareil de stockage de données et dispositif associé WO2023070935A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202180006938.2A CN116635831A (zh) 2021-10-28 2021-12-30 一种数据存储方法、装置及相关设备

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
IN202131049367 2021-10-28
IN202131049367 2021-10-28

Publications (1)

Publication Number Publication Date
WO2023070935A1 true WO2023070935A1 (fr) 2023-05-04

Family

ID=86160103

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/142795 WO2023070935A1 (fr) 2021-10-28 2021-12-30 Procédé et appareil de stockage de données et dispositif associé

Country Status (2)

Country Link
CN (1) CN116635831A (fr)
WO (1) WO2023070935A1 (fr)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117240873B (zh) * 2023-11-08 2024-03-29 阿里云计算有限公司 云存储系统、数据读写方法、设备及存储介质

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050223277A1 (en) * 2004-03-23 2005-10-06 Eacceleration Corporation Online storage system
CN104050102A (zh) * 2014-06-26 2014-09-17 北京思特奇信息技术股份有限公司 一种电信系统中的对象存储方法及装置
CN107943867A (zh) * 2017-11-10 2018-04-20 中国电子科技集团公司第三十二研究所 支持异构存储的高性能层次化存储系统
CN111104069A (zh) * 2019-12-20 2020-05-05 北京金山云网络技术有限公司 分布式存储系统的多区域数据处理方法、装置及电子设备
CN111782152A (zh) * 2020-07-03 2020-10-16 深圳市欢太科技有限公司 数据存储方法、数据恢复方法、装置、服务器及存储介质

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050223277A1 (en) * 2004-03-23 2005-10-06 Eacceleration Corporation Online storage system
CN104050102A (zh) * 2014-06-26 2014-09-17 北京思特奇信息技术股份有限公司 一种电信系统中的对象存储方法及装置
CN107943867A (zh) * 2017-11-10 2018-04-20 中国电子科技集团公司第三十二研究所 支持异构存储的高性能层次化存储系统
CN111104069A (zh) * 2019-12-20 2020-05-05 北京金山云网络技术有限公司 分布式存储系统的多区域数据处理方法、装置及电子设备
CN111782152A (zh) * 2020-07-03 2020-10-16 深圳市欢太科技有限公司 数据存储方法、数据恢复方法、装置、服务器及存储介质

Also Published As

Publication number Publication date
CN116635831A (zh) 2023-08-22

Similar Documents

Publication Publication Date Title
JP6791834B2 (ja) 記憶システム及び制御ソフトウェア配置方法
CN109729129B (zh) 存储集群系统的配置修改方法、存储集群及计算机系统
CN108664496B (zh) 数据迁移方法及装置
US7487390B2 (en) Backup system and backup method
KR101242458B1 (ko) 지능형 분산 스토리지 서비스 시스템 및 방법
US10838829B2 (en) Method and apparatus for loading data from a mirror server and a non-transitory computer readable storage medium
US8195777B2 (en) System and method for adding a standby computer into clustered computer system
US9201747B2 (en) Real time database system
US9875056B2 (en) Information processing system, control program, and control method
US9317381B2 (en) Storage system and data management method
US20140188957A1 (en) Hierarchical storage system and file management method
CN110825704B (zh) 一种读数据方法、写数据方法及服务器
WO2019148841A1 (fr) Système de stockage distribué, procédé de traitement de données et nœud de stockage
KR100936238B1 (ko) 파일 입출력과 복제의 균형적 수행을 위한 지연복제 시스템및 방법
US9984139B1 (en) Publish session framework for datastore operation records
US10445295B1 (en) Task-based framework for synchronization of event handling between nodes in an active/active data storage system
CN113282564B (zh) 数据存储方法、系统、节点和存储介质
US7849264B2 (en) Storage area management method for a storage system
CN114385561A (zh) 文件管理方法和装置及hdfs系统
WO2023070935A1 (fr) Procédé et appareil de stockage de données et dispositif associé
KR20170045928A (ko) 인메모리 데이터베이스를 이용한 데이터 관리 방법 및 그 장치
CN111225003B (zh) 一种nfs节点配置方法和装置
WO2022151593A1 (fr) Procédé, appareil et dispositif de récupération de données, support et produit de programme
CN111459416B (zh) 一种基于分布式存储的热迁移系统及其迁移方法
CN104468674B (zh) 数据迁移方法及装置

Legal Events

Date Code Title Description
WWE Wipo information: entry into national phase

Ref document number: 202180006938.2

Country of ref document: CN

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21962276

Country of ref document: EP

Kind code of ref document: A1