WO2024087914A1 - 数据同步方法及计算设备 - Google Patents

数据同步方法及计算设备 Download PDF

Info

Publication number
WO2024087914A1
WO2024087914A1 PCT/CN2023/118095 CN2023118095W WO2024087914A1 WO 2024087914 A1 WO2024087914 A1 WO 2024087914A1 CN 2023118095 W CN2023118095 W CN 2023118095W WO 2024087914 A1 WO2024087914 A1 WO 2024087914A1
Authority
WO
WIPO (PCT)
Prior art keywords
storage device
data
load
state
moment
Prior art date
Application number
PCT/CN2023/118095
Other languages
English (en)
French (fr)
Inventor
方贤才
陈琪
Original Assignee
超聚变数字技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 超聚变数字技术有限公司 filed Critical 超聚变数字技术有限公司
Publication of WO2024087914A1 publication Critical patent/WO2024087914A1/zh

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/27Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • H04L67/10Protocols in which an application is distributed across nodes in the network
    • H04L67/1095Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes

Definitions

  • the present disclosure relates to the field of computer technology, and in particular to a data synchronization method and a computing device.
  • the method for synchronizing data between the storage device at the production end and the storage device at the disaster recovery end usually adopts asynchronous remote replication technology. After the storage device at the production end and the storage device at the disaster recovery end establish an asynchronous remote replication relationship, the storage device at the production end can transmit data to the storage device at the disaster recovery end according to a preset period.
  • data replication may affect the production business of the storage device at the production end.
  • the embodiments of the present application provide a data synchronization method and a computing device, which can adaptively trigger data synchronization between a storage device at a production end and a storage device at a disaster recovery end, reduce the data difference between the storage device at the production end and the storage device at the disaster recovery end, and reduce the amount of data loss when a failure occurs.
  • the present application provides a data synchronization method, which is applied to a first storage device, and the method includes: when the load of the first storage device is less than or equal to a specified threshold, obtaining difference data; the difference data includes business data of input and output operations performed on the first storage device within a first time period, the first time period is the time period between the last time data synchronization to the second storage device started and the current time, the first storage device is a storage device at the business data production end; the second storage device is a storage device at the business data disaster recovery end; and synchronizing the difference data to the second storage device.
  • the method obtains the difference data between the last time the data synchronization to the second storage device was started and the current time when the first storage device determines that its own load is less than or equal to the specified threshold, and sends the obtained difference data to the second storage device at the current time, disaster recovery backup of business data is achieved.
  • the load of the first storage device itself is less than or equal to the specified threshold, indicating that the first storage device needs to execute less production business, and disaster recovery backup at this time has less impact on the storage performance of the production end (i.e., the first storage device).
  • difference data when the load of the first storage device is less than or equal to a specified threshold, difference data is obtained, including: querying an attribute state of a data synchronization flag, the data synchronization flag is stored in the first storage device and is used to characterize whether the current load of the first storage device is less than or equal to the specified threshold; when the queried attribute state is a first state, difference data is obtained, and the first state is used to indicate that the current load of the first storage device is less than or equal to the specified threshold.
  • the attribute state of the data synchronization flag is the first state
  • obtaining the difference data can ensure that the difference data is obtained when the current load of the first storage device is less than or equal to the specified threshold.
  • setting the data synchronization flag helps to realize that the "computation process of comparing the load of the first storage device with the specified threshold" and the "process of managing and executing data synchronization operations" belong to two threads, thereby improving the efficiency of determining whether to obtain difference data.
  • the method also includes: when the queried attribute state is the second state, querying the attribute state of the data synchronization flag at subsequent detection moments in sequence until the query of the attribute state is stopped when the first detection moment is reached, the detection moment is the moment when the first storage device detects whether the load of the first storage device is less than or equal to a specified threshold, the first detection moment is the detection moment when the first attribute state queried subsequently in chronological order is the first state; and obtaining difference data at the first detection moment.
  • the continuous execution of the data synchronization operation can be interrupted.
  • the data synchronization operation can be adjusted in time according to the load situation of the first storage device, so that the first storage device can adaptively adjust whether to synchronize data to the second storage device.
  • the method also includes: when the queried attribute state is the second state, and in the process of querying the attribute state of the data synchronization mark at subsequent detection moments, the attribute state is not queried to be the first state, and when the time interval between the current moment and the last synchronization start moment reaches a preset time period, obtaining difference data, and the synchronization start moment is the moment to start synchronizing data to the second storage device.
  • querying the attribute status of the data synchronization flag includes: at the first moment of receiving the synchronization completion response returned by the second storage device, querying the attribute status of the data synchronization flag, the synchronization completion response being used to indicate that the business data last synchronized from the first storage device to the second storage device has been synchronized.
  • acquiring difference data includes: when the queried attribute state at the first moment is the first state, acquiring difference data.
  • the first storage device when the first storage device receives the synchronization completion response returned by the second storage device, the first storage device can immediately query the attribute status of the data synchronization flag, so that a continuous data synchronization process can be implemented when the attribute status is the first state, and data synchronization can be continuously performed in an adaptive manner to reduce the amount of data loss when a failure occurs.
  • differential data when the load of the first storage device is less than or equal to a specified threshold, differential data is obtained, including: generating a first snapshot of the business data in the first storage device when it is determined that the load of the first storage device is less than or equal to the specified threshold; and obtaining differential data based on the first snapshot and a data transformation log between the last time data synchronization to the second storage device was started and the current time, where the data transformation log is a log recorded when the first storage device performs input and output operations.
  • the load is characterized by at least one of the following parameters: CPU occupancy of a processor of the first storage device, memory occupancy, and bandwidth of a service data input/output IO interface.
  • parameters that characterize the load of the first storage device including CPU occupancy, memory occupancy, and bandwidth of the business data input and output IO interface, are helpful in determining the time for data synchronization to the second storage device, balancing the timing of the first storage device executing production business and executing synchronization operations, and giving priority to ensuring that the impact of the synchronization operation on the load of the first storage device does not cause the first storage device to be unable to complete production business at a certain efficiency.
  • synchronizing the difference data to the second storage device includes: when the data amount of the difference data is greater than a specified data amount, synchronizing the difference data to the second storage device.
  • the load of the first storage device when it is determined that the load of the first storage device is less than or equal to the specified threshold, it can be determined whether to start data synchronization with the second storage device according to the size of the difference data. If the amount of the difference data is less than or equal to the specified amount of data, data synchronization with the second storage device will not be started until it is determined that the load of the first storage device is less than or equal to the specified threshold and the amount of the acquired difference data is greater than the specified amount of data, then data synchronization with the second storage device is started.
  • the above method can reduce the number of times the first storage device synchronizes data with the second storage device, thereby reducing system power consumption.
  • the present application provides a data synchronization device, which is used to execute any one of the data synchronization methods provided in the first aspect above.
  • the present application may divide the data synchronization device into functional modules according to the method provided in the first aspect above.
  • each functional module may be divided according to each function, or two or more functions may be inherited in one processing module.
  • the present application may divide the data synchronization device into a processing module and a synchronization module, etc. according to the function.
  • the description of the possible technical solutions and beneficial effects executed by each of the functional modules divided above can refer to the technical solutions provided in the first aspect above or its corresponding possible implementation, and will not be repeated here.
  • an embodiment of the present application provides a computing device, the computing device comprising a processor and a memory, the memory being used to store computer program instructions, the computer program instructions being loaded and executed by the processor so that the computing device implements the data synchronization method as described in the above aspects.
  • an embodiment of the present application provides a computer-readable storage medium, in which at least one computer program instruction is stored, and the computer program instruction is loaded and executed by a processor to implement the data synchronization method as described in the above aspects.
  • an embodiment of the present application provides a computer program product, the computer program product including computer instructions, the computer instructions stored in a computer-readable storage medium.
  • a processor of a computing device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computing device executes the data synchronization method provided in various optional implementations of the first aspect above.
  • FIG1 is a schematic diagram of a data synchronization system according to an exemplary embodiment
  • FIG2 is a schematic diagram of establishing a remote replication consistency group between logical volumes involved in the embodiment shown in FIG1 ;
  • FIG3 is a flow chart of a data synchronization method according to an exemplary embodiment
  • FIG4 is a schematic diagram of determining a data synchronization start time involved in the embodiment shown in FIG3 ;
  • FIG5 is a flow chart of a data synchronization method according to an exemplary embodiment
  • FIG6 is a schematic diagram of a flow chart of an adaptive detection module determining a data synchronization flag involved in the embodiment shown in FIG5 ;
  • FIG7 is a schematic diagram of a performance detection and data synchronization process involved in the embodiment shown in FIG5 ;
  • FIG8 is a schematic diagram of an adaptive continuous data replication process involved in the embodiment shown in FIG5 ;
  • FIG9 is a schematic diagram of the structure of a data synchronization device provided by an exemplary embodiment of the present application.
  • FIG. 10 is a schematic diagram of the structure of a computing device provided by an exemplary embodiment of the present application.
  • a and/or B can mean: A exists alone, A and B exist at the same time, and B exists alone.
  • the character "/" generally indicates that the related objects are in an "or” relationship.
  • the business recovery time of data disaster recovery solutions can be represented by the recovery time objective (RTO), which is used to indicate the shortest time period from the occurrence of a disaster to the recovery of service functions of the business system;
  • the data recovery volume of data disaster recovery solutions can be represented by the data recovery point objective (RPO), which is used to indicate the amount of data loss that the business system can tolerate.
  • RTO recovery time objective
  • RPO data recovery point objective
  • the related art does not consider whether the production side is busy when triggering the production side to synchronize data to the disaster recovery side, so it will cause the production side to execute synchronization operations in parallel with the business operations being executed on multiple production sides, thereby affecting the completion efficiency of the production business operations being executed.
  • the embodiment of the present application proposes a data synchronization method, which detects the load of the storage device on the production side before performing data synchronization, that is, if it is detected that the load of the storage device on the production side is less than or equal to the specified threshold at the current time, subsequent data synchronization operations can be performed, which balances the relationship between the production side performing production business and executing synchronization operations, and uses the business load index of the production side to realize adaptive adjustment of the synchronization time, thereby reducing the impact of data synchronization on the load of the storage device on the production side.
  • FIG. 1 is a schematic diagram of a data synchronization system involved in an embodiment of the present application, wherein the data synchronization system includes a production data center 100 and a disaster recovery data center 200.
  • the production data center 100 may include a production host and a first storage device 110.
  • the first storage device 110 includes an IO interface 101, a processor 102, a buffer memory, and a first memory 103.
  • the disaster recovery data center 200 may include a disaster recovery host and a second storage device 210.
  • the second storage device 210 may include a buffer memory and a second memory 201.
  • the production host may be a server, and the first storage device 110 is a storage device for storing business data at the production end.
  • the first storage device 110 may be a computing device
  • the second storage device 210 is a storage device for storing replicated business data at the disaster recovery end, and the second storage device 210 may also be a computing device.
  • the computing device may be a smart phone, a tablet computer, an e-book reader, a personal portable computer, a server, etc.
  • the IO interface 101 can be used to implement communication between the production host and the first storage device 110, wherein the production host can send IO instructions to the first storage device 110 through the IO interface 101, and the IO instructions are used to indicate the input and output operations of business data.
  • the processor 102 in the first storage device 110 can be used to process production business, and can also be used to execute the data synchronization method of this solution.
  • the buffer memory can be used to temporarily store operation instructions and business data.
  • the first memory 103 may be a disk array, which may be divided into a plurality of master logical volumes according to a logical unit number (LUN), including master LUN001 to master LUN n; similarly, the second storage 201 may also be a disk array, and the disk array may be divided into a plurality of slave logical volumes according to LUN, including slave LUN001 to slave LUN n.
  • a remote replication consistency group (replication group) may be established between the master logical volume and the slave logical volume, and since the master and slave logical volumes are in one-to-one correspondence, the number of master and slave logical volumes may be the same, and each master logical volume corresponds to one slave logical volume.
  • FIG2 is a schematic diagram of establishing a remote replication consistency group between logical volumes involved in an embodiment of the present application.
  • the master logical volume in the first storage 103 can be divided into master LUN001, master LUN002, and master LUN003 according to LUN
  • the slave logical volume included in the second storage 201 corresponding to the master logical volume can be divided into slave LUN001, slave LUN002, and slave LUN003 according to LUN.
  • a remote replication consistency relationship between master LUN001 and master LUN002 and slave LUN001 and slave LUN002 can be established, and master LUN001 and master LUN002 and slave LUN001 and slave LUN002 are used as a remote replication consistency group 1, and remote replication task 1 is performed through the remote replication consistency group 1.
  • master LUN001 and master LUN002 and slave LUN001 and slave LUN002 are used as a remote replication consistency group 1, and remote replication task 1 is performed through the remote replication consistency group 1.
  • the business data that can be stored in the master LUN003 is a single data type
  • a remote replication consistency relationship can be established between the master LUN003 and the slave LUN003, and the master LUN003 and the slave LUN003 can be used as a remote replication consistency group 2 to execute remote replication task 2 through the remote replication consistency group 2.
  • the production host and the disaster recovery host can be connected via an Ethernet protocol (IP) or a fiber channel (FC).
  • IP Ethernet protocol
  • FC fiber channel
  • the first storage device 110 and the second storage device 210 can also be connected via IP or FC. And through the above communication connection, the first storage device 110 can copy the stored business data to the second storage device 210, and when the first storage device 110 fails, the first storage device 110 can receive the business data returned by the second storage device 210 to recover the data.
  • IP Ethernet protocol
  • FC fiber channel
  • FIG3 shows a flow chart of a data synchronization method provided by an exemplary embodiment of the present application.
  • the data synchronization method may be executed by a first storage device, for example, the first storage device may be the first storage device 110 in the data synchronization system shown in FIG1 .
  • the data synchronization method includes the following steps:
  • S301 acquiring difference data when the load of the first storage device is less than or equal to a specified threshold.
  • the first storage device may determine whether its own load is less than or equal to a specified threshold.
  • the first storage device can determine whether its own load at the current moment is less than or equal to a specified threshold. If it is determined that the load of the first storage device at the current moment is less than or equal to the specified threshold, the difference data can be obtained.
  • the first storage device first needs to perform initial data synchronization with the second storage device. After the first storage device and the second storage device achieve initial data synchronization, the first storage device can begin to determine whether the load of the first storage device at the current moment is less than or equal to the specified threshold. If it is determined that the load of the first storage device is less than or equal to the specified threshold, the difference data at the current moment can be obtained.
  • the difference data can be business data including input and output operations performed on the first storage device within a first time period.
  • the first time period is the time period between the last time data synchronization to the second storage device started to the current moment.
  • the first storage device can use the moment when it determines that its load is less than or equal to the specified threshold as the synchronization start time. That is, the synchronization start time is the moment when the data synchronization operation is started to be performed to the second storage device, or it can be the moment when the differential data that needs to be sent to the second storage device is started.
  • the specified threshold in the embodiment of the present application can be preset by the user based on the needs.
  • the value of the specified threshold is positively correlated with the priority of synchronizing data to the second storage unit. That is to say, if it is necessary to synchronize data to the second storage device as soon as possible, the value of the specified threshold can be appropriately increased to ensure that the first storage device synchronizes data to the second storage device. If the efficiency of the production end in executing production business needs to be guaranteed, the value of the specified threshold can be appropriately reduced to ensure that when the first storage device synchronizes data with the second storage device, the second storage device will not affect the production business it is executing.
  • the specified threshold can be dynamically updated and adjusted in real time according to the business load of the production end, which is not limited here.
  • FIG4 is a schematic diagram of determining a data synchronization start time involved in an embodiment of the present application.
  • the first storage device completes the initial synchronization process with the second storage device, that is, from the initial moment to the end of moment 1, the process of synchronizing the initial data with the second storage device is completed, and then at moment 1, it is determined whether the load of the first storage device at moment 1 is less than or equal to the specified threshold. If the load of the first storage device is less than or equal to the specified threshold, moment 1 can be used as the start time of this data synchronization, and the first data synchronization process to the second storage device can be started.
  • the difference data acquired at moment 1 that needs to be transmitted to the second storage device can be the business data for performing input and output operations on the first storage device between the initial moment and moment 1, that is, the difference data that needs to be sent to the second storage device during the first synchronization process includes the difference data between the business data in the first storage device at moment 1 and the business data in the first storage device at the initial moment.
  • the first synchronization time of the first data synchronization process performed by the first storage device to the second storage device is from moment 1 to moment 2, it can be determined at moment 2 whether the load of the first storage device at moment 2 is less than or equal to the specified threshold.
  • a first snapshot is generated for the business data in the first storage device; based on the first snapshot and a data transformation log between the last time data synchronization to the second storage device was started and the current time, differential data is obtained, where the data transformation log is a log recorded when the first storage device performs input and output operations.
  • the first storage device when the first storage device determines that its own load at the current moment is greater than a specified threshold, it does not obtain difference data at the current moment and sends difference data to the second storage device, and determines in chronological order whether the load at subsequent moments is less than or equal to the specified threshold until it is determined that the load of the first storage device at the current moment is less than or equal to the specified threshold, and obtains difference data.
  • the subsequent determination of whether the load of the first storage device is less than or equal to the specified threshold may be made in real time, or may be made according to a certain period, and when the current moment reaches the moment indicated by the period, it is determined whether the load of the first storage device is less than or equal to the specified threshold.
  • the first storage device can continue to determine whether the load of the first storage device is less than or equal to the specified threshold at subsequent times. If the first storage device does not determine that its load is less than or equal to the specified threshold until the current time reaches time 3, time 3 is determined as the data synchronization start time for performing the second data synchronization with the second storage device, and the second data synchronization process with the second storage device starts at time 3.
  • the difference data acquired at time 3 that needs to be transmitted to the second storage device this time can be the business data for performing input and output operations on the first storage device between time 1 of the last synchronization start and the current time 3, that is, the difference data that needs to be sent to the second storage device during the second synchronization process includes the difference data between the business data in the first storage device at time 3 and the business data in the first storage device at time 1.
  • the second synchronization time of the second data synchronization process performed by the first storage device to the second storage device is from time 3 to time 4, it can continue to determine at time 4 whether the load of the first storage device at time 4 is less than or equal to the specified threshold.
  • difference data is obtained when the first storage device determines that its load is greater than a specified threshold at the current moment, and the time interval between the current moment and the last synchronization start moment reaches a preset time period.
  • a preset time period can be set in advance. Even if the first storage device determines that its load does not meet the condition of being less than or equal to the specified threshold at the current moment, it can obtain differential data when the preset time period is reached, which can stabilize the upper limit of RPO to a certain extent.
  • the first storage device can continue to determine whether the load of the first storage device itself is less than or equal to the specified threshold at subsequent times. If the first storage device still has not determined that its own load is less than or equal to the specified threshold until the current time reaches time 3, and the time interval between time 3 and time 1 reaches the preset time period, the second data synchronization process to the second storage device can be started at time 3.
  • the difference data acquired at time 3 that needs to be transmitted to the second storage device this time can be the business data for performing input and output operations on the first storage device between time 1 when the last synchronization was started and the current time 3.
  • the difference data that needs to be sent to the second storage device during the second synchronization process includes the difference data between the business data in the first storage device at time 3 and the business data in the first storage device at time 1.
  • the second synchronization time of the second data synchronization process performed by the first storage device to the second storage device is from time 3 to time 4, it can be determined at time 4 whether the load of the first storage device at time 4 is less than or equal to the specified threshold.
  • S304 The first storage device synchronizes the difference data to the second storage device.
  • the first storage device after the first storage device obtains the difference data, it copies the difference data and sends it to the second storage device, so as to synchronize the business data in the first storage device to the second storage device.
  • the synchronization of the differential data by the first storage device is a process for indicating copying the differential data to the second storage device.
  • the differential data is transmitted according to the primary logical volume in the first storage device to which the differential data belongs, through a remote replication consistency group established between the primary logical volume storing the differential data and the slave logical volume corresponding to the primary logical volume in the second storage device, so that the differential data is backed up in the slave logical volume corresponding to the primary logical volume in the second storage device, thereby storing disaster-tolerant business data in the second storage device.
  • the first storage device may synchronize the difference data to the second storage device.
  • the first storage device can determine whether the size of the difference data is greater than a specified data amount, and when it is determined that the size of the difference data is greater than the specified data amount, the difference data can be synchronized to the second storage device.
  • the specified data amount can be preset according to the RPO requirement.
  • the synchronization of the business data to the second storage device takes 10 minutes to complete.
  • the primary logical volume of the first storage device changes by 100MB.
  • the first storage device can immediately start to synchronize the 100MB difference data; if the synchronization process takes 5 minutes, and the primary logical volume of the first storage device changes by 50MB during the 5 minutes, since it is determined that the load of the first storage device is less than or equal to the specified threshold and 50MB is greater than the specified data volume of 20MB, the first storage device continues to synchronize the 50MB data; if the synchronization process takes 2 minutes, and another 20MB change occurs during the period; since 20MB does not exceed the specified data volume, the data synchronization will continue at the next moment when the load of the first storage device is less than or equal to the specified threshold and the difference data is greater than 20MB, thereby reducing the number of data synchronizations and reducing the RPO of the system.
  • the power consumption of the system if it is determined that the load of the first storage device is less than or equal to the specified threshold and 100MB is greater than the specified data volume of 20MB, the first storage device can immediately start to synchronize the 100MB difference data; if the
  • the second storage device returns a synchronization completion response to the first storage device.
  • the second storage device after the second storage device completely receives the differential data sent by the first storage device and stores it in its own logical volume, it can determine that the process of synchronizing data from the first storage device to the second storage device is completed, and then return a synchronization completion response to the first storage device.
  • the synchronization completion response is used to indicate that the business data synchronized to the second storage device last time has been synchronized.
  • the first storage device after the first storage device receives the synchronization completion response, it can start a new round of determining whether its own load is less than or equal to a specified threshold, and re-execute steps S301 to S303 to start the next round of data synchronization from the first storage device to the second storage device.
  • the first storage device can determine whether the current load is less than a specified threshold when it determines that its own load condition needs to be determined through the data replication management module in the processor. That is to say, during the normal operation of the business system of the first storage device, the first storage device can monitor and record its own load condition in real time, and after receiving the synchronization completion response returned by the second storage device, it can start to obtain its own load condition at the current moment and compare it with the specified threshold, so that the first storage device can determine whether its own load condition can be synchronized with the second storage device without affecting the production business before starting the next data synchronization with the second storage device.
  • the processor in the first storage device may include an adaptive detection module in addition to the data replication management module.
  • the adaptive detection module may be used to detect whether the load condition of the first storage device at the current moment is less than or equal to a specified threshold, and to modify the attribute state of the data synchronization flag in real time for the data replication management module to query.
  • the data replication management module determines whether to start synchronizing business data to the second storage device according to the attribute state of the queried data synchronization flag.
  • FIG5 shows a flow chart of a data synchronization method provided by an exemplary embodiment of the present application.
  • the data synchronization method may be executed by the first storage device.
  • the data synchronization method specifically includes the following steps:
  • the first storage device determines the attribute state of the data synchronization flag at the current moment.
  • the first storage device may determine the attribute state of the data synchronization flag stored in the first storage device at the current moment.
  • the data synchronization flag may be stored in the first storage device, and is used to indicate whether the current load of the first storage device is less than or equal to a specified threshold.
  • the attribute state of the data synchronization flag when it is determined that the current load of the first storage device is less than or equal to a specified threshold, the attribute state of the data synchronization flag is determined to be a first state; when it is determined that the current load of the first storage device is greater than the specified threshold, the attribute state of the data synchronization flag is determined to be a second state.
  • the first state is used to indicate that the current load of the first storage device is less than or equal to a specified threshold
  • the second state is used to indicate that the current load of the first storage device is greater than a specified threshold
  • the data synchronization flag stored in the first storage device can be represented as STOP_REPLICATION.
  • the attribute state of the data synchronization flag STOP_REPLICATION can be marked as FALSE, that is, the first state of the data synchronization flag STOP_REPLICATION can be marked as FALSE. If it is determined that the current load is greater than the specified threshold, the attribute state of the data synchronization flag STOP_REPLICATION can be marked as TRUE, that is, the second state of the data synchronization flag STOP_REPLICATION can be marked as TRUE.
  • the load may be characterized by at least one of the following parameters: CPU occupancy of the processor of the first storage device, memory occupancy, and bandwidth of a service data input/output IO interface.
  • the load of the first storage device can be represented by the CPU occupancy of the first storage device
  • the CPU occupancy The situation may be the average value of the CPU usages of the current moment and the three most recent records before the current moment.
  • the memory occupancy may be the memory occupancy rate detected at the current moment.
  • the bandwidth of the IO interface may be the bandwidth of the IO interface detected at the current moment.
  • the first storage device when the load of the first storage device is required to be characterized by at least two of the above parameters, it can be determined that when each parameter characterizing the load is less than its corresponding specified threshold, the first storage device can determine that the attribute state of the data synchronization flag at the current moment is the first state.
  • the specified thresholds corresponding to different parameters representing the load may be the same or different.
  • the specified threshold corresponding to the average value of the CPU occupancy rate may be preset as a specified percentage.
  • the specified threshold corresponding to the memory occupancy rate may also be a specified percentage.
  • the specified threshold corresponding to the bandwidth of the IO interface may be a specified percentage of the historical bandwidth peak value. For example, the specified percentage may be 80%.
  • other performance of the first storage device may also affect the determination of the attribute state of the data synchronization flag at the current moment. Therefore, for the parameters characterizing the performance in the first storage device, when the load of the first storage device is less than or equal to a specified threshold and the performance of the first storage device meets the preset indicators, it can be determined that the attribute state of the data synchronization flag at the current moment is the first state. However, when the load of the first storage device is less than or equal to a specified threshold and the performance of the first storage device does not meet the preset indicators, it can be determined that the attribute state of the data synchronization flag at the current moment is the second state.
  • FIG6 is a flowchart of an adaptive detection module involved in an embodiment of the present application for determining a data synchronization flag.
  • the first storage device can check whether the CPU load of the first storage device at the current moment is greater than the threshold corresponding to the CPU load (S11). If it is checked that the CPU load of the first storage device at the current moment is less than or equal to the threshold corresponding to the CPU load, then continue to check whether the memory load of the first storage device is greater than the threshold corresponding to the memory load (S12).
  • the memory load of the first storage device at the current moment is less than or equal to the threshold corresponding to the memory load, then continue to check whether the business bandwidth load of the first storage device is greater than the threshold corresponding to the business bandwidth load (S13). If it is checked that the business bandwidth load of the first storage device at the current moment is equal to the threshold corresponding to the business bandwidth load, then check whether the first storage device has a greater business bandwidth load than the threshold corresponding to the business bandwidth load.
  • the first storage device sets the data synchronization flag STOP_REPLICATION to FALSE, and starts the process of adaptive data synchronization to the second storage device (S15), then sleeps for a time interval, and cyclically performs adaptive performance detection (S16); if at least one of the parameters characterizing the load is determined to be greater than the corresponding threshold value, the data synchronization flag STOP_REPLICATION of the first storage device is set to TRUE, and cyclic adaptive performance detection is started again (S17).
  • S32 The first storage device queries the attribute status of the data synchronization flag at the current moment.
  • the first storage device since the first storage device can determine the attribute status of the data synchronization flag after detecting the relationship between the load and the specified threshold at the current moment, the first storage device only needs to query the attribute status of the data synchronization flag at the current moment to determine whether the first storage device supports synchronizing data to the second storage device at the current moment.
  • the first storage device may query whether the data synchronization flag STOP_REPLICATION is TRUE or FALSE at the current moment.
  • the first storage device can query the attribute state of the data synchronization flag at the current moment after the last time the data synchronization to the second storage device is completed at the current moment.
  • the attribute state of the log is the first state, start to obtain difference data between the business data at the current moment and the business data when the data was last synchronized to the second storage device.
  • the first storage device may query the data synchronization flag of the first storage device at the current moment. If the attribute state of the data synchronization flag of the first storage device at the current moment is the first state, the difference data at the current moment may be obtained.
  • the data synchronization flag may be stored in the first storage device, and is used to indicate whether the current load of the first storage device is less than or equal to a specified threshold.
  • the first storage device queries the attribute status of the data synchronization flag at the first moment of receiving the synchronization completion response returned by the second storage device.
  • the synchronization completion response is used to indicate that the business data that the first storage device last synchronized with the second storage device has been synchronized. That is, after the data synchronization with the second storage device is completed last time, that is, when all the business data that needs to be sent to the second storage device is sent and successfully written to the corresponding slave logical volume in the second storage device, a synchronization completion response indicating that the data synchronization is completed can be returned to the first storage device, and the first moment can be either the completion moment of the last synchronization of data from the first storage device to the second storage device, or the starting moment of the current data synchronization from the first storage device to the second storage device when the attribute state of the data synchronization flag is queried to be the first state.
  • the first state can be used to indicate that the current load of the first storage device is less than or equal to the specified threshold.
  • difference data is acquired.
  • FIG7 is a schematic diagram of a performance detection and data synchronization process involved in an embodiment of the present application.
  • the performance detection process and the data synchronization process are two parallel threads in time.
  • the performance detection process can detect whether the load of the first storage device at the current moment is less than or equal to the specified threshold according to a preset period, and set the attribute state of the data synchronization flag at the current moment to the first state when it is determined that the load of the first storage device is less than or equal to the specified threshold, and set the attribute state of the data synchronization flag at the current moment to the second state when it is determined that the load of the first storage device is greater than the specified threshold.
  • the attribute state of the data synchronization flag is the first state at time 1
  • the first data synchronization to the second storage device is started, and the first data synchronization to the second storage device is ended when the current moment reaches time 2, and the attribute state of the current data synchronization flag is queried when the current moment reaches time 2. Since the attribute state of the current data synchronization flag is queried at the current moment, that is, time 2, as the first state, the differential data to be copied and synchronized to the second storage device can be obtained.
  • the first storage device when the first storage device finds that the attribute state of the data synchronization flag at the current moment is the second state, the first storage device continues to query the attribute state of the data synchronization flag at the next moment in chronological order until the first storage device finds that the attribute state of the data synchronization flag is changed to the first state, and then starts to obtain differential data.
  • the first storage device when the first storage device finds that the attribute state of the data synchronization flag at the current moment is the second state, the first storage device continues to query the attribute state of the data synchronization flag at each subsequent moment in chronological order until it finds that the attribute state of the data synchronization flag of the first storage device changes from the second state to the first state, and starts to obtain difference data.
  • the first storage device may query the attribute status of the data synchronization flag according to a preset period.
  • the period for updating the data synchronization flag of the first storage device can be a preset period, that is, only when the attribute state of the data synchronization flag is updated according to the preset period, it is necessary to query whether the attribute state of the data synchronization flag is changed from the second state to the first state.
  • the first storage device sequentially queries the attribute state of the data synchronization mark at subsequent detection moments until the first detection moment is reached and stops querying the attribute state to obtain differential data at the first detection moment.
  • the detection time may be the time when the first storage device detects whether the load of the first storage device is less than or equal to a specified threshold, and the first detection time may be the detection time when the first attribute state subsequently queried in chronological order is the first state.
  • the second data synchronization of the first storage device to the second storage device is completed, and then the attribute state of the current data synchronization mark can be queried at moment 3. Since the attribute state queried at moment 3 is the second state, the continuous replication process can be paused, and it is determined whether the current attribute state has changed when the current moment reaches the next inspection moment. Since it is determined that the attribute state of the data synchronization mark at the detection moment is still not the second state, the attribute state of the data synchronization mark at the next detection moment is queried when the current moment reaches the next detection moment. If the next detection moment is moment 4, since the queried data synchronization mark changes to the first state when the current moment reaches moment 4, moment 4 is used as the starting time for the third data synchronization with the second storage device, and the acquisition of differential data begins.
  • the synchronization start time is the time when data synchronization with the second storage device starts.
  • the preset time period may be a time interval preset in the asynchronous remote replication technology.
  • the attribute state of the data synchronization flag can be queried. Since the attribute state of the data synchronization flag at the moment when the data synchronization is completed is the second state, the attribute state of the data synchronization flag is queried when the current moment reaches the subsequent detection moment. Since the attribute state of the data synchronization flag at each detection moment until time 6 is the second state, and the time interval between time 5 and time 6 reaches the preset time period, time 6 can be used as the data start time to obtain difference data.
  • the first storage device synchronizes the difference data to the second storage device, and when the synchronization of the difference data is completed, the second storage device returns a synchronization completion response to the first storage device.
  • FIG8 is a schematic diagram of an adaptive continuous data replication process involved in an embodiment of the present application.
  • the first storage device starts adaptive data replication when receiving the synchronization completion response returned when the last data synchronization to the second storage device is completed, and queries to determine whether the data synchronization flag STOP_REPLICATION is FALSE.
  • the query data synchronization flag STOP_REPLICATION is FALSE, it means that the continuous replication is not stopped, that is, the current data synchronization process can continue with the previous data synchronization process; if the query data synchronization flag STOP_REPLICATION is TRU, it means that the continuous replication is stopped, that is, there is an interval between the previous data synchronization process and the current data synchronization process (S21). If the query data synchronization flag STOP_REPLICATION is TRUE, it enters sleep mode, and the sleep time interval can be modified. The sleep time interval is configurable by default of 1s.
  • the sleep time interval can be the time interval between detection moments (S22), and it is determined whether the interval between the current moment and the last synchronization start moment is greater than the preset time period. (S23), if it is greater than the preset time period, execute S24, if it is less than or equal to the preset time period, execute S21.
  • the first storage device can record the synchronization start time RG_REP_START_TIME (S24), the first storage device clears the original snapshot RGNO_BEFORE_REP_SNAP, and regenerates the snapshot RGNO_BEFORE_REP_SNAP, where RGNO can be used to represent the identity of the remote replication consistency group (S25).
  • the difference data of the current snapshot is obtained, and the difference data is copied to the second storage device at the disaster recovery end (S26), and the second storage device can return a response that the difference data is successfully written (S27).
  • the synchronization of the business data to the second storage device takes 10 minutes to complete. During these 10 minutes, the main logical volume of the first storage device changes by 100MB.
  • the first storage device can immediately start to synchronize the 100MB difference data; if this synchronization process takes 5 minutes, and during these 5 minutes, the main logical volume of the first storage device changes by 50MB, and then the first storage device continues to synchronize the 50MB data; if this synchronization process takes 2 minutes, during which time 20MB of changes occur; if it takes 1 minute to synchronize the 20MB, it can continue to synchronize until convergence.
  • the interval between synchronization processes can be made shorter and shorter, and the data transmitted each time can be smaller, thereby maximizing the system's RPO.
  • the data synchronization device includes at least one of the hardware structure and software modules corresponding to the execution of each function.
  • the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of this application.
  • the embodiment of the present application can divide the data recovery device into functional units according to the above method example.
  • each functional unit can be divided according to each function, or two or more functions can be integrated into one processing unit.
  • the above integrated unit can be implemented in the form of hardware or in the form of software functional units. It should be noted that the division of units in the embodiment of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation.
  • FIG. 9 shows a schematic diagram of the structure of a data synchronization device 400 provided by an exemplary embodiment of the present application.
  • the data synchronization device 400 is applied to a first storage device, and the data synchronization device 400 includes:
  • the processing module 410 is used to obtain difference data when the load of the first storage device is less than or equal to a specified threshold; the difference data includes business data of input and output operations performed on the first storage device in a first time period, the first time period is a time period from the last time when data synchronization to the second storage device was started to the current time, the first storage device is a storage device at the business data production end; the second storage device is a storage device at the business data disaster recovery end;
  • the synchronization module 420 is used to synchronize the difference data to the second storage device.
  • the processing module 410 may be used to execute S301 as shown in FIG. 3
  • the synchronization module 420 may be used to execute S304 as shown in FIG. 3 .
  • the processing module 410 further includes:
  • a query submodule used for querying the attribute status of a data synchronization flag, the data synchronization flag being stored in the first storage device and being used for indicating whether the current load of the first storage device is less than or equal to the specified threshold;
  • the acquisition submodule is used to acquire difference data when the queried attribute state is the first state.
  • the first state is used to indicate that the current load of the first storage device is less than or equal to the specified threshold.
  • the processing module 410 may be configured to execute S31 as shown in FIG. 5 .
  • the processing module 410 further includes:
  • the queried attribute state is the second state
  • querying the attribute state of the data synchronization flag at subsequent detection moments in sequence until stopping querying the attribute state when a first detection moment is reached the detection moment being the moment when the first storage device detects whether the load of the first storage device is less than or equal to the specified threshold
  • the first detection moment being the first detection moment at which the attribute state is the first state in the subsequent query in chronological order
  • the second state is used to indicate that the current load of the first storage device is greater than the specified threshold
  • the difference data is acquired at the first detection moment.
  • the processing module 410 further includes:
  • the attribute state queried is the second state, and in the process of querying the attribute state of the data synchronization flag at subsequent detection moments, the attribute state is not queried to be the first state, and when the time interval between the current moment and the last synchronization start moment reaches a preset time period, the difference data is obtained, and the synchronization start moment is the moment to start synchronizing data to the second storage device.
  • the query submodule includes:
  • the attribute status of the data synchronization flag is queried, and the synchronization completion response is used to indicate that the business data synchronized from the first storage device to the second storage device last time has been synchronized.
  • the processing module 410 further includes:
  • the processing module 410 further includes:
  • the difference data is obtained based on the first snapshot and a data transformation log between the last time data synchronization to the second storage device was started and the current time, wherein the data transformation log is a log recorded when the first storage device performs input and output operations.
  • the load is characterized by at least one of the following parameters: CPU occupancy, memory occupancy, and bandwidth of a service data input/output IO interface of the first storage device.
  • the processing module 410 further includes:
  • the difference data is obtained when the at least two parameters do not exceed the respective specified thresholds of the at least two parameters.
  • the synchronization module 420 includes:
  • the difference data is synchronized to the second storage device.
  • some or all of the functions implemented in the processing module 410 and the synchronization module 420 in the data synchronization apparatus may be implemented by the processor 102 in FIG. 1 executing program codes in the first storage device 110 in FIG. 1 .
  • Fig. 10 shows a schematic diagram of the structure of a computing device 1100 provided by an exemplary embodiment of the present application.
  • the computing device 1100 may be an electronic device such as a smart phone, a tablet computer, an e-book, a portable personal computer, a smart wearable device, etc.
  • the computing device 1100 in the present application may include one or more of the following components: a processor 1110 and a memory 1120 .
  • the processor 1110 may include one or more processing cores.
  • the processor 1110 uses various interfaces and lines to connect various parts in the entire terminal, and executes various functions of the terminal and processes data by running or executing instructions, programs, code sets or instruction sets stored in the memory 1120, and calling data stored in the memory 1120.
  • the processor 1110 can be implemented in at least one hardware form of digital signal processing (DSP), field-programmable gate array (FPGA), and programmable logic array (PLA).
  • DSP digital signal processing
  • FPGA field-programmable gate array
  • PDA programmable logic array
  • the processor 1110 can integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), and a modem.
  • the CPU mainly processes the operating system, user interface, and application programs; the modem is used to process wireless communications. It is understandable that the above-mentioned modem may not be integrated into the processor 1110, but may be implemented separately through a communication chip.
  • the memory 1120 may include a random access memory (RAM) or a read-only memory (ROM).
  • the memory 1120 includes a non-transitory computer-readable storage medium.
  • the memory 1120 may be used to store instructions, programs, codes, code sets or instruction sets.
  • the memory 1120 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above-mentioned various method embodiments, etc.
  • the operating system may be an Android system (including a system deeply developed based on the Android system), an IOS system developed by Apple (including a system deeply developed based on the IOS system), or other systems.
  • the data storage area may also store data created by the terminal during use (such as a phone book, audio and video data, chat record data), etc.
  • the structure of the computing device 1100 shown in the above figures does not constitute a limitation on the computing device 1100, and the computing device 1100 may include more or fewer components than shown, or combine certain components, or arrange components differently.
  • the terminal also includes radio frequency circuits, shooting components, sensors, audio circuits, wireless fidelity (WiFi) components, power supplies, Bluetooth components, and other components, which will not be described in detail here.
  • WiFi wireless fidelity
  • the embodiment of the present application also provides a computer-readable storage medium, in which at least one computer instruction is stored, and the at least one computer instruction is loaded and executed by a processor to implement the data synchronization method described in each of the above embodiments.
  • a computer-readable storage medium in which at least one computer instruction is stored, and the at least one computer instruction is loaded and executed by a processor to implement the data synchronization method described in each of the above embodiments.
  • the embodiment of the present application also provides a chip.
  • the chip integrates a control circuit and one or more ports for realizing the functions of the above-mentioned data synchronization device.
  • the functions supported by the chip can be referred to above and will not be repeated here.
  • a person of ordinary skill in the art can understand that all or part of the steps of implementing the above-mentioned embodiment can be completed by instructing the relevant hardware through a program.
  • the program can be stored in a computer-readable storage medium.
  • the storage medium mentioned above can be a read-only memory, a random access memory, etc.
  • the above-mentioned processing unit or processor can be a central processing unit, a general-purpose processor, an application specific integrated circuit (ASIC), a microprocessor (digital signal processor, DSP), a field programmable gate array (field programmable gate array, FPGA) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof.
  • ASIC application specific integrated circuit
  • DSP digital signal processor
  • FPGA field programmable gate array
  • the embodiments of the present application also provide a computer program product including instructions, which, when executed on a computer, enables the computer to perform any of the methods in the above embodiments.
  • the computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function according to the embodiments of the present application is generated in whole or in part.
  • the computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device.
  • Computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, computer instructions may be transmitted from a website, a computer, a server, or a data center.
  • Computer-readable storage media can be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that includes one or more available media integrated therein. Available media can be magnetic media (e.g., floppy disks, hard disks, tapes), optical media (e.g., DVDs), or semiconductor media (e.g., SSDs), etc.
  • the above-mentioned device for storing computer instructions or computer programs provided in the embodiments of the present application such as but not limited to, the above-mentioned memory, computer-readable storage medium and communication chip, etc., all have non-volatile (non-transitory).
  • the functions described in the embodiments of the present application can be implemented with hardware, software, firmware or any combination thereof. When implemented using software, these functions can be stored in a computer-readable storage medium or transmitted as one or more instructions or codes on a computer-readable storage medium.
  • Computer-readable storage media include computer storage media and communication media, wherein the communication medium includes any medium that is convenient for transmitting a computer program from one place to another.
  • the storage medium can be any available medium that a general or special-purpose computer can access.

Landscapes

  • Engineering & Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Computing Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

本申请实施例公开了一种数据同步方法及计算设备,属于计算机技术领域。可以自适应触发生产端的存储设备与容灾端的存储设备的数据同步,减少生产端的存储设备与容灾端的存储设备的数据差异,降低故障发生时的数据丢失量。该方法包括:在第一存储设备的负载小于或等于指定阈值的情况下,获取差异数据;差异数据包括第一时间段内,针对第一存储设备执行输入输出操作的业务数据,第一时间段是上一次开始向第二存储设备同步数据的时刻至当前时刻之间的时间段,第一存储设备是业务数据生产端的存储设备;第二存储设备是业务数据容灾端的存储设备;将差异数据同步到第二存储设备中。

Description

数据同步方法及计算设备
本申请要求于2022年10月24日提交国家知识产权局、申请号为202211302779.3、申请名称为“数据同步方法及计算设备”的中国专利申请的优先权,其全部内容通过引用结合在本申请中。
技术领域
本公开涉及计算机技术领域,特别涉及数据同步方法及计算设备。
背景技术
随着数字化技术的发展,数据保护以及保证业务连续性对企业越来越重要,关键数据的丢失会对企业造成重大的损失,所以针对关键数据的保护,通常需要指定对应的存储数据的容灾方案。
当前,生产端的存储设备向容灾端的存储设备进行数据同步的方法通常采用异步远程复制技术,在生产端的存储设备与容灾端的存储设备建立异步远程复制关系后,生产端的存储设备可以按照预设的周期定时向容灾端的存储设备传输数据。
上述相关技术中,由于生产端的存储设备需要严格按照预设的周期来触发数据复制同步,数据复制可能会影响生产端的存储设备进行生产业务。
发明内容
本申请实施例提供了一种数据同步方法及计算设备,可以自适应触发生产端的存储设备与容灾端的存储设备的数据同步,减少生产端的存储设备与容灾端的存储设备的数据差异,降低故障发生时的数据丢失量。
第一方面,本申请提供了一种数据同步方法,应用于第一存储设备,该方法包括:在第一存储设备的负载小于或等于指定阈值的情况下,获取差异数据;差异数据包括第一时间段内,针对第一存储设备执行输入输出操作的业务数据,第一时间段是上一次开始向第二存储设备同步数据的时刻至当前时刻之间的时间段,第一存储设备是业务数据生产端的存储设备;第二存储设备是业务数据容灾端的存储设备;将差异数据同步到第二存储设备中。
可以理解的是,由于该方法在第一存储设备确定自身的负载小于或等于指定阈值时通过获取上一次开始向第二存储设备同步数据的时刻到当前时刻之间变化的差异数据,并且在当前时刻将获取到的差异数据向第二存储设备发送,以实现业务数据的容灾备份。第一存储设备自身的负载小于或等于指定阈值,说明第一存储设备需要执行的生产业务较少,此时进行容灾备份,对生产端(即第一存储设备)的存储性能的影响较少。
在一种可能的实现方式中,在第一存储设备的负载小于或等于指定阈值的情况下,获取差异数据,包括:查询数据同步标志的属性状态,数据同步标志存储在第一存储设备中,用于表征第一存储设备当前的负载是否小于或等于指定阈值;在查询到的属性状态为第一状态的情况下,获取差异数据,第一状态用于指示第一存储设备当前的负载小于或等于指定阈值。
可以理解的是,在查询到数据同步标志的属性状态为第一状态的情况下,获取差异数据,可以保证在第一存储设备当前的负载小于或等于指定阈值时获取差异数据,并且,设置数据同步标志,有助于实现“比较第一存储设备的负载与指定阈值的运算过程”与“管理执行数据同步操作的过程”分属两条线程,所以提高了确定是否获取差异数据的效率。
在另一种可能的实现方式中,方法还包括:在查询到的属性状态为第二状态的情况下,依次在后续的检测时刻下查询数据同步标志的属性状态,直到达到第一检测时刻时停止查询属性状态,检测时刻是第一存储设备检测第一存储设备的负载是否小于或等于指定阈值的时刻,第一检测时刻是按照时间顺序在后续查询到的第一个属性状态是第一状态的检测时刻;在第一检测时刻获取差异数据。
可以理解的是,若当前时刻的数据同步标志的属性状态为第二状态,则可以中断连续执行数据同步的操作,通过依次在当前时刻达到后续检测时刻时,查询属性状态是否发生改变,可以及时按照第一存储设备的负载情况调整数据同步的操作,使得第一存储设备可以自适应调整是否向第二存储设备同步数据。
在另一种可能的实现方式中,方法还包括:在查询到的属性状态为第二状态,且依次在后续的检测时刻下查询数据同步标志的属性状态的过程中未查询到属性状态是第一状态时,当前时刻与上一次的同步启动时刻之间的时间间隔达到预设时间段的情况下,获取差异数据,同步启动时刻是开始向第二存储设备同步数据的时刻。
可以理解的是,若后续查询数据同步标志的属性状态是否变换为第二状态的过程中,当前时刻与上一次的同步启动时刻之间的时间间隔到达了预设时间段,则可以不用考虑负载是否小于或等于指定阈值,获取差异数据,直接启动向第二存储设备同步数据的过程,避免了由于第一存储设备的负载大于指定阈值的时间较长,所导致的累积大量数据未向第二存储设备及时同步的问题。
在另一种可能的实现方式中,查询数据同步标志的属性状态,包括:在接收到第二存储设备返回的同步完成响应的第一时刻,查询数据同步标志的属性状态,同步完成响应用于指示第一存储设备上一次向第二存储设备同步的业务数据已经同步完成。
在查询到的属性状态为第一状态的情况下,获取差异数据,包括:在查询到第一时刻的属性状态为第一状态的情况下,获取差异数据。
可以理解的是,在第一存储设备接收到第二存储设备返回的同步完成响应时,第一存储设备可以立即查询数据同步标志的属性状态,使得在属性状态为第一状态时实现连续的数据同步过程,以自适应的方式连续进行数据同步,减少故障发生时的数据丢失量。
在另一种可能的实现方式中,在第一存储设备的负载小于或等于指定阈值的情况下,获取差异数据,包括:在确定第一存储设备的负载小于或等于指定阈值的时刻,对第一存储设备中的业务数据生成第一快照;基于第一快照,以及上一次开始向第二存储设备同步数据的时刻到当前时刻之间的数据变换日志,获取差异数据,数据变换日志是第一存储设备执行输入输出操作时记录的日志。
可以理解的是,通过快照与数据变换日志相结合的方式获取差异数据,可以保证获取差异数据的准确性。
在另一种可能的实现方式中,负载通过以下参数中的至少一种来表征:第一存储设备的处理器CPU的占用情况、内存占用情况、业务数据输入输出IO接口的带宽情况。
可以理解的是,通过包括CPU的占用情况、内存占用情况、业务数据输入输出IO接口的带宽情况在内的表征第一存储设备的负载的参数,有利于确定向第二存储设备进行数据同步的时间,有利于平衡第一存储设备执行生产业务以及执行同步操作的时机,优先确保同步操作对第一存储设备负载的影响不会导致第一存储设备无法按照一定效率完成生产业务。
在一种可能的实现方式中,将差异数据同步到第二存储设备中,包括:在差异数据的数据量大于指定数据量的情况下,将差异数据同步到第二存储设备中。
可以理解的是,在确定第一存储设备的负载小于或等于指定阈值的情况下,可以根据差异数据的大小确定是否开始向第二存储设备执行数据同步,若差异数据的数据量小于或等于指定数据量,则不会开始向第二存储设备执行数据同步,直到在确定第一存储设备的负载小于或等于指定阈值,且获取到的差异数据的数据量大于指定数据量的情况下,开始向第二存储设备执行数据同步。可以通过上述方式可以降低第一存储设备向第二存储设备进行数据同步的次数,从而降低系统功耗。
第二方面,本申请提供了一种数据同步装置,该数据同步装置用于执行上述第一方面提供的任意一种数据同步方法。
在一种可能的实现方式中,本申请可以根据上述第一方面提供的方法,对该数据同步装置进行功能模块的划分。例如,可以对应各个功能划分各个功能模块,也可以将两个或两个以上的功能继承在一个处理模块中。示例性的,本申请可以按照功能将该数据同步装置划分为处理模块以及同步模块等。上述划分的各个功能模块执行的可能的技术方案和有益效果的描述均可以参考上述第一方面或其相应的可能的实现方式提供的技术方案,此处不再赘述。
第三方面,本申请实施例提供了一种计算设备,计算设备包括处理器和存储器,该存储器用于存储计算机程序指令,该计算机程序指令由处理器加载并执行以使计算设备实现如上述方面所述的数据同步方法。
第四方面,本申请实施例提供了一种计算机可读存储介质,该计算机可读存储介质中存储有至少一条计算机程序指令,所述计算机程序指令由处理器加载并执行以实现如上述方面所述的数据同步方法。
第五方面,本申请实施例提供了一种计算机程序产品,该计算机程序产品包括计算机指令,该计算机指令存储在计算机可读存储介质中。计算设备的处理器从计算机可读存储介质读取该计算机指令,处理器执行该计算机指令,使得该计算设备执行上述第一方面的各种可选实现方式中提供的数据同步方法。
本申请中第二方面到第五方面及其各种实现方式的具体描述,可以参考第一方面及其各种实现方式中的详细描述;并且,第二方面到第五方面及其各种实现方式的有益效果,可以参考第一方面及其各种实现方式中的有益效果分析,此处不再赘述。
本申请的这些方面或其他方面在以下的描述中会更加简明易懂。
附图说明
图1是根据一示例性实施例示出的一种数据同步系统的示意图;
图2是图1所示实施例中涉及的一种逻辑卷之间建立远程复制一致性组的示意图;
图3是根据一示例性实施例示出的一种数据同步方法的流程示意图;
图4是图3所示实施例中涉及的一种数据同步启动时刻确定的示意图;
图5是根据一示例性实施例示出的一种数据同步方法的流程示意图;
图6是图5所示实施例中涉及的一种自适应检测模块对数据同步标志确定的流程示意图;
图7是图5所示实施例中涉及的一种性能检测及数据同步过程的示意图;
图8是图5所示实施例中涉及的一种自适应连续数据复制流程示意图;
图9是本申请一个示例性实施例提供的数据同步装置的结构示意图;
图10是本申请一个示例性实施例提供的计算设备的结构示意图。
具体实施方式
为使本申请的目的、技术方案和优点更加清楚,下面将结合附图对本申请实施方式作进一步地详细描述。
在本文中提及的“多个”是指两个或两个以上。“和/或”,描述关联对象的关联关系,表示可以存在三种关系,例如,A和/或B,可以表示:单独存在A,同时存在A和B,单独存在B这三种情况。字符“/”一般表示前后关联对象是一种“或”的关系。
企业为了保证在应用发生故障或者生产端的存储设备中存储的数据丢失的情况下,生产端的存储设备存储的业务数据中的关键数据不会丢失,需要对上述业务数据进行容灾备份,并且基于容灾备份的数据进行数据恢复以实现对业务数据保护。当前,存储设备通常会根据业务数据的不同类型,划分出不同的逻辑卷(logical volume),不同的逻辑卷可以用于存储不同类型的业务数据,针对生产端的存储设备的逻辑卷上数据的保护方法,可以通过存储间的数据复制技术实现,通过远程复制技术将生产数据中心中的逻辑卷与容灾站点的逻辑卷之间建立连接关系,以实现一定距离下的数据同步过程,从而达到对生产数据中的数据容灾保护的作用。而针对数据容灾保护的方案,在故障发生后的业务恢复时长以及数据恢复量是衡量容灾方案的重要指标,数据容灾方案的业务恢复时长可以通过恢复时间目标(recovery time objective,RTO)表征,RTO用于指示从灾难发生到业务系统恢复服务功能所需要的最短时间周期;数据容灾方案的数据恢复量可以通过数据恢复点目标(recovery point objective,RPO)表征,用于指示业务系统所能容忍的数据丢失量。当生产端发生故障时,由于生产业务数据的重要性,需要对存储端业务数据恢复点目标提出更高的要求,尽可能减低RPO这也是存储厂商重点考虑的需要解决问题。
相关技术中,存在生产端的存储设备与容灾端的存储设备的数据差异较大,RPO较长,导致故障发生时的数据丢失量较多等问题,另一方面,相关技术中没有考虑触发生产端向容灾端同步数据时生产端是否繁忙,所以会导致生产端执行同步操作与多个生产侧正在执行的业务操作之间发生并行的情况,从而影响正在执行的生产业务操作的完成效率。本申请实施例提出一种数据同步方法,在进行数据同步之前对生产端的存储设备进行负载情况的检测,即若检测到生产端的存储设备的负载在当前时刻小于或等于指定阈值时,才可以进行后续的数据同步操作,平衡了生产端进行生产业务与执行同步操作之间的关系,利用生产端的业务负载指标实现了自适应的调整进行同步的时刻,从而减少了数据同步对生产端存储设备的负载影响。
请参考图1,图1是本申请实施例涉及的一种数据同步系统的示意图,该数据同步系统中包括生产数据中心100以及容灾数据中心200。生产数据中心100中可以包括生产主机以及第一存储设备110。第一存储设备110中包括IO接口101、处理器102、缓冲存储器以及第一存储器103。容灾数据中心200中可以包括容灾主机以及第二存储设备210。第二存储设备210中可以包括缓冲存储器以及第二存储器201。
其中,生产主机可以是服务器,第一存储设备110是生产端的用于存储业务数据的存储设备。该第一存储设备110可以是计算设备,第二存储设备210是容灾端用于存储复制的业务数据的存储设备,该第二存储设备210同样也可以是计算设备。计算设备可以是智能手机、平板电脑、电子书阅读器、个人便携式计算机、服务器等。
可选的,IO接口101可以用于实现生产主机与第一存储设备110之间的通信,其中,生产主机可以通过IO接口101向第一存储设备110发送IO指令,IO指令用于指示业务数据的输入输出操作。第一存储设备110中的处理器102可以用于处理生产业务,也可以用于执行本方案的数据同步方法。缓冲存储器可以用于临时存储操作指令以及业务数据。第一存储器 103可以是磁盘阵列,该磁盘阵列可以按照逻辑单元号(logical unit number,LUN)划分为多个主逻辑卷,包括主LUN001到主LUN n;同样的,第二存储器201也可以是磁盘阵列,并且该磁盘阵列按照LUN划分为多个从逻辑卷,包括从LUN001到从LUN n。其中,主逻辑卷与从逻辑卷之间可以建立远程复制一致性组(replication group),由于主从逻辑卷是一一对应的,所以主从逻辑卷的数量可以是相同的,并且每一个主逻辑卷对应一个从逻辑卷。
比如,图2是本申请实施例涉及的一种逻辑卷之间建立远程复制一致性组的示意图。如图2所示,第一存储器103中的主逻辑卷按照LUN可以划分为主LUN001、主LUN002以及主LUN003等,与主逻辑卷对应的,在第二存储器201中包括的从逻辑卷按照LUN可以划分为从LUN001、从LUN002以及从LUN003等。若主LUN001与主LUN002中可存储的业务数据的类型相同,则可以建立主LUN001以及主LUN002与从LUN001以及从LUN002之间的远程复制一致性关系,将主LUN001以及主LUN002与从LUN001以及从LUN002作为一个远程复制一致性组1,通过该远程复制一致性组1执行远程复制任务1。若主LUN003中可存储的业务数据为单独的一种数据类型,则可以建立主LUN003与从LUN003之间的远程复制一致性关系,将主LUN003与从LUN003作为一个远程复制一致性组2,通过该远程复制一致性组2执行远程复制任务2。
在一种可能的实现方式中,生产主机与容灾主机之间可以通过以太网协议(internet protocol,IP)连接,也可以通过光纤通道(fibre channel,FC)连接。第一存储设备110与第二存储设备210之间也可以通过IP或者FC连接。并且通过上述通信连接可以实现第一存储设备110将存储的业务数据复制到第二存储设备210的操作,也可以实现在第一存储设备110发生故障时,第一存储设备110接收第二存储设备210返回的业务数据以恢复数据。
图3示出了本申请一个示例性实施例提供的数据同步方法的流程示意图。其中,该数据同步方法可以由第一存储设备执行,比如,该第一存储设备可以是图1所示的数据同步系统中的第一存储设备110。该数据同步方法包括如下步骤:
S301,在第一存储设备的负载小于或等于指定阈值的情况下,获取差异数据。
在本申请实施例中,第一存储设备可以在向第二存储设备同步数据之前,第一存储设备可以确定自身的负载是否小于或等于指定阈值。
在一种可能的实现方式中,若当前时刻第一存储设备没有处于向第二存储设备进行数据同步的过程,第一存储设备可以确定在当前时刻自身的负载是否小于或等于指定阈值,若确定当前时刻第一存储设备的负载小于或等于指定阈值,则可以获取差异数据。
其中,在第一存储设备与第二存储设备已经建立远程复制一致性组的情况下,第一存储设备首先需要与第二存储设备进行初始的数据同步,在第一存储设备与第二存储设备实现初始数据同步后,第一存储设备可以开始确定当前时刻的第一存储设备的负载是否小于或等于指定阈值,若确定第一存储设备的负载小于或等于指定阈值,则可以获取当前时刻下的差异数据,差异数据可以是包括第一时间段内针对第一存储设备执行输入输出操作的业务数据,第一时间段是上一次开始向第二存储设备同步数据的时刻至当前时刻之间的时间段。
也就是说,第一存储设备可以将确定自身的负载小于或等于指定阈值的时刻作为本次的同步启动时刻,即同步启动时刻是开始向第二存储设备执行数据同步操作的时刻,也可以是开始获取需要向第二存储设备发送的差异数据的时刻。
本申请实施例中的指定阈值可以是用户基于需求预先设置的,比如,指定阈值的数值大小与向第二存储单元同步数据的优先程度呈正相关,也就是说,若需要尽快向第二存储设备同步数据,则可以将指定阈值的数值适当提高,以保证第一存储设备向第二存储设备同步数 据更加及时;若需要保证生产端执行生产业务的效率,则可以将指定阈值的数值适当降低,以保证第一存储设备向第二存储设备同步数据时第二存储设备不会影响自身执行的生产业务。或者,指定阈值可以根据生产端的业务负载来实时动态更新调整,在此不受限制。
比如,图4是本申请实施例涉及的一种数据同步启动时刻确定的示意图。如图4所示,当第一存储设备向第二存储设备完成初始同步过程时,即从初始时刻开始直到时刻1结束,完成向第二存储设备同步初始数据的过程,然后在时刻1确定时刻1下第一存储设备的负载是否小于或等于指定阈值,若第一存储设备的负载小于或等于指定阈值则可以将时刻1作为本次数据同步启动时刻,开始向第二存储设备进行第一次数据同步过程,在时刻1获取到的需要传输给第二存储设备的差异数据可以是在初始时刻到时刻1之间的针对第一存储设备执行输入输出操作的业务数据,也就是说第一次同步过程需要发送给第二存储设备的差异数据包括在时刻1下第一存储设备中的业务数据相较于初始时刻的第一存储设备中的业务数据的差异数据。接下来,由于第一存储设备向第二存储设备进行的第一次数据同步过程的第一次同步时间是从时刻1开始到时刻2结束,则可以在时刻2继续确定时刻2下的第一存储设备的负载是否小于或等于指定阈值。
在一种可能的实现方式中,在确定第一存储设备的负载小于指定阈值的时刻,对第一存储设备中的业务数据生成第一快照;基于第一快照,以及上一次开始向第二存储设备同步数据的时刻到当前时刻之间的数据变换日志,获取差异数据,该数据变换日志是第一存储设备执行输入输出操作时记录的日志。
S302,在第一存储设备的负载大于指定阈值的情况下,按照时间顺序继续确定第一存储设备的负载是否小于或等于指定阈值,直到确定第一存储设备的负载小于或等于指定阈值的时刻,获取差异数据。
本申请实施例中,在第一存储设备确定当前时刻下自身的负载大于指定阈值的情况下,在当前时刻下不获取差异数据并向第二存储设备发送差异数据,按照时间顺序确定后续时刻的负载是否小于或等于指定阈值,直到确定当前时刻的第一存储设备的负载小于或等于指定阈值,并且获取差异数据。
在一种可能的实现方式中,后续继续确定第一存储设备的负载是否小于或等于指定阈值的时刻可以是实时确定,或者,也可以是按照一定的周期,在当前时刻达到周期指示的时刻时确定第一存储设备的负载是否小于或等于指定阈值。
比如,如图4所示,若在时刻2下的第一存储设备的负载大于指定阈值,则第一存储设备可以在后续时刻继续确定第一存储设备的负载是否小于或等于指定阈值,若直到当前时刻达到时刻3时第一存储设备才确定自身的负载小于或等于指定阈值,则将时刻3确定为向第二存储设备进行第二次数据同步的数据同步启动时刻,并且在时刻3开始向第二存储设备进行第二次数据同步过程,在时刻3获取到的需要在本次传输给第二存储设备的差异数据可以是在上一次同步启动的时刻1到当前的时刻3之间的针对第一存储设备执行输入输出操作的业务数据,也就是说第二次同步过程需要发送给第二存储设备的差异数据包括在时刻3下第一存储设备中的业务数据相较于时刻1下第一存储设备中的业务数据的差异数据。接下来,由于第一存储设备向第二存储设备进行的第二次数据同步过程的第二次同步时间是从时刻3开始到时刻4结束,则可以在时刻4继续确定时刻4下的第一存储设备的负载是否小于或等于指定阈值。
S303,在第一存储设备的负载大于指定阈值,且按照时间顺序尚未确定第一存储设备的负载小于或等于指定阈值时,当前时刻与上一次的同步启动时刻之间的时间间隔达到预设时 间段的情况下,获取差异数据。
本申请实施例中,在当前时刻下第一存储设备确定自身的负载大于指定阈值,并且当前时刻与上一次同步启动时刻之间的时间间隔达到预设时间段的情况下,获取差异数据。
也就是说,若第一存储设备向第二存储设备进行一次数据同步后的一段时间中,第一存储设备的负载均大于指定阈值,为了使第一存储设备中的业务数据与第二存储设备中容灾备份的业务数据不会相差过多,所以可以预先设置一个预设时间段,使得第一存储设备即使确定自身的负载在当前时刻不满足小于或等于指定阈值的条件,也可以在达到预设时间段时获取差异数据,一定程度上可以稳定RPO的上限。
比如,如图4所示,若在时刻2下的第一存储设备的负载大于指定阈值,则第一存储设备可以在后续时刻继续确定第一存储设备自身的负载是否小于或等于指定阈值,若直到当前时刻达到时刻3时第一存储设备仍未确定得到自身的负载小于或等于指定阈值,并且时刻3与时刻1之间的时间间隔达到预设时间段,则可以在时刻3开始向第二存储设备进行第二次数据同步过程,在时刻3获取到的需要在本次传输给第二存储设备的差异数据可以是在上一次同步启动的时刻1到当前的时刻3之间的针对第一存储设备执行输入输出操作的业务数据,也就是说第二次同步过程需要发送给第二存储设备的差异数据包括在时刻3下第一存储设备中的业务数据相较于时刻1下第一存储设备中的业务数据的差异数据。接下来,由于第一存储设备向第二存储设备进行的第二次数据同步过程的第二次同步时间是从时刻3开始到时刻4结束,则可以在时刻4继续确定时刻4下的第一存储设备的负载是否小于或等于指定阈值。
S304,第一存储设备将差异数据同步到第二存储设备中。
在本申请实施例中,第一存储设备获取到差异数据后,将差异数据复制发送给第二存储设备,以实现第一存储设备中的业务数据同步到第二存储设备中的目的。
其中,第一存储设备将差异数据同步是用于指示将差异数据拷贝到第二存储设备的过程。
在一种可能的实现方式中,获取到当前的差异数据后,按照差异数据所属的第一存储设备中的主逻辑卷,通过存有差异数据的主逻辑卷与第二存储设备中与主逻辑卷对应的从逻辑卷之间建立的远程复制一致性组传输差异数据,使得第二存储设备中与主逻辑卷对应的从逻辑卷中备份有差异数据,从而使得第二存储设备中的存储有容灾的业务数据。
在一种可能的实现方式中,在差异数据的数据量大于指定数据量的情况下,第一存储设备可以将差异数据同步到第二存储设备中。
其中,在第一存储设备获取差异数据后,第一存储设备可以确定差异数据的大小是否大于指定数据量,在确定差异数据的大小大于指定数据量时,可以将该差异数据同步到第二存储设备。指定数据量可以是按照RPO需求预设的。
示例性的,当某时刻下触发了第一存储设备向第二存储设备执行业务数据同步操作,本次的业务数据向第二存储设备同步使用了10分钟完成,在这10分钟期间,第一存储设备的主逻辑卷发生了100MB的变化,所以在历时10分钟的同步过程结束之后,在确定第一存储设备的负载小于或等于指定阈值,且100MB大于指定数据量20MB的情况下,第一存储设备可以立即开始对这100MB的差异数据继续进行同步;假如这次同步过程用了5分钟,而这5分钟期间第一存储设备的主逻辑卷发生了50MB的变化,由于确定了第一存储设备的负载小于或等于指定阈值,且50MB大于指定数据量20MB,所以第一存储设备继续同步这50MB数据;假如这次同步过程用了2分钟,期间又产生了20MB的变化;由于20MB不超过指定数据量,所以在下一个第一存储设备的负载小于或等于指定阈值,且差异数据大于20MB的时刻继续进行数据同步,从而在保证系统一定的RPO的情况下,减少数据同步的次数,降低 系统的功耗。
S305,第二存储设备向第一存储设备返回同步完成响应。
在本申请实施例中,第二存储设备在完全接收第一存储设备发送的差异数据并存储在自身的逻辑卷中之后,可以确定本次第一存储设备向第二存储设备同步数据的过程完成,并且然后向第一存储设备返回同步完成响应。
其中,该同步完成响应用于指示上一次向第二存储设备同步的业务数据已经同步完成。
在一种可能的实现方式中,在第一存储设备接收到同步完成响应后可以开始新一轮的对自身的负载是否小于或等于指定阈值的确定,并且重新执行S301至S303的步骤,以开启下一轮的第一存储设备向第二存储设备进行数据同步的过程。
上述实施例中,第一存储设备可以通过处理器中的数据复制管理模块,在确定需要确定自身的负载情况时,确定当前的负载是否小于指定阈值,也就是说,在第一存储设备的业务系统正常运行的过程中,第一存储设备可以实时监控并记录自身的负载情况,在接收到第二存储设备返回的同步完成响应后可以开始获取当前时刻自身的负载情况并且与指定阈值进行比较,使得第一存储设备可以在开启下一次向第二存储设备同步数据之前确定自身的负载情况是否可以在不影响生产业务的前提下,实现向第二存储设备同步数据。
在另一种可能的实现方式中,第一存储设备中的处理器中除了包括数据复制管理模块,还可以包括自适应检测模块。该自适应检测模块可以用于对当前时刻下第一存储设备的负载情况是否小于或等于指定阈值进行检测,并且实时修改数据同步标志的属性状态,以供数据复制管理模块进行查询,数据复制管理模块按照查询到的数据同步标志的属性状态确定是否启动向第二存储设备同步业务数据。图5示出了本申请一个示例性实施例提供的数据同步方法的流程示意图。其中,该数据同步方法可以由第一存储设备执行。该数据同步方法具体包括如下步骤:
S31,第一存储设备确定当前时刻下数据同步标志的属性状态。
在本申请实施例中,第一存储设备可以确定当前时刻下存储在第一存储设备中的数据同步标志的属性状态。
其中,数据同步标志可以存储在第一存储设备中,并且用于表征第一存储设备当前的负载是否小于或等于指定阈值。
在一种可能的实现方式中,当确定第一存储设备当前的负载小于或等于指定阈值时,将数据同步标志的属性状态确定为第一状态;当确定第一存储设备当前的负载大于指定阈值时,将数据同步标志的属性状态确定为第二状态。
其中,第一状态用于指示第一存储设备当前的负载小于或等于指定阈值。第二状态用于指示第一存储设备当前的负载大于指定阈值。
示例性的,存储在第一存储设备中的数据同步标志可以表示为STOP_REPLICATION,当第一存储设备确定当前时刻下的负载是否小于或等于指定阈值后,确定当前负载小于或等于指定阈值,则可以将数据同步标志STOP_REPLICATION的属性状态标记为FALSE,也就是说,数据同步标志STOP_REPLICATION的第一状态可以标记为FALSE。若确定当前负载大于指定阈值,则可以将数据同步标志STOP_REPLICATION的属性状态标记为TRUE,也就是说,数据同步标志STOP_REPLICATION的第二状态可以标记为TRUE。
在一种可能的实现方式中,负载可以通过以下参数中的至少一种来表征:第一存储设备的处理器CPU的占用情况、内存占用情况、业务数据输入输出IO接口的带宽情况。
其中,若第一存储设备的负载可以通过第一存储设备的CPU占用情况来表征,CPU占用 情况可以是当前时刻以及当前时刻之前最近记录的三次CPU占用率的平均值。
若第一存储设备的负载可以通过第一存储设备的内存占用情况来表征,内存占用情况可以是在当前时刻下检测到的内存占用率。
若第一存储设备的负载可以通过第一存储设备的IO接口的带宽情况来表征,IO接口的带宽情况可以是当前时刻检测到的IO接口的带宽。
在一种可能的实现方式中,当需要第一存储设备的负载通过上述至少两种参数表征,则可以确定在表征负载的各个参数均小于各自对应的指定阈值的情况下,第一存储设备可以确定当前时刻下的数据同步标志的属性状态为第一状态。
其中,与表征负载的不同参数各自对应的指定阈值可以是相同或者不同的,与CPU占用率的平均值对应的指定阈值可以是预先设定为指定百分比,同样,与内存占用率对应的指定阈值也可以是指定百分比,与IO接口的带宽对应的指定阈值可以是历史带宽峰值的指定百分比。比如,指定百分比可以是80%。
在一种可能的实现方式中,除了第一存储设备的负载,第一存储设备的其他性能也可以影响确定当前时刻下数据同步标志的属性状态,因此针对第一存储设备中的表征性能的参数,在第一存储设备的负载小于或等于指定阈值的同时第一存储设备的性能满足预设指标的情况下,可以确定当前时刻下的数据同步标志的属性状态为第一状态,但是在第一存储设备的负载小于或等于指定阈值的同时第一存储设备的性能不满足预设指标的情况下,可以确定当前时刻下的数据同步标志的属性状态为第二状态。
示例性的,图6是本申请实施例涉及的一种自适应检测模块对数据同步标志确定的流程示意图。如图6所示,第一存储设备可以检查当前时刻下第一存储设备的CPU负载是否大于CPU负载对应的阈值(S11),若检查到当前时刻下第一存储设备的CPU负载小于或等于CPU负载对应的阈值,则继续检查第一存储设备的内存负载是否大于内存负载对应的阈值(S12),若检查到当前时刻下第一存储设备的内存负载小于或等于内存负载对应的阈值,则继续检查第一存储设备的业务带宽负载是否大于业务带宽负载对应的阈值(S13),若检查到当前时刻下第一存储设备的业务带宽负载于等于业务带宽负载对应的阈值,则检查第一存储设备扩展的性能指标负载是否大于性能指标负载对应的阈值(S14),若第一存储设备扩展的性能指标负载小于或等于性能指标负载对应的阈值,则第一存储设备设置数据同步标志STOP_REPLICATION为FALSE,并且启动向第二存储设备进行自适应数据同步的过程(S15),然后休眠一个时间间隔,循环自适应性能检测(S16),若表征负载的各个参数中的至少一个确定大于对应的阈值,则将第一存储设备设置数据同步标志STOP_REPLICATION为TRUE,并再次开始进行循环自适应性能检测(S17)。
S32,第一存储设备查询当前时刻下数据同步标志的属性状态。
在本申请实施例中,由于第一存储设备在当前时刻下检测负载与指定阈值之间的关系后可以确定数据同步标志的属性状态,所以第一存储设备仅需要查询当前时刻下的数据同步标志的属性状态就可以确定当前时刻下第一存储设备是否支持向第二存储设备同步数据。
示例性的,第一存储设备可以查询当前时刻下的数据同步标志STOP_REPLICATION为TRUE还是FALSE。
S33,第一存储设备在查询到的数据同步标志的属性状态为第一状态的情况下,获取差异数据。
在本申请实施例中,第一存储设备可以在当前时刻达到上一次向第二存储设备同步数据完成后的时刻,查询当前时刻的数据同步标志的属性状态,当查询到当前时刻的数据同步标 志的属性状态为第一状态的情况时,开始获取当前时刻的业务数据与上一次开始向第二存储设备同步数据时的业务数据之间的差异数据。
在一种可能的实现方式中,第一存储设备可以查询当前时刻第一存储设备的数据同步标志,若当前时刻第一存储设备的数据同步标志的属性状态是第一状态,则可以获取当前时刻的差异数据。
其中,数据同步标志可以存储在第一存储设备中,用于表征第一存储设备当前的负载是否小于或等于指定阈值。
在一种可能的实现方式中,第一存储设备在接收到第二存储设备返回的同步完成响应的第一时刻,查询数据同步标志的属性状态。
其中,同步完成响应用于指示第一存储设备上一次向第二存储设备同步的业务数据已经同步完成。也就是说,在上一次向第二存储设备完成数据同步后,也就是需要发送给第二存储设备的业务数据全部发送完成并且成功写入第二存储设备中对应的从逻辑卷时,可以向第一存储设备返回表示数据同步完成的同步完成响应,并且该第一时刻既可以是上一次第一存储设备向第二存储设备同步数据的完成时刻,也可以是在查询到数据同步标志的属性状态为第一状态时第一存储设备当前向第二存储设备进行数据同步的起始时刻。第一状态可以用于指示第一存储设备当前的负载小于或等于指定阈值。
在一种可能的实现方式中,在查询到第一时刻的属性状态为第一状态的情况下,获取差异数据。
示例性的,图7是本申请实施例涉及的一种性能检测及数据同步过程的示意图。性能检测过程以及数据同步过程在时间上是并行的两条线程,性能检测过程可以按照预设周期检测当前时刻下的第一存储设备的负载是否小于或等于指定阈值,并且在确定第一存储设备的负载小于或等于指定阈值时将当前时刻下的数据同步标志的属性状态设置为第一状态,在确定第一存储设备的负载大于指定阈值时将当前时刻下的数据同步标志的属性状态设置为第二状态,如图7所示,由于在时刻1时数据同步标志的属性状态为第一状态,所以开始向第二存储设备进行第一次数据同步,在当前时刻到达时刻2时向第二存储设备进行第一次数据同步结束,并且在当前时刻到达时刻2时查询当前的数据同步标志的属性状态,由于在当前时刻,即时刻2时查询到当前的数据同步标志的属性状态为第一状态,可以开始获取本次待复制同步给第二存储设备的差异数据。
S34,第一存储设备在查询到当前时刻下的数据同步标志的属性状态是第二状态的情况下,按照时间顺序继续查询接下来的时刻的数据同步标志的属性状态,直到第一存储设备查询到数据同步标志的属性状态变换为第一状态时,开始获取差异数据。
在本申请实施例中,第一存储设备在查询到当前时刻下的数据同步标志的属性状态是第二状态的情况下,按照时间顺序继续在接下来的各个时刻查询数据同步标志的属性状态,直到查询到第一存储设备的数据同步标志的属性状态由第二状态变换为第一状态,开始获取差异数据。
在一种可能的实现方式中,第一存储设备可以按照预设周期查询数据同步标志的属性状态。
也就是说,由于第一存储设备可以按照预设周期确定数据同步标志的属性状态,所以第一存储设备的数据同步标志更新的周期可以是预设周期,即只有在按照预设周期更新数据同步标志的属性状态的时刻,才需要查询数据同步标志的属性状态是否由第二状态变换为第一状态。
其中,第一存储设备在查询到的数据同步标志的属性状态为第二状态的情况下,依次在后续的检测时刻下查询数据同步标志的属性状态,直到达到第一检测时刻时停止查询属性状态,在第一检测时刻获取差异数据。
其中,检测时刻可以是第一存储设备检测第一存储设备的负载是否小于或等于指定阈值的时刻,第一检测时刻可以是按照时间顺序在后续查询到的第一个属性状态是第一状态的检测时刻。
示例性的,如图7所示,若当前时刻到达时刻3时,第一存储设备向第二存储设备进行第二次数据同步结束,然后可以在时刻3查询当前的数据同步标志的属性状态,由于在时刻3时查询到的属性状态是第二状态,所以可以将暂停连续复制过程,在当前时刻到达下一个检查时刻时确定当前的属性状态是否发生变化,由于确定该检测时刻下的数据同步标志的属性状态仍未第二状态,所以继续在当前时刻到达再下一个检测时刻时对该再下一个检测时刻的数据同步标志的属性状态进行查询,若该再下一个检测时刻是时刻4,则由于当前时刻到达时刻4时,查询到的数据同步标志变换为第一状态,所以将时刻4作为向第二存储设备进行第三次数据同步的起始时间,开始获取差异数据。
S35,在查询到的属性状态为第二状态,且依次在后续的检测时刻下查询数据同步标志的属性状态的过程中未查询到属性状态是第一状态时,当前时刻与上一次的同步启动时刻之间的时间间隔达到预设时间段的情况下,获取差异数据。
在本申请实施例中,在S34的执行过程中,若后续检测数据同步标志的属性状态是否变换为第二状态的过程中,当前时刻与上一次的同步启动时刻之间的时间间隔到达了预设时间段,则可以不用考虑负载是否小于或等于指定阈值,获取差异数据,直接启动向第二存储设备同步数据的过程。
其中,同步启动时刻是开始向第二存储设备同步数据的时刻。预设时间段可以是异步远程复制技术中预设的时间间隔。
示例性的,如图7所示,若第一存储设备在时刻5启动向第二存储设备同步数据的过程完成时,可以查询数据同步标志的属性状态,由于该次数据同步完成时刻的数据同步标志的属性状态为第二状态,所以在当前时刻到达后续检测时刻时对数据同步标志的属性状态进行查询,由于直到时刻6,各个检测时刻下的数据同步标志的属性状态均为第二状态,并且时刻5与时刻6之间的时间间隔达到了预设时间段,所以可以将时刻6作为数据启动时刻,获取差异数据。
S36,第一存储设备将差异数据同步到第二存储设备中,在差异数据同步完成时,第二存储设备向第一存储设备返回同步完成响应。
本申请实施例的具体实现方式参见S304以及S305,在此不再赘述。
示例性的,图8是本申请实施例涉及的一种自适应连续数据复制流程示意图。如图8所示,第一存储设备在接收到上一次向第二存储设备同步数据完成时返回的同步完成响应时启动自适应数据复制,查询确定数据同步标志STOP_REPLICATION是否为FALSE,若查询数据同步标志STOP_REPLICATION是FALSE则表示连续复制不停止,即可以接着上一次的数据同步过程继续进行本次的数据同步过程;若查询数据同步标志STOP_REPLICATION是TRU,则表示连续复制停止,即上一次的数据同步过程与本次的数据同步过程之间存在间隔(S21),若查询数据同步标志STOP_REPLICATION是TRUE,则进入休眠,可以修改休眠的时间间隔,该休眠的时间间隔默认1s是可配置的,该休眠的时间间隔可以是检测时刻之间的时间间隔(S22),判断当前时刻与上一次同步启动时刻之间的间隔是否大于预设时间段 (S23),若大于预设时间段,则执行S24,若小于或等于预设时间段,则执行S21。第一存储设备可以记录同步启动时刻RG_REP_START_TIME(S24),第一存储设备清除原快照RGNO_BEFORE_REP_SNAP,重新生成快照RGNO_BEFORE_REP_SNAP,其中,RGNO可以用于表示远程复制一致性组的身份标识(S25)。基于RGNO_BEFORE_REP_SNAP以及DCL获取当前快照的差异数据,将差异数据复制到容灾端的第二存储设备(S26),第二存储设备可以返回差异数据写入成功的响应(S27)。
本申请实施例中,当某时刻下触发了第一存储设备向第二存储设备执行业务数据同步操作,本次的业务数据向第二存储设备同步使用了10分钟完成,在这10分钟期间,第一存储设备的主逻辑卷发生了100MB的变化,所以在历时10分钟的同步过程结束之后,在确定第一存储设备的负载小于或等于指定阈值的情况下,第一存储设备可以立即开始对这100MB的差异数据继续进行同步;假如这次同步过程用了5分钟,而这5分钟期间第一存储设备的主逻辑卷发生了50MB的变化,然后第一存储设备继续同步这50MB数据;假如这次同步过程用了2分钟,期间又产生了20MB的变化;若再同步这20MB用了1分钟可以一直往下同步到收敛,通过异步复制追赶的方式最后可以使得同步过程的间隔越来越短,每次传输的数据越来越小,从而可以最大程度地保证系统的RPO。
上述主要从方法的角度对本申请实施例的方案进行了介绍。可以理解的是,数据同步装置为了实现上述功能,其包含了执行各个功能相应的硬件结构和软件模块中的至少一个。本领域技术人员应该很容易意识到,结合本文中所公开的实施例描述的各示例的单元及算法步骤,本申请能够以硬件或硬件和计算机软件的结合形式来实现。某个功能究竟以硬件还是计算机软件驱动硬件的方式来执行,取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能,但是这种实现不应认为超出本申请的范围。
本申请实施例可以根据上述方法示例对数据恢复装置进行功能单元的划分,例如,可以对应各个功能划分各个功能单元,也可以将两个或两个以上的功能集成在一个处理单元中。上述集成的单元既可以采用硬件的形式实现,也可以采用软件功能单元的形式实现。需要说明的是,本申请实施例中对单元的划分是示意性的,仅仅为一种逻辑功能划分,实际实现时可以有另外的划分方式。
示例性的,图9示出了本申请一个示例性实施例提供的数据同步装置400的结构示意图。该数据同步装置400应用于第一存储设备中,该数据同步装置400包括:
处理模块410,用于在所述第一存储设备的负载小于或等于指定阈值的情况下,获取差异数据;所述差异数据包括第一时间段内,针对所述第一存储设备执行输入输出操作的业务数据,所述第一时间段是上一次开始向第二存储设备同步数据的时刻至当前时刻之间的时间段,第一存储设备是业务数据生产端的存储设备;所述第二存储设备是业务数据容灾端的存储设备;
同步模块420,用于将所述差异数据同步到所述第二存储设备中。
例如,结合图3,处理模块410可以用于执行如图3所示的S301,同步模块420可以用于执行如图3所示的S304。
在一种可能的实现方式中,所述处理模块410,还包括:
查询子模块,用于查询数据同步标志的属性状态,所述数据同步标志存储在所述第一存储设备中,用于表征所述第一存储设备当前的所述负载是否小于或等于所述指定阈值;
获取子模块,用于在查询到的所述属性状态为第一状态的情况下,获取差异数据,所述 第一状态用于指示所述第一存储设备当前的所述负载小于或等于所述指定阈值。
例如,结合图5,处理模块410可以用于执行如图5所示的S31。
在一种可能的实现方式中,所述处理模块410,还包括:
在查询到的所述属性状态为第二状态的情况下,依次在后续的检测时刻下查询所述数据同步标志的属性状态,直到达到第一检测时刻时停止查询所述属性状态,所述检测时刻是所述第一存储设备检测所述第一存储设备的所述负载是否小于或等于所述指定阈值的时刻,所述第一检测时刻是按照时间顺序在后续查询到的第一个所述属性状态是所述第一状态的检测时刻,所述第二状态用于指示所述第一存储设备当前的负载大于所述指定阈值;
在所述第一检测时刻获取所述差异数据。
在一种可能的实现方式中,所述处理模块410,还包括:
在查询到的所述属性状态为所述第二状态,且依次在后续的检测时刻下查询所述数据同步标志的属性状态的过程中未查询到所述属性状态是所述第一状态时,当前时刻与上一次的同步启动时刻之间的时间间隔达到预设时间段的情况下,获取所述差异数据,所述同步启动时刻是开始向所述第二存储设备同步数据的时刻。
在一种可能的实现方式中,所述查询子模块,包括:
在接收到所述第二存储设备返回的同步完成响应的第一时刻,查询所述数据同步标志的属性状态,所述同步完成响应用于指示所述第一存储设备上一次向所述第二存储设备同步的业务数据已经同步完成。
所述处理模块410,还包括:
在查询到所述第一时刻的所述属性状态为所述第一状态的情况下,获取差异数据。
在一种可能的实现方式中,所述处理模块410,还包括:
在确定所述第一存储设备的所述负载小于所述指定阈值的时刻,对所述第一存储设备中的业务数据生成第一快照;
基于所述第一快照,以及上一次开始向所述第二存储设备同步数据的时刻到当前时刻之间的数据变换日志,获取所述差异数据,所述数据变换日志是所述第一存储设备执行输入输出操作时记录的日志。
在一种可能的实现方式中,所述负载通过以下参数中的至少一种来表征:所述第一存储设备的处理器CPU的占用情况、内存占用情况、业务数据输入输出IO接口的带宽情况。
在一种可能的实现方式中,所述处理模块410,还包括:
当通过至少两种参数表征所述第一存储设备的负载时,在所述至少两种参数均不超过所述至少两种参数各自的所述指定阈值的情况下,获取所述差异数据。
在一种可能的实现方式中,所述同步模块420,包括:
在所述差异数据的数据量大于指定数据量的情况下,将所述差异数据同步到所述第二存储设备中。
关于上述可选方式的具体描述可以参见前述的方法实施例,此处不再赘述。此外,上述提供的任一种数据同步装置的解释以及有益效果的描述均可参考上述对应的方法实施例,不再赘述。
作为示例,结合图1,数据同步装置中的处理模块410和同步模块420中的部分或全部实现的功能可以通过图1中的处理器102执行图1中的第一存储设备110中的程序代码实现。
图10示出了本申请一个示例性实施例提供的计算设备1100的结构示意图。该计算设备1100可以是智能手机、平板电脑、电子书、便携式个人计算机、智能穿戴设备等电子设备。 本申请中的计算设备1100可以包括一个或多个如下部件:处理器1110以及存储器1120。
处理器1110可以包括一个或者多个处理核心。处理器1110利用各种接口和线路连接整个终端内的各个部分,通过运行或执行存储在存储器1120内的指令、程序、代码集或指令集,以及调用存储在存储器1120内的数据,执行终端的各种功能和处理数据。可选地,处理器1110可以采用数字信号处理(digital signal processing,DSP)、现场可编程门阵列(field-programmable gate array,FPGA)、可编程逻辑阵列(programmable logic array,PLA)中的至少一种硬件形式来实现。处理器1110可集成中央处理器(Central Processing Unit,CPU)、图像处理器(graphics processing unit,GPU)和调制解调器等中的一种或几种的组合。其中,CPU主要处理操作系统、用户界面和应用程序等;调制解调器用于处理无线通信。可以理解的是,上述调制解调器也可以不集成到处理器1110中,单独通过一块通信芯片进行实现。
存储器1120可以包括随机存储器(random access memory,RAM),也可以包括只读存储器(read-only memory,ROM)。可选地,该存储器1120包括非瞬时性计算机可读介质(non-transitory computer-readable storage medium)。存储器1120可用于存储指令、程序、代码、代码集或指令集。存储器1120可包括存储程序区和存储数据区,其中,存储程序区可存储用于实现操作系统的指令、用于实现至少一个功能的指令(比如触控功能、声音播放功能、图像播放功能等)、用于实现上述各个方法实施例的指令等,该操作系统可以是安卓(Android)系统(包括基于Android系统深度开发的系统)、苹果公司开发的IOS系统(包括基于IOS系统深度开发的系统)或其它系统。存储数据区还可以存储终端在使用中所创建的数据(比如电话本、音视频数据、聊天记录数据)等。
除此之外,本领域技术人员可以理解,上述附图所示出的计算设备1100的结构并不构成对计算设备1100的限定,计算设备1100可以包括比图示更多或更少的部件,或者组合某些部件,或者不同的部件布置。比如,终端中还包括射频电路、拍摄组件、传感器、音频电路、无线保真(wireless fidelity,WiFi)组件、电源、蓝牙组件等部件,在此不再赘述。
本申请实施例还提供了一种计算机可读存储介质,该计算机可读存储介质中存储有至少一条计算机指令,该至少一条计算机指令由处理器加载并执行以实现如上各个实施例所述的数据同步方法。关于上述提供的任一种计算机可读存储介质中相关内容的解释及有益效果的描述,均可以参考上述对应的实施例,此处不再赘述。
本申请实施例还提供了一种芯片。该芯片中集成了用于实现上述数据同步装置的功能的控制电路和一个或者多个端口。可选的,该芯片支持的功能可以参考上文,此处不再赘述。本领域普通技术人员可以理解实现上述实施例的全部或部分步骤可通过程序来指令相关的硬件完成。所述的程序可以存储于一种计算机可读存储介质中。上述提到的存储介质可以是只读存储器,随机接入存储器等。上述处理单元或处理器可以是中央处理器,通用处理器、特定集成电路(application specific integrated circuit,ASIC)、微处理器(digital signal processor,DSP),现场可编程门阵列(field programmable gate array,FPGA)或者其他可编程逻辑器件、晶体管逻辑器件、硬件部件或者其任意组合。
本申请实施例还提供了一种包含指令的计算机程序产品,当该指令在计算机上运行时,使得计算机执行上述实施例中的任意一种方法。该计算机程序产品包括一个或多个计算机指令。在计算机上加载和执行计算机程序指令时,全部或部分地产生按照本申请实施例的流程或功能。计算机可以是通用计算机、专用计算机、计算机网络、或者其他可编程装置。计算机指令可以存储在计算机可读存储介质中,或者从一个计算机可读存储介质向另一个计算机可读存储介质传输,例如,计算机指令可以从一个网站站点、计算机、服务器或者数据中心 通过有线(例如同轴电缆、光纤、数字用户线(digital subscriber line,DSL))或无线(例如红外、无线、微波等)方式向另一个网站站点、计算机、服务器或数据中心进行传输。计算机可读存储介质可以是计算机能够存取的任何可用介质或者是包含一个或多个可以用介质集成的服务器、数据中心等数据存储设备。可用介质可以是磁性介质(例如,软盘、硬盘、磁带),光介质(例如,DVD)、或者半导体介质(例如SSD)等。
应注意,本申请实施例提供的上述用于存储计算机指令或者计算机程序的器件,例如但不限于,上述存储器、计算机可读存储介质和通信芯片等,均具有非易失性(non-transitory)。本领域技术人员应该可以意识到,在上述一个或多个示例中,本申请实施例所描述的功能可以用硬件、软件、固件或它们的任意组合来实现。当使用软件实现时,可以将这些功能存储在计算机可读存储介质中或者作为计算机可读存储介质上的一个或多个指令或代码进行传输。计算机可读存储介质包括计算机存储介质和通信介质,其中通信介质包括便于从一个地方向另一个地方传送计算机程序的任何介质。存储介质可以是通用或专用计算机能够存取的任何可用介质。
以上所述仅为本申请的可选实施例,并不用以限制本申请,凡在本申请的精神和原则之内,所作的任何修改、等同替换、改进等,均应包含在本申请的保护范围之内。

Claims (10)

  1. 一种数据同步方法,其特征在于,应用于第一存储设备;所述方法包括:
    在所述第一存储设备的负载小于或等于指定阈值的情况下,获取差异数据;所述差异数据包括第一时间段内,针对所述第一存储设备执行输入输出操作的业务数据,所述第一时间段是上一次开始向第二存储设备同步数据的时刻至当前时刻之间的时间段;所述第一存储设备是业务数据生产端的存储设备,所述第二存储设备是业务数据容灾端的存储设备;
    将所述差异数据同步到所述第二存储设备中。
  2. 根据权利要求1所述的方法,其特征在于,所述在所述第一存储设备的负载小于或等于指定阈值的情况下,获取差异数据,包括:
    查询数据同步标志的属性状态,所述数据同步标志存储在所述第一存储设备中,用于表征所述第一存储设备当前的所述负载是否小于或等于所述指定阈值;
    在查询到的所述属性状态为第一状态的情况下,获取差异数据,所述第一状态用于指示所述第一存储设备当前的所述负载小于或等于所述指定阈值。
  3. 根据权利要求2所述的方法,其特征在于,所述方法还包括:
    在查询到的所述属性状态为第二状态的情况下,依次在后续的检测时刻下查询所述数据同步标志的属性状态,直到达到第一检测时刻时停止查询所述属性状态,所述检测时刻是所述第一存储设备检测所述第一存储设备的所述负载是否小于或等于所述指定阈值的时刻,所述第一检测时刻是按照时间顺序在后续查询到的第一个所述属性状态是所述第一状态的检测时刻,所述第二状态用于指示所述第一存储设备当前的负载大于所述指定阈值;
    在所述第一检测时刻获取所述差异数据。
  4. 根据权利要求3所述的方法,其特征在于,所述方法还包括:
    在查询到的所述属性状态为所述第二状态,且依次在后续的检测时刻下查询所述数据同步标志的属性状态的过程中未查询到所述属性状态是所述第一状态时,当前时刻与上一次的同步启动时刻之间的时间间隔达到预设时间段的情况下,获取所述差异数据,所述同步启动时刻是开始向所述第二存储设备同步数据的时刻。
  5. 根据权利要求2至4任一所述的方法,其特征在于,所述查询数据同步标志的属性状态,包括:
    在接收到所述第二存储设备返回的同步完成响应的第一时刻,查询所述数据同步标志的属性状态,所述同步完成响应用于指示所述第一存储设备上一次向所述第二存储设备同步的业务数据已经同步完成;
    所述在查询到的所述属性状态为第一状态的情况下,获取差异数据,包括:
    在查询到所述第一时刻的所述属性状态为所述第一状态的情况下,获取差异数据。
  6. 根据权利要求1至5任一所述的方法,其特征在于,所述在所述第一存储设备的负载小于或等于指定阈值的情况下,获取差异数据,包括:
    在确定所述第一存储设备的所述负载小于所述指定阈值的时刻,对所述第一存储设备中的业务数据生成第一快照;
    基于所述第一快照,以及上一次开始向所述第二存储设备同步数据的时刻到当前时刻之间的数据变换日志,获取所述差异数据,所述数据变换日志是所述第一存储设备执行输入输出操作时记录的日志。
  7. 根据权利要求1至6任一所述的方法,其特征在于,所述负载通过以下参数中的至少一种来表征:所述第一存储设备的处理器CPU的占用情况、内存占用情况、业务数据输入输 出IO接口的带宽情况。
  8. 根据权利要求1至7任一所述的方法,其特征在于,所述在所述第一存储设备的负载小于或等于指定阈值的情况下,获取差异数据,包括:
    当通过至少两种参数表征所述第一存储设备的负载时,在所述至少两种参数均不超过所述至少两种参数各自的所述指定阈值的情况下,获取所述差异数据。
  9. 根据权利要求1至8任一所述的方法,其特征在于,所述将所述差异数据同步到所述第二存储设备中,包括:
    在所述差异数据的数据量大于指定数据量的情况下,将所述差异数据同步到所述第二存储设备中。
  10. 一种计算设备,其特征在于,所述计算设备包括处理器和存储器;所述处理器与所述存储器耦合;所述存储器用于存储计算机指令,所述计算机指令由所述处理器加载并执行以使计算设备实现如权利要求1至9任一所述的数据同步方法。
PCT/CN2023/118095 2022-10-24 2023-09-11 数据同步方法及计算设备 WO2024087914A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202211302779.3 2022-10-24
CN202211302779.3A CN115827774A (zh) 2022-10-24 2022-10-24 数据同步方法及计算设备

Publications (1)

Publication Number Publication Date
WO2024087914A1 true WO2024087914A1 (zh) 2024-05-02

Family

ID=85525299

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/118095 WO2024087914A1 (zh) 2022-10-24 2023-09-11 数据同步方法及计算设备

Country Status (2)

Country Link
CN (1) CN115827774A (zh)
WO (1) WO2024087914A1 (zh)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115827774A (zh) * 2022-10-24 2023-03-21 超聚变数字技术有限公司 数据同步方法及计算设备

Citations (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104375904A (zh) * 2014-10-30 2015-02-25 浪潮电子信息产业股份有限公司 一种基于快照差异化数据传输的容灾备份方法
RU2646309C1 (ru) * 2017-04-17 2018-03-02 Алексей Николаевич Бегаев Способ резервного копирования
CN107851052A (zh) * 2016-07-12 2018-03-27 华为技术有限公司 一种用于备份计算机网络中计算设备的数据的装置和方法
CN110780040A (zh) * 2019-10-29 2020-02-11 腾讯科技(深圳)有限公司 有害气体数据处理方法、装置、电子设备及存储介质
CN110879825A (zh) * 2018-09-06 2020-03-13 阿里巴巴集团控股有限公司 一种数据同步方法及装置
CN111881083A (zh) * 2020-07-24 2020-11-03 北京浪潮数据技术有限公司 一种文件同步方法、装置、设备及介质
US20210176314A1 (en) * 2018-08-24 2021-06-10 SZ DJI Technology Co., Ltd. Method and system for synchronizing data, movable platform, and readable-storage medium
CN113268395A (zh) * 2021-04-30 2021-08-17 厦门科灿信息技术有限公司 业务数据的处理方法、处理装置及终端
CN113535483A (zh) * 2021-09-15 2021-10-22 统信软件技术有限公司 一种文件备份方法、装置及计算设备
CN113722152A (zh) * 2021-08-05 2021-11-30 四川大学 一种数据增量备份方法、装置、设备及存储介质
CN113987065A (zh) * 2021-09-28 2022-01-28 携程科技(上海)有限公司 数据库漂移方法、系统、电子设备和存储介质
CN114356650A (zh) * 2021-11-19 2022-04-15 科大讯飞股份有限公司 数据备份方法、装置、设备、系统及存储介质
CN115033578A (zh) * 2021-03-03 2022-09-09 腾讯科技(深圳)有限公司 一种业务数据更新的方法、相关装置及存储介质
CN115827774A (zh) * 2022-10-24 2023-03-21 超聚变数字技术有限公司 数据同步方法及计算设备

Patent Citations (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104375904A (zh) * 2014-10-30 2015-02-25 浪潮电子信息产业股份有限公司 一种基于快照差异化数据传输的容灾备份方法
CN107851052A (zh) * 2016-07-12 2018-03-27 华为技术有限公司 一种用于备份计算机网络中计算设备的数据的装置和方法
RU2646309C1 (ru) * 2017-04-17 2018-03-02 Алексей Николаевич Бегаев Способ резервного копирования
US20210176314A1 (en) * 2018-08-24 2021-06-10 SZ DJI Technology Co., Ltd. Method and system for synchronizing data, movable platform, and readable-storage medium
CN110879825A (zh) * 2018-09-06 2020-03-13 阿里巴巴集团控股有限公司 一种数据同步方法及装置
CN110780040A (zh) * 2019-10-29 2020-02-11 腾讯科技(深圳)有限公司 有害气体数据处理方法、装置、电子设备及存储介质
CN111881083A (zh) * 2020-07-24 2020-11-03 北京浪潮数据技术有限公司 一种文件同步方法、装置、设备及介质
CN115033578A (zh) * 2021-03-03 2022-09-09 腾讯科技(深圳)有限公司 一种业务数据更新的方法、相关装置及存储介质
CN113268395A (zh) * 2021-04-30 2021-08-17 厦门科灿信息技术有限公司 业务数据的处理方法、处理装置及终端
CN113722152A (zh) * 2021-08-05 2021-11-30 四川大学 一种数据增量备份方法、装置、设备及存储介质
CN113535483A (zh) * 2021-09-15 2021-10-22 统信软件技术有限公司 一种文件备份方法、装置及计算设备
CN113987065A (zh) * 2021-09-28 2022-01-28 携程科技(上海)有限公司 数据库漂移方法、系统、电子设备和存储介质
CN114356650A (zh) * 2021-11-19 2022-04-15 科大讯飞股份有限公司 数据备份方法、装置、设备、系统及存储介质
CN115827774A (zh) * 2022-10-24 2023-03-21 超聚变数字技术有限公司 数据同步方法及计算设备

Also Published As

Publication number Publication date
CN115827774A (zh) 2023-03-21

Similar Documents

Publication Publication Date Title
US8930309B2 (en) Interval-controlled replication
WO2017124938A1 (zh) 一种数据同步方法、装置及系统
WO2022063284A1 (zh) 数据同步方法、装置、设备及计算机可读介质
WO2017177941A1 (zh) 主备数据库切换方法和装置
WO2024087914A1 (zh) 数据同步方法及计算设备
US11429594B2 (en) Synchronization between primary database and secondary database
EP3319258B1 (en) Service take-over method and storage device, and service take-over apparatus
US20160196226A1 (en) Method and Apparatuses for Monitoring System Bus
US11860849B2 (en) Watermark-based techniques for change-data-capture
WO2022142665A1 (zh) 基于数据库集群的数据处理方法、装置及电子设备
US11934665B2 (en) Systems and methods for ephemeral storage snapshotting
US11256640B2 (en) File transfer method, apparatus, device between BMCs and storage medium
WO2014094303A1 (zh) 一种监控记录管理方法与装置
WO2020133473A1 (zh) 一种备份数据的方法、装置和系统
CN113326251B (zh) 数据管理方法、系统、设备和存储介质
US10140183B2 (en) Efficient state tracking for clusters
WO2015074260A1 (zh) 数据同步方法和数据同步系统
US11995436B2 (en) Out-of-band updating method and system of expander
CN112650629A (zh) 区块链索引数据恢复方法、装置、设备和计算机存储介质
JP6788188B2 (ja) 制御装置および制御プログラム
CN117395263B (zh) 一种数据同步方法、装置、设备和存储介质
CN115794939A (zh) 一种异步复制方法及装置
CN117743303A (zh) 一种数据库扩容方法、装置及电子设备
CN117389772A (zh) 数据库系统的故障处理方法及计算设备
CN117389771A (zh) 数据库系统的故障处理方法及计算设备