WO2025001030A1 - 基于多目标最优副本管理策略的混合云数据备份管理方法 - Google Patents

基于多目标最优副本管理策略的混合云数据备份管理方法 Download PDF

Info

Publication number
WO2025001030A1
WO2025001030A1 PCT/CN2023/143183 CN2023143183W WO2025001030A1 WO 2025001030 A1 WO2025001030 A1 WO 2025001030A1 CN 2023143183 W CN2023143183 W CN 2023143183W WO 2025001030 A1 WO2025001030 A1 WO 2025001030A1
Authority
WO
WIPO (PCT)
Prior art keywords
backup
objective function
objective
optimal
data
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
PCT/CN2023/143183
Other languages
English (en)
French (fr)
Inventor
雷光钰
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
China Telecom Cloud Technology Co Ltd
Original Assignee
China Telecom Cloud Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by China Telecom Cloud Technology Co Ltd filed Critical China Telecom Cloud Technology Co Ltd
Publication of WO2025001030A1 publication Critical patent/WO2025001030A1/zh
Anticipated expiration legal-status Critical
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/14—Error detection or correction of the data by redundancy in operations
    • G06F11/1446—Point-in-time backing up or restoration of persistent data
    • G06F11/1458—Management of the backup or restore process
    • G06F11/1464—Management of the backup or restore process for networked environments
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/14—Error detection or correction of the data by redundancy in operations
    • G06F11/1446—Point-in-time backing up or restoration of persistent data
    • G06F11/1448—Management of the data involved in backup or backup restore
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/14—Error detection or correction of the data by redundancy in operations
    • G06F11/1446—Point-in-time backing up or restoration of persistent data
    • G06F11/1458—Management of the backup or restore process
    • G06F11/1461—Backup scheduling policy
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/14—Error detection or correction of the data by redundancy in operations
    • G06F11/1446—Point-in-time backing up or restoration of persistent data
    • G06F11/1458—Management of the backup or restore process
    • G06F11/1469—Backup restoration techniques

Definitions

  • the present invention belongs to the field of cloud computing disaster recovery, and in particular relates to a hybrid cloud data backup management method based on a multi-objective optimal copy management strategy.
  • Cloud data backup and recovery is of great significance in the process of government/enterprise cloud migration. It involves backing up data from the local computer room to the cloud, or backing up data from the cloud to the cloud, so that data can be restored when it is lost, damaged or attacked. It is of great significance to ensure data security, improve data accessibility, reduce costs and simplify data management.
  • the backup and recovery of massive data makes traditional data protection methods appear to have high maintenance costs and long recovery time, and reduce data security.
  • the copy data management strategy is an efficient backup data protection solution on the cloud and an important part of protecting key information for governments, enterprises and individuals.
  • An excellent copy backup management strategy can cope with data loss caused by failures or natural disasters, protect business continuity, respond to network security threats, and improve data recovery speed.
  • Data backup management is the key to ensuring data security and business continuity. By implementing an effective data backup strategy, the risk of data loss can be reduced, the interests of governments, enterprises and customers can be protected, and it has a significant impact on the data stability, reliability and response speed of the entire management system.
  • the reliability of backup data is crucial. Reliability means that the backup data should be reliable and can be restored in case of any problems. In addition, during the backup and recovery process, it is also necessary to ensure that the backup data can be restored by repairing the data file when an error occurs.
  • the latency of the backup system is one of its important characteristics.
  • Cloud storage systems often store multiple copies of files and place them on different nodes to ensure the security of the stored information and reduce the latency of user access.
  • a cloud storage system when a file is accessed less frequently and there are too many copies of the file, it will increase the system's unnecessary storage overhead and storage cost; When the access volume is high and there are few copies of the file, the access latency for users to access the file will increase, which cannot meet user needs.
  • the load balancing of the backup system is also an important factor to consider.
  • the load balancing rate of the server not only affects the resource utilization of the server, but also has a great impact on the entire system.
  • the purpose of the present invention is to provide a hybrid cloud data backup management method based on a multi-objective optimal copy management strategy to make up for the shortcomings of the above-mentioned technical solutions.
  • backup and recovery management system backup and recovery of different types of data can be achieved through one platform.
  • the reliability of the backup data, the overall system latency and load balancing issues are considered at the same time, a multi-objective optimization model is established, and the optimal copy management strategy of the Pareto equilibrium of the three problems is obtained, and the backup copy is backed up to the underlying storage space of the system based on the optimal strategy. It is achieved that while simplifying user operations and improving business efficiency, the reliability of backup data and the stability of the backup system are improved, the latency of the backup system is reduced, and the stable operation of the business system is ensured.
  • the present invention provides a hybrid cloud data backup management method based on a multi-objective optimal copy management strategy, comprising the following steps:
  • the reliability objective function, transmission delay objective function, and load balancing objective function of complete data backup in several storage nodes are defined respectively;
  • the data is allocated for backup placement on each storage node based on the optimal placement solution matrix.
  • the reliability objective function is as follows:
  • ⁇ j is the failure rate of the jth storage node t j in the storage cluster T
  • u is the usage time of the storage node
  • tj is the failure rate function of storage node tj
  • K represents the data file
  • m is the number of storage nodes
  • T represents the cluster composed of m storage nodes
  • tkij represents the replica on node tj .
  • the transmission delay objective function is as follows:
  • vj is the data sending rate of backup node tj
  • ci is the size of data file ki
  • the load balancing objective function is as follows:
  • ⁇ j is the weight of each backup node t j .
  • optimization objective function is as follows:
  • the process of using a decomposition-based multi-objective evolutionary algorithm to obtain an optimal placement solution matrix that satisfies the optimization objective function includes:
  • the maximum number of iterations is set and it is iterated.
  • the last weight vector and the corresponding placement solution matrix are iteratively updated based on the number of iterations. When the maximum number of iterations is reached, the iteration is stopped to obtain the optimal placement solution matrix.
  • the process of updating each weight vector and the corresponding placement scheme matrix based on the number of iterations includes:
  • an updated non-dominated solution set is obtained and used as an optimal frontier solution set
  • a trade-off solution is selected from the optimal frontier solution set, and the trade-off solution is used as an optimal placement solution matrix.
  • ⁇ represents the neighborhood individual The weight vector of .
  • the dominated solution in INL(h-1) is retained.
  • the solution is to discard all the Dominated solution, if all solutions of INL(h-1) do not dominate Will Add INL(h), where INL(h-1) represents the non-dominated solution set after the last update. represents the new placement solution matrix, and INL(h) represents the current non-dominated solution set.
  • the present invention provides a hybrid cloud data backup management system based on a multi-objective optimal copy management strategy, characterized in that the system includes a memory, a processor, and a computer program stored in the memory and capable of running on the processor, and when the processor executes the computer program, it implements the above-mentioned hybrid cloud data backup management method based on the multi-objective optimal copy management strategy.
  • the beneficial effect of the present invention is that the hybrid cloud data backup and recovery management method based on the multi-objective optimal copy management strategy of the present invention comprehensively considers the reliability of backup data, the latency of the backup system and the stability of the backup system, and obtains the optimal placement plan of the backup copy of the golden copy through the multi-objective optimization algorithm based on decomposition, thereby reducing the risk of data loss and damage.
  • users can set the backup and recovery strategy, and can flexibly and quickly perform data backup and recovery, helping administrators to quickly locate faults and maximize the efficiency and performance of the system.
  • FIG1 is a flow chart of a multi-objective optimal replica management method according to an embodiment of the present invention.
  • FIG. 2 is a schematic diagram of the design principle of a hybrid cloud data backup and recovery management system based on a multi-objective optimal copy management strategy in an embodiment of the present invention
  • FIG. 3 is an architecture diagram of a hybrid cloud data backup and recovery management system based on a multi-objective optimal copy management strategy in an embodiment of the present invention
  • FIG. 4 is a multi-target backup copy management strategy based on the CDM golden copy concept in an embodiment of the present invention. Design schematics;
  • FIG5 is a roadmap of a multi-target optimal copy management method in an embodiment of the present invention.
  • this embodiment provides a hybrid cloud data backup management method and system based on a multi-objective optimal copy management strategy, wherein the method is shown in FIG. 1 and includes:
  • S200 Acquire an optimization objective function for data backup management based on the reliability objective function, the transmission delay objective function, and the load balancing objective function;
  • S400 Allocate backup placement of data on each storage node based on the optimal placement solution matrix.
  • Step 1 User and permission management settings. Implement user authentication and permission management functions, and perform full life cycle management of the backup and recovery system.
  • Step 1.1 User authentication function implementation. When a user logs in, the user's account name and password need to be compared with the account name and password saved in the system. If the two match, it means that the user has passed the authentication. Otherwise, the user's access is denied and re-authentication is required.
  • Step 1.2 Implement the permission management function.
  • Set role permissions for different users such as ordinary users, Operation and maintenance administrators, etc., set permissions for different user roles.
  • Step 2 Backup strategy management: Set the backup method and backup strategy to schedule the operation of various backup tasks.
  • Step 2.1 Backup mode setting.
  • the present invention supports two backup modes: real-time backup and scheduled backup. Users can select the corresponding backup mode according to different needs.
  • Step 2.2 Backup strategy setting.
  • the present invention supports three backup strategies: full backup, incremental backup, and differential backup. When the user backs up for the first time, a full backup is performed by default. For subsequent backup operations, the user can select a backup strategy according to needs.
  • Step 3 Data backup. Start data backup, establish an encrypted connection between the data node and the backup node and transfer data, and perform data backup according to the backup method applied. If the user chooses to back up to a third-party backup software, go to step 3.1; if the user chooses to back up to the system's underlying storage space, go to step 3.2.
  • Step 3.1 The backup and recovery management system is connected to the third-party backup software and directly backs up to the storage space of the third-party backup software according to the backup policy set by the user.
  • Step 3.2 The system's underlying storage space adopts a multi-target optimal backup copy management solution. Based on the CDM golden copy concept, the steps and principles of designing a multi-target backup copy management strategy are shown in Figures 3 and 4 respectively:
  • T (t 1 , t 2 , ..., t m )
  • K (k 1 , k 2 , ..., k n )
  • the same replica cannot be placed on the same storage node.
  • the placement scheme of data replicas in the backup node is represented by the matrix TK:
  • Step 3.2.1 As shown in Figure 5, set the corresponding objective optimization function from the three aspects of backup system reliability, latency and load balancing:
  • Step 3.2.1.1 Reliability objective function design.
  • the failure rate of the j-th storage node tj in the storage cluster T be ⁇ j .
  • the failure rate of the storage node is related to its usage time u (the unit of u is hours).
  • the failure rate function of the storage node tj is:
  • Data file K, K (k 1 , k 2 , ..., k n ), for each data file k i , if there is at least one complete backup copy in the m storage nodes, then the i-th data file k i is considered reliable.
  • the reliability of k i is defined as:
  • the reliability objective function of the entire system for a complete backup of data file K is defined as:
  • Step 3.2.1.2 Design of latency objective function.
  • the total network access latency of the j-th storage node tj in the storage cluster T be Tim_delj (including processing latency, queuing latency, sending latency, and propagation latency).
  • the sending latency of the a-th access to the backup copy of k i on the storage node tj is:
  • vj is the data transmission rate of backup node tj ;
  • ci is the size of data file ki .
  • the access to the copies of ki on different backup nodes is independent of each other, so the total access delay function of ki is defined as:
  • Step 3.2.1.3 Load balancing objective function design. Using weighted round-robin method, each backup storage node Point t j is assigned a corresponding weight ⁇ j according to its hardware carrying capacity, and ⁇ j satisfies
  • the present invention uses the absolute value of the difference between the actual load of all storage nodes tj and the theoretical load calculated based on the hardware carrying capacity as the standard for measuring system load balancing. The smaller the standard deviation, the more balanced the load. Define the load balancing objective function:
  • Step 3.2.2 Define the multi-objective copy management strategy optimization objective function.
  • the optimization objective function :
  • X is called the decision variable, and here it is the placement plan matrix TK.
  • the optimal placement plan matrix can be searched through multi-objective optimization.
  • f 1 (X) s 1 (TK)
  • ⁇ (k i ) represents the size of the i-th data file k i
  • ⁇ (t j ) represents the capacity of the j-th storage node t j .
  • the calculation units of ⁇ (k i ) and ⁇ (t j ) must be consistent.
  • ⁇ j represents the weight assigned to the storage node according to its hardware carrying capacity.
  • Step 3.2.3 uses a decomposition-based multi-objective evolutionary algorithm to solve the above multi-objective problem.
  • the specific steps are:
  • Step 3.2.3.2 Use the Tchebycheff decomposition algorithm to decompose the multi-objective problem consisting of l objectives into P sub-problems, each of which is:
  • z * is the reference point corresponding to the multi-objective function. Is it that the child asks The minimum value for which problem f k (X) is true, representing the reference point for subproblems f k (X).
  • X p (0) represents the pth placement solution matrix at the time of initialization
  • the feasible domain is the range of values in the placement solution matrix that meets the constraints.
  • Step 3.2.3.5 Initialize the neighborhood vector and find the distance p-th weight vector
  • ⁇ is the evolution parameter.
  • Step 3.2.3.6.3 Update the population: For each neighborhood individual in B(p) b ⁇ B(p), if but Where gte represents the subproblem obtained based on the Tchebycheff decomposition algorithm, and ⁇ represents the neighborhood individual The weight vector of .
  • Step 3.2.3.6.4 Update the non-dominated solution set INL(h): In INL(h), keep the dominated solutions in INL(h-1) The solution is to discard all the INL(h-1) Dominated solution, if all solutions of INL(h-1) do not dominate Will Add INL(h).
  • Step 3.2.3.6.6 Select a trade-off solution TK optimal from the Pareto optimal solution set based on the decision maker's decision preference or randomly.
  • TK optimal is the Pareto optimal placement matrix that meets the three objectives of reliability, latency, and load balancing. Complete the placement allocation of backup copies on each node based on TK optimal .
  • Step 3.2.4 After the backup is completed, the backup system notifies the management node that the data backup is complete.
  • the management node generates a backup log based on the backup information and archives it, and notifies the user that the backup is complete.
  • Step 4.1 The backup node transmits the data to the recovery node. After the sender and the receiver establish a connection, the sender first encrypts the data. After receiving the data, the receiver decrypts it and performs a checksum test on the decrypted data. If the checksum test passes, it proves that the data transmission is correct and the data can be written to the disk. If erroneous data occurs during the transmission process, the receiver can ask the sender to resend the erroneous data.
  • Step 4.2 After data recovery is completed, the recovery node notifies the management node that recovery is complete.
  • the management node generates a recovery log based on the data recovery information and archives it, and notifies the user that data recovery is complete.
  • Step 5 Log management. Create a log file for each backup task process and recovery task process to track the task process information: record the periodic backup and recovery time, task error information, etc., to facilitate the search of task records and error location.
  • Step 6 Monitoring and management.
  • the backup and recovery management system provides a unified monitoring screen to monitor the entire backup and recovery process. Users can view the data backup and recovery status in real time and monitor the available capacity of the system's underlying storage space and the status of each storage node.
  • the present invention abstracts the reliability, latency and load balancing problems of the backup system into a multi-objective optimization problem.
  • the performance of the three aspects of the system is optimized simultaneously in the form of vectors.
  • the reliability of the system is improved and the system latency is minimized.
  • the Pareto optimal solution obtained enables the system to achieve optimal performance under multiple objectives, thereby improving the performance of the system in multiple aspects.
  • the present invention has the following advantages and effects:
  • This patent invention designs a hybrid cloud data backup and recovery management system that integrates different types of data and backup space on a system platform. Users can operate through the management interface on the platform and back up different types of data on the local/cloud to third-party storage software or system underlying storage space according to their needs, simplifying user operations and improving business efficiency.
  • the hybrid cloud data backup and recovery management system based on a multi-objective optimal copy management strategy invented by this patent comprehensively considers the reliability of backup data, the latency of the backup system, and the stability of the backup system. Through a decomposition-based multi-objective optimization algorithm, it obtains the optimal placement plan for the backup copies of the golden copy, thereby reducing the risk of data loss and damage.
  • the hybrid cloud data backup and recovery management system based on the multi-objective optimal copy management strategy invented by this patent is managed with a graphical interface, records log information in real time, and provides a unified monitoring platform. Users can set backup and recovery strategies, and can flexibly and quickly perform data backup and recovery, helping administrators to quickly locate faults and maximize system efficiency and performance.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Quality & Reliability (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

本发明提供了一种基于多目标最优副本管理策略的数据备份管理方法与系统，包括：分别对数据进行完整备份的可靠性目标函数、发送时延目标函数、负载均衡目标函数进行定义；基于可靠性目标函数、发送时延目标函数、负载均衡目标函数获取数据备份管理的优化目标函数；采用基于分解的多目标进化算法获取满足优化目标函数的最优放置方案矩阵，基于最优放置方案矩阵对数据进行各个节点上的备份放置分配。本发明的综合考虑备份数据的可靠性、备份系统的时延和备份系统的稳定性，通过基于分解的多目标优化算法，得到黄金副本的备份副本最优放置方案，降低数据丢失和损坏的风险。

Description

基于多目标最优副本管理策略的混合云数据备份管理方法 技术领域
本发明属于云计算灾备领域，特别是涉及一种基于多目标最优副本管理策略的混合云数据备份管理方法。
背景技术
随着大数据时代的发展，数据的重要性上升到关键资源的层次。云数据备份恢复在政府/企业上云过程中具有重要意义。它涉及将数据从本地机房备份到云端，或者将数据从云上备份到云端，以便在数据丢失、损坏或遭受攻击时进行恢复。对于确保数据安全、提高数据可访问性、降低成本、简化数据管理具有重要意义。但是海量数据的备份和恢复让传统的数据保护方法显得维护成本高和恢复时间长，数据安全性降低。
数据副本管理策略(CDM)是云上一种高效的备份数据保护解决方案，是政府、企业和个人保护关键信息的重要组成部分。一个优秀的备份副本管理策略能够应对故障或自然灾害导致的数据丢失、保护业务连续性、应对网络安全威胁、提高数据恢复速度等。数据备份管理是确保数据安全和业务连续性的关键。通过实施有效的数据备份策略，可以降低数据丢失的风险，保护政府、企业和客户的利益，对于整个管理系统的数据稳定性、可靠性和相应速度等具有重大的影响。
在副本管理策略中，备份数据的可靠性是至关重要的。可靠性是指备份的数据应该是可靠的，能够在发生任何问题时都能被恢复。此外，在备份和恢复的过程中，还需要确保备份数据能够在出现错误时通过修复数据文件而得到恢复。
在副本管理策略中，备份系统的时延是其重要特性之一。云存储系统往往将文件以副本的形式存储多份并放在不同的节点上，以保证其存储信息的安全性和降低用户访问的延迟。在云存储系统中，当一个文件访问量较少，且该文件的副本数目过多时会增加系统不必要存储开销和存储成本；而当一个文件的 访问量很高，且该文件的副本较少时，则会增加用户访问该文件的访问时延，无法满足用户需求。
在副本管理策略中，备份系统的负载均衡也是需要考虑的重要因素。服务器的负载均衡率不仅影响了服务器的资源利用率，并对整个系统都有较大的影响。
然而，在实际的副本管理策略中，备份系统在三个方面的追求可能是潜在冲突的，无法同时达到最优。
发明内容
本发明的目的是提供一种基于多目标最优副本管理策略的混合云数据备份管理方法，用于弥补上述技术方案的不足。在备份恢复管理系统上，能够通过一个平台实现对不同类型数据的备份与恢复。在备份模块中，同时考虑备份数据的可靠性、系统整体时延和负载均衡问题，建立多目标优化模型，获取三种问题Pareto平衡的最优副本管理策略，基于最优策略将备份副本备份到系统底层存储空间中。实现了在简化用户操作提高业务效率的同时，提高备份数据的可靠性和备份系统的稳定性，降低备份系统时延，保证业务系统运行稳定。
一方面为实现上述目的，本发明提供了一种基于多目标最优副本管理策略的混合云数据备份管理方法，包括以下步骤：
分别对数据在若干个存储节点中进行完整备份的可靠性目标函数、发送时延目标函数、负载均衡目标函数进行定义；
基于所述可靠性目标函数、发送时延目标函数、负载均衡目标函数获取数据备份管理的优化目标函数；
采用基于分解的多目标进化算法获取满足所述优化目标函数的最优放置方案矩阵；
基于所述最优放置方案矩阵对数据进行各个存储节点上的备份放置分配。
可选地，所述可靠性目标函数如下所示：
其中，表示第i个数据文件ki的可靠性，χj为存储集群T中第j个存储节点tj的故障率，u存储节点的使用时间，为存储节点tj的失效率函数，K表示数据文件，m为存储节点数量，T表示m个存储节点组成的集群，tkij表示节点tj上的副本。
可选地，所述发送时延目标函数如下所示：
其中，vj为备份节点tj的数据发送速率；ci为数据文件ki的大小，表示ki在节点tj上的总访问量；表示ki的备份副本在存储节点tj上的第a个访问的发送时延。
可选地，所述负载均衡目标函数如下所示：
其中，σj为每个备份节点tj的权重，且
可选地，所述优化目标函数如下所示：
其中，X为决策变量，此处为放置方案矩阵TK，f1(X)＝s1(TK)， ρ(ki)表示第i个数据文件ki的大小，Γ(tj)表示第j个存储节点tj的容量大小。
可选地，采用基于分解的多目标进化算法获取满足所述优化目标函数的最优放置方案矩阵的过程包括：
初始化种群数量并随机生成与种群数量相同的权重向量；
基于Tchebycheff分解算法将所述优化目标函数进行分解为与种群数量相同的若干个子问题；
初始化进化次数，随机生成种群数量相同的放置方案矩阵；
初始化前沿解集以及优化目标函数对应的参考点；
初始化邻域向量，获取距离最后一个权重向量最近的若干个权重向量，并基于若干个权重向量构建索引集；
设置最大迭代次数并进行迭代，基于迭代次数对最后一个权重向量和对应的放置方案矩阵进行迭代更新，当达到最大迭代次数时停止迭代，获取所述最优放置方案矩阵。
可选地，基于迭代次数对每个权重向量和对应的放置方案矩阵进行更新的过程包括：
基于所述索引集随机获取索引号，获取所述索引号对应的放置方案矩阵；
基于对应的放置方案矩阵获取新放置方案矩阵，基于所述新放置方案矩阵分别对参考点、种群以及非支配解集进行迭代更新；
当达到最大迭代次数时，获取更新后的非支配解集并作为最优前沿解集，在所述最优前沿解集中选择一个权衡解，将所述权衡解作为最优放置方案矩阵。
可选地，所述参考点的更新过程中，则表示子问题fk的参考点，表示新放置方案矩阵；
所述种群的更新过程中，对于索引集中的每一个邻域个体若则其中gte表示基于 Tchebycheff分解算法得到的子问题，λ表示邻域个体的权重向量。
可选地，所述非支配解集的更新过程中，保留INL(h-1)中支配的解，舍弃INL(h-1)中所有被支配的解，若所有INL(h-1)的解都不支配将加入INL(h)，其中，INL(h-1)表示上一次更新后的非支配解集，表示新放置方案矩阵，INL(h)表示当前的非支配解集。
另一方面为实现上述目的，本发明提供了一种基于多目标最优副本管理策略的混合云数据备份管理系统，其特征在于，所述系统包括存储器、处理器及存储在存储器上并能够在处理器上运行的计算机程序，所述处理器执行所述计算机程序时实现上述基于多目标最优副本管理策略的混合云数据备份管理方法。
与现有技术相比，本发明的有益效果在于，本发明的基于多目标最优副本管理策略的混合云数据备份恢复管理方法，综合考虑备份数据的可靠性、备份系统的时延和备份系统的稳定性，通过基于分解的多目标优化算法，得到黄金副本的备份副本最优放置方案，降低数据丢失和损坏的风险。用户通过本发明可以对备份恢复策略进行设置，能够灵活、快速的进行数据备份与恢复，帮助管理员快速定位故障，最大程度地提高系统的效率和性能。
附图说明
通过阅读下文优选实施方式的详细描述，各种其他的优点和益处对于本领域普通技术人员将变得清楚明了。附图仅用于示出优选实施方式的目的，而并不认为是对本发明的限制。而且在整个附图中，用相同的参考符号表示相同的部件。在附图中：
图1为本发明实施例中的多目标最优副本管理方法流程图；
图2为本发明实施例中的基于多目标最优副本管理策略的混合云数据备份恢复管理系统设计原理示意图；
图3为本发明实施例中的基于多目标最优副本管理策略的混合云数据备份恢复管理系统架构图；
图4为本发明实施例中基于CDM黄金副本思想的多目标备份副本管理策略 设计示意图；
图5为本发明实施例中的多目标最优副本管理方法路线图。
具体实施方式
下面将参照附图更详细地描述本公开的示例性实施例。虽然附图中显示了本公开的示例性实施例，然而应当理解，可以以各种形式实现本公开而不应被这里阐述的实施例所限制。相反，提供这些实施例是为了能够更透彻地理解本公开，并且能够将本公开的范围完整的传达给本领域的技术人员。需要说明的是，在不冲突的情况下，本发明中的实施例及实施例中的特征可以相互组合。下面将参考附图并结合实施例来详细说明本发明。
如图1-5所示，本实施例中提供一种基于多目标最优副本管理策略的混合云数据备份管理方法与系统，其中方法如图1所示，包括：
S100：分别对数据在若干个存储节点中进行完整备份的可靠性目标函数、发送时延目标函数、负载均衡目标函数进行定义；
S200：基于所述可靠性目标函数、所述发送时延目标函数、所述负载均衡目标函数获取数据备份管理的优化目标函数；
S300：采用基于分解的多目标进化算法获取满足所述优化目标函数的最优放置方案矩阵；
S400：基于所述最优放置方案矩阵对数据进行各个存储节点上的备份放置分配。
系统的工作原理如图2所示，包括：
步骤1：用户和权限管理设置。实现用户认证和权限管理功能，并对备份恢复系统进行全生命周期管理。
步骤1.1：用户认证功能实现。用户登陆时，需要将用户处的账户名密码，与系统中保存的账户名密码进行比较，若两者匹配，则表明用户通过认证，否则拒绝用户访问并要求重新进行认证。
步骤1.2：权限管理功能实现。为不同的用户设置角色权限，如普通用户、 运维管理员等，对不同用户角色进行权限设置。
步骤2：备份策略管理。对备份方式和备份策略进行设置，用于调度各种备份任务的运行。
步骤2.1：备份方式设置。本发明支持实时备份和定时备份两种备份方式。用户可以根据不同的需求，选择相应的备份方式。
步骤2.2：备份策略设置。本发明支持完全备份、增量备份和差异备份三种备份策略。当用户第一次备份时，默认进行全量备份。后续备份操作，用户可根据需求进行备份策略选择。
步骤3：数据备份。开始进行数据备份，从数据节点与备份节点建立加密连接并传输数据，按照其申请的备份方式进行数据备份。如用户选择备份至第三方备份软件，则转至步骤3.1；若用户选择备份到系统底层存储空间中，则转至步骤3.2。
步骤3.1：备份恢复管理系统对接第三方备份软件，根据用户设置的备份策略，直接备份至第三方备份软件存储空间。
步骤3.2：系统底层存储空间采用多目标最优备份副本管理方案。基于CDM黄金副本思想，设计多目标备份副本管理策略的步骤及原理分别如图3、图4所示：
对由m个异构且独立的存储节点组成的集群T，T＝(t1,t2,…,tm)，用户将其数据文件K，K＝(k1,k2,…,kn)，备份到存储系统中。
相同的副本不能放置在同一个存储节点上，数据副本在备份节点中的放置方案通过矩阵TK来表示：
其中，
步骤3.2.1如图5所示，从备份系统可靠性、时延和负载均衡三方面，设置相应的目标优化函数：
步骤3.2.1.1可靠性目标函数设计。记存储集群T中第j个存储节点tj的故障率为χj，存储节点的失效率与其使用时间u有关(u的单位为小时),则存储节点tj的失效率函数为数据文件K，K＝(k1,k2,…,kn)，对其每一个数据文件ki，若在m个存储节点中，至少拥有一个完整的备份副本，则认为第i个数据文件ki是可靠的。ki的可靠性定义为：
整个系统，对数据文件K的进行完整备份的可靠性目标函数定义为：
步骤3.2.1.2时延目标函数设计。记存储集群T中第j个存储节点tj的网络访问总时延为Tim_delj(包括处理时延、排队时延、发送时延、传播时延)。考虑数据文件响应时间和传输时间，ki的备份副本在存储节点tj上的第a个访问的发送时延：
其中，vj为备份节点tj的数据发送速率；ci为数据文件ki的大小。ki在不同备份节点上的副本的访问是相互独立的，则ki的总访问时延函数定义为：
其中，表示ki在节点tj上的总访问量。则整个系统，对数据文件K的平均延时目标函数定义为：
步骤3.2.1.3负载均衡目标函数设计。采用加权轮询法，每个备份存储节 点tj，根据其硬件承载能力，为其附上相应的权重σj，σj满足本发明将所有存储节点tj的实际负载与基于硬件承载能力计算出来的理论负载差的绝对值作为衡量系统负载均衡的标准。标准差越小，负载越均衡。定义负载均衡目标函数：
步骤3.2.2定义多目标副本管理策略优化目标函数。在进行备份时，既要提供可靠的云备份服务、保障时延，还要避免某个存储节点的负载过高而出现故障。基于此，定义优化目标函数：
其中，X被称为决策变量，此处为放置方案矩阵TK。经过多目标优化可搜寻到最优放置方案矩阵。f1(X)＝s1(TK)，ρ(ki)表示第i个数据文件ki的大小，Γ(tj)表示第j个存储节点tj的容量大小。ρ(ki)和Γ(tj)的计算单位要求保持一致。σj表示根据存储节点硬件承载能力为其分配的权重。
步骤3.2.3利用基于分解的多目标进化算法解决上述多目标问题。具体步骤为：
步骤3.2.3.1初始化种群大小为P，随机生成Tchebycheff分解算法均匀分布的P个权重向量p＝1,2,…,P，其中l是目标函数个数。
步骤3.2.3.2利用Tchebycheff分解算法将l个目标构成的多目标问题分解为P个子问题，每个子问题为：
其中，z*为多目标函数对应的参考点。是使得子问 题fk(X)成立的最小值，表示子问题fk(X)的参考点。
步骤3.2.3.3初始化进化次数h＝0,从可行域中随机生成P个放置方案矩阵{X1(0),...,Xp(0),…XP(0)}。其中，Xp(0)表示初始化时第p个放置方案矩阵，可行域为满足约束条件的放置方案矩阵中各数值取值范围。
步骤3.2.3.4初始化Pareto前沿解集INL(0)＝Φ，初始化参考点。
步骤3.2.3.5初始化邻域向量，找出距离第p个权重向量最近的Neb个权重向量，并将与最近的Neb个权重向量的索引集记为B(p)＝{p1,p2,...,pNeb}，pNeb＝1,2,...,P。
步骤3.2.3.6设置最大迭代次数H，从h＝1开始迭代，设置h＝H时迭代终止，对每一个权重向量和其对应的个体p＝1,2,…,P根据迭代次数h作更新操作：
步骤3.2.3.6.1进化算法更新个体：
从B(p)中随机选择出索引号c和d，找到其对应的个体和由和产生新解
其中，ξ为进化参数。
步骤3.2.3.6.2更新参考点z*：对每一个分量k＝1,2,L,l，若表示子问题fk的参考点。
步骤3.2.3.6.3更新种群：对于B(p)中的每一个邻域个体b∈B(p)，若则其中gte表示基于Tchebycheff分解算法得到的子问题，λ表示邻域个体的权重向量。
步骤3.2.3.6.4更新非支配解集INL(h)：在INL(h)中，保留INL(h-1)中支配的解，舍弃中INL(h-1)所有被支配的解，若所有INL(h-1)的解都不支配将加入INL(h)。
步骤3.2.3.6.5如果h＝H，得到的非支配解集INL(H)即为放置方案矩阵TK的Pareto最优前沿解集，迭代终止，转至步骤3.2.3.6；否则h＝h+1，转 至步骤3.2.3.1
步骤3.2.3.6.6根据决策者的决策偏好或者随机从Pareto最优解集选择一个权衡解TKoptimal，该TKoptimal即为满足可靠性、延时和负载均衡三种目标Pareto最优的放置方案矩阵。根据TKoptimal完成备份副本在各个节点上的放置分配。
步骤3.2.4：备份完成后，备份系统通知管理节点数据备份已完成，管理节点通过备份信息生成备份日志并进行存档，并通知用户备份完成。
步骤4：数据恢复。备份系统收到数据恢复请求，按照请求中的恢复时间点，通过备份选择算法计算出合适的备份数据，进行数据恢复。具体步骤为：
步骤4.1：备份节点将数据向恢复节点进行传输。在发送端和接收端建立连接后，发送端先将数据进行加密。接收端接收到数据后进行解密操作，并对解密后的数据进行校验值检测。如果通过了校验检测，证明数据传输正确无误，可以将数据写入到磁盘当中。若传输过程中出现错误数据，接收端可以要求发送端对错误数据进行重新发送。
步骤4.2：数据恢复完成后，恢复节点通知管理节点恢复已完成，管理节点通过数据恢复信息生成恢复日志并进行存档，并通知用户数据恢复完成。
步骤5：日志管理。对每一个备份任务流程和恢复任务流程分别建立一个日志文件，跟踪任务流程的信息：记录周期备份和恢复的时间、任务错误信息等，便于查找任务记录和错误定位。
步骤6：监控管理。备份恢复管理系统提供统一的监控大屏，对整个备份恢复流程进行监控，用户可以实时查看数据备份恢复情况，对系统底层存储空间可用容量、各存储节点状态进行监控。
本发明基于多目标优化的思想，将备份系统的可靠性、时延问题和负载均衡问题抽象成多目标优化问题。以向量的形式同时优化系统3个方面的性能。在保持负载均衡的前提下，提升系统的可靠性并使系统时延最低。获取的Pareto最优解，使系统能够达到在多个目标下的性能最优，从而提升系统在多个方面的性能表现。
本发明相对现有技术而言，所具有的优点和效果：
(1)本专利发明设计了一种混合云数据备份恢复管理系统，把不同类型的数据和备份空间在一个系统平台上进行对接整合。用户可以在平台上通过管理界面进行操作，根据其需求将本地/云端上不同的类型数据备份到第三方存储软件或者系统底层存储空间，简化用户操作，提高业务效率。
(2)本专利发明的基于多目标最优副本管理策略的混合云数据备份恢复管理系统，综合考虑备份数据的可靠性、备份系统的时延和备份系统的稳定性，通过基于分解的多目标优化算法，得到黄金副本的备份副本最优放置方案，降低数据丢失和损坏的风险。
(3)本专利发明的基于多目标最优副本管理策略的混合云数据备份恢复管理系统，以图形化的界面进行管理，实时记录日志信息，并且提供统一的监控平台，用户可以对备份恢复策略进行设置，能够灵活、快速的进行数据备份与恢复，帮助管理员快速定位故障，最大程度地提高系统的效率和性能。
最后应当说明的是：以上实施例仅用以说明本发明的技术方案而非对其限制，尽管参照上述实施例对本发明进行了详细的说明，所属领域的普通技术人员应当理解：依然可以对本发明的具体实施方式进行修改或者等同替换，而未脱离本发明精神和范围的任何修改或者等同替换，其均应涵盖在本发明的权利要求保护范围之内。

Claims (10)

  1. 一种基于多目标最优副本管理策略的数据备份管理方法，其特征在于，包括以下步骤：
    分别对数据在若干个存储节点中进行完整备份的可靠性目标函数、发送时延目标函数、负载均衡目标函数进行定义；
    基于所述可靠性目标函数、所述发送时延目标函数、所述负载均衡目标函数获取数据备份管理的优化目标函数；
    采用基于分解的多目标进化算法获取满足所述优化目标函数的最优放置方案矩阵；
    基于所述最优放置方案矩阵对数据进行各个存储节点上的备份放置分配。
  2. 根据权利要求1所述的基于多目标最优副本管理策略的数据备份管理方法，其特征在于，所述可靠性目标函数如下所示：
    其中，表示第i个数据文件ki的可靠性，χj为存储集群T中第j个存储节点tj的故障率，u存储节点的使用时间，为存储节点tj的失效率函数，K表示数据文件，m为存储节点数量，T表示m个存储节点组成的集群，tkij表示节点tj上的副本。
  3. 根据权利要求1所述的基于多目标最优副本管理策略的数据备份管理方法，其特征在于，所述发送时延目标函数如下所示：
    其中，vj为备份节点tj的数据发送速率；ci为数据文件ki的大小，表示ki在节点tj上的总访问量；表示ki的备份副本在存储节点tj上的第a个访问的发送时延。
  4. 根据权利要求1所述的基于多目标最优副本管理策略的数据备份管理方法，其特征在于，所述负载均衡目标函数如下所示：
    其中，σj为每个备份节点tj的权重，且
  5. 根据权利要求1所述的基于多目标最优副本管理策略的数据备份管理方法，其特征在于，所述优化目标函数如下所示：
    max S(X)＝[f1(X),f2(X),f3(X)]T


    其中，X为决策变量，此处为放置方案矩阵TK，f1(X)＝s1(TK)，ρ(ki)表示第i个数据文件ki的大小，Γ(tj)表示第j个存储节点tj的容量大小。
  6. 根据权利要求1所述的基于多目标最优副本管理策略的数据备份管理方法，其特征在于，采用基于分解的多目标进化算法获取满足所述优化目标函数 的最优放置方案矩阵的过程包括：
    初始化种群数量并随机生成与种群数量相同的权重向量；
    基于Tchebycheff分解算法将所述优化目标函数进行分解为与种群数量相同的若干个子问题；
    初始化进化次数，随机生成种群数量相同的放置方案矩阵；
    初始化前沿解集以及优化目标函数对应的参考点；
    初始化邻域向量，获取距离最后一个权重向量最近的若干个权重向量，并基于若干个权重向量构建索引集；
    设置最大迭代次数并进行迭代，基于迭代次数对最后一个权重向量和对应的放置方案矩阵进行迭代更新，当达到最大迭代次数时停止迭代，获取所述最优放置方案矩阵。
  7. 根据权利要求6所述的基于多目标最优副本管理策略的数据备份管理方法，其特征在于，基于迭代次数对每个权重向量和对应的放置方案矩阵进行更新的过程包括：
    基于所述索引集随机获取索引号，获取所述索引号对应的放置方案矩阵；
    基于对应的放置方案矩阵获取新放置方案矩阵，基于所述新放置方案矩阵分别对参考点、种群以及非支配解集进行迭代更新；
    当达到最大迭代次数时，获取更新后的非支配解集并作为最优前沿解集，在所述最优前沿解集中选择一个权衡解，将所述权衡解作为最优放置方案矩阵。
  8. 根据权利要求7所述的基于多目标最优副本管理策略的混合云数据备份管理方法，其特征在于，所述参考点的更新过程中，若则表示子问题fk的参考点，表示新放置方案矩阵；
    所述种群的更新过程中，对于索引集中的每一个邻域个体若则其中gte表示基于Tchebycheff分解算法得到的子问题，λ表示邻域个体的权重向量。
  9. 根据权利要求7所述的基于多目标最优副本管理策略的混合云数据备份 管理方法，其特征在于，所述非支配解集的更新过程中，保留INL(h-1)中支配的解，舍弃INL(h-1)中所有被支配的解，若所有INL(h-1)的解都不支配将加入INL(h)，其中，INL(h-1)表示上一次更新后的非支配解集，表示新放置方案矩阵，INL(h)表示当前的非支配解集。
  10. 一种基于多目标最优副本管理策略的混合云数据备份管理系统，其特征在于，所述系统包括存储器、处理器及存储在存储器上并能够在处理器上运行的计算机程序，所述处理器执行所述计算机程序时实现上述权利要求1至9任一一项所述的方法。
PCT/CN2023/143183 2023-12-12 2023-12-29 基于多目标最优副本管理策略的混合云数据备份管理方法 Pending WO2025001030A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202311701095.5A CN117931513A (zh) 2023-12-12 2023-12-12 基于多目标最优副本管理策略的混合云数据备份管理方法
CN202311701095.5 2023-12-12

Publications (1)

Publication Number Publication Date
WO2025001030A1 true WO2025001030A1 (zh) 2025-01-02

Family

ID=90769325

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/143183 Pending WO2025001030A1 (zh) 2023-12-12 2023-12-29 基于多目标最优副本管理策略的混合云数据备份管理方法

Country Status (2)

Country Link
CN (1) CN117931513A (zh)
WO (1) WO2025001030A1 (zh)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN120105463A (zh) * 2025-05-09 2025-06-06 兰州微宝宝信息技术有限责任公司 基于遗传算法的数据保护方法
CN120750734A (zh) * 2025-07-03 2025-10-03 广州翌拓软件开发有限公司 面向异地容灾云系统的资源动态调度系统及方法
CN121008961A (zh) * 2025-08-07 2025-11-25 中国人民解放军国防科技大学 面向多智能体集群的数据备份方法、装置、设备及介质

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150347430A1 (en) * 2014-05-30 2015-12-03 International Business Machines Corporation Techniques for enabling coarse-grained volume snapshots for virtual machine backup and restore
CN109889573A (zh) * 2019-01-14 2019-06-14 武汉理工大学 混合云中基于ngsa多目标的副本放置方法
CN111124762A (zh) * 2019-12-30 2020-05-08 航天科工网络信息发展有限公司 一种基于改进粒子群算法的动态副本放置方法
CN111859703A (zh) * 2020-07-30 2020-10-30 暨南大学 一种基于热量感知的数据中心节能数据副本放置方法

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111258980B (zh) * 2020-01-18 2024-02-27 广州大鱼创福科技有限公司 一种云存储系统中基于组合预测的动态文件放置方法
CN113112143B (zh) * 2021-04-08 2024-04-23 杭州电子科技大学 云制造环境下多级别制造商协同生产任务分配方法及系统

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150347430A1 (en) * 2014-05-30 2015-12-03 International Business Machines Corporation Techniques for enabling coarse-grained volume snapshots for virtual machine backup and restore
CN109889573A (zh) * 2019-01-14 2019-06-14 武汉理工大学 混合云中基于ngsa多目标的副本放置方法
CN111124762A (zh) * 2019-12-30 2020-05-08 航天科工网络信息发展有限公司 一种基于改进粒子群算法的动态副本放置方法
CN111859703A (zh) * 2020-07-30 2020-10-30 暨南大学 一种基于热量感知的数据中心节能数据副本放置方法

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN120105463A (zh) * 2025-05-09 2025-06-06 兰州微宝宝信息技术有限责任公司 基于遗传算法的数据保护方法
CN120105463B (zh) * 2025-05-09 2025-11-14 兰州微宝宝信息技术有限责任公司 基于遗传算法的数据保护方法
CN120750734A (zh) * 2025-07-03 2025-10-03 广州翌拓软件开发有限公司 面向异地容灾云系统的资源动态调度系统及方法
CN121008961A (zh) * 2025-08-07 2025-11-25 中国人民解放军国防科技大学 面向多智能体集群的数据备份方法、装置、设备及介质
CN121008961B (zh) * 2025-08-07 2026-04-07 中国人民解放军国防科技大学 面向多智能体集群的数据备份方法、装置、设备及介质

Also Published As

Publication number Publication date
CN117931513A (zh) 2024-04-26

Similar Documents

Publication Publication Date Title
US12468824B2 (en) Maintaining encryption during storage system replication
US12050689B2 (en) Host anomaly-based generation of snapshots
US11687418B2 (en) Automatic generation of recovery plans specific to individual storage elements
US11755751B2 (en) Modify access restrictions in response to a possible attack against data stored by a storage system
US12014065B2 (en) Multi-cloud orchestration as-a-service
Shakarami et al. Data replication schemes in cloud computing: a survey
Lin et al. QoS-aware data replication for data-intensive applications in cloud computing systems
US20210216666A1 (en) Inter-I/O Relationship Based Detection of a Security Threat to a Storage System
US20220083657A1 (en) Independent Security Threat Detection and Remediation by Storage Systems in a Synchronous Replication Arrangement
CN117931513A (zh) 基于多目标最优副本管理策略的混合云数据备份管理方法
CN113302584A (zh) 基于云的存储系统的存储管理
US11625481B2 (en) Selective throttling of operations potentially related to a security threat to a storage system
US20220050898A1 (en) Selective Control of a Data Synchronization Setting of a Storage System Based on a Possible Ransomware Attack Against the Storage System
US20250173447A1 (en) Protection of Snapshots of Storage System Data Using a Dedicated Data Store
US20230205591A1 (en) System Having Dynamic Power Management
US12204657B2 (en) Similar block detection-based detection of a ransomware attack
US12554865B2 (en) Quorum-based authorization of requests associated with storage systems
US12411962B2 (en) Managed run-time environment-based detection of a ransomware attack
US12248566B2 (en) Snapshot deletion pattern-based determination of ransomware attack against data maintained by a storage system
US20240281544A1 (en) Multi-Party Authorization for Requests Initiated by a Storage Management System
US20250363227A1 (en) Detection of Data Exfiltration from a Storage System
US20250217501A1 (en) Storage System-based Enhancement of a Network Monitoring Service that Monitors for Anomalous Outgoing Network Traffic from Within a Managed Network
CN105827744A (zh) 云存储平台的数据处理方法
CN120066733A (zh) 面向边缘环境的分布式任务调度系统
WO2024182639A1 (en) System having dynamic power management

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23943501

Country of ref document: EP

Kind code of ref document: A1