CN117033071A - Disaster recovery data high-value quick and orderly recovery method based on file heat and multi-objective optimization - Google Patents
Disaster recovery data high-value quick and orderly recovery method based on file heat and multi-objective optimization Download PDFInfo
- Publication number
- CN117033071A CN117033071A CN202310855044.1A CN202310855044A CN117033071A CN 117033071 A CN117033071 A CN 117033071A CN 202310855044 A CN202310855044 A CN 202310855044A CN 117033071 A CN117033071 A CN 117033071A
- Authority
- CN
- China
- Prior art keywords
- data
- recovery
- disaster recovery
- disaster
- objective optimization
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000011084 recovery Methods 0.000 title claims abstract description 132
- 238000000034 method Methods 0.000 title claims abstract description 39
- 238000005457 optimization Methods 0.000 title claims abstract description 31
- 230000008569 process Effects 0.000 claims abstract description 6
- 230000002776 aggregation Effects 0.000 claims description 5
- 238000004220 aggregation Methods 0.000 claims description 5
- 230000005540 biological transmission Effects 0.000 claims description 4
- 238000004364 calculation method Methods 0.000 claims description 4
- 230000002068 genetic effect Effects 0.000 claims description 4
- 230000004044 response Effects 0.000 abstract description 2
- 238000009472 formulation Methods 0.000 description 4
- 239000000203 mixture Substances 0.000 description 4
- 238000009826 distribution Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 239000000243 solution Substances 0.000 description 2
- 238000003860 storage Methods 0.000 description 2
- 230000004931 aggregating effect Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000035772 mutation Effects 0.000 description 1
- 239000012088 reference solution Substances 0.000 description 1
- 230000033772 system development Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1446—Point-in-time backing up or restoration of persistent data
- G06F11/1458—Management of the backup or restore process
- G06F11/1461—Backup scheduling policy
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1446—Point-in-time backing up or restoration of persistent data
- G06F11/1448—Management of the data involved in backup or backup restore
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1446—Point-in-time backing up or restoration of persistent data
- G06F11/1458—Management of the backup or restore process
- G06F11/1469—Backup restoration techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/12—Computing arrangements based on biological models using genetic models
- G06N3/126—Evolutionary algorithms, e.g. genetic algorithms or genetic programming
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Quality & Reliability (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biophysics (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Physiology (AREA)
- Genetics & Genomics (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention relates to a high-value quick and orderly recovery method of disaster recovery data based on file heat and multi-objective optimization. Compared with the prior art, the method and the system can realize the rapid and orderly recovery of a large number of disaster recovery data of the power metering system, improve the throughput of the system in the process of recovering the disaster recovery data, reduce the response time of user access, improve the data service capability of the system and ensure the maximization of the service value of the power metering disaster recovery data.
Description
Technical Field
The invention belongs to the technical field of electric power metering data, and particularly relates to a high-value quick and orderly recovery method of disaster recovery data based on file heat and multi-objective optimization
The background technology is as follows:
under the big data background, along with the large-scale access of novel power equipment of huge amount to the distribution network, electric energy metering device in the distribution network can produce a large amount of multisource data information, and various electric power metering information is complicated, and a large amount of valuable data resources can't exert its effect for the usability of data drops by a wide margin, is difficult to provide effective support for power system's development.
The disaster recovery technology is an important way for improving the availability of data, and by adding copies of the same data and dispersedly placing the copies in the cluster nodes, the number of users served at the same time can be increased, the speed of data access is increased by parallel reading, and the availability of the electric power metering data is obviously improved.
However, the existing disaster recovery technology is difficult to formulate an effective strategy in the data recovery process, and the disaster recovery data between departments is always lack of effective integration due to the isolated micro-grid and different acquisition standards. Therefore, how to recover the disaster recovery data of power metering with high value and fast order is a problem to be solved.
The traditional data disaster recovery scheme generally adopts a nearby principle, the priority of the data is not divided, the scheme can complete timely recovery of disaster recovery data when the server load rate is low, but under the condition of high load rate, serious loss is easily caused by timely recovery of critical data, the high-value rapid and orderly recovery of disaster recovery data can be realized by adopting a multi-objective optimization algorithm, and the rapid and efficient convergence of performance is achieved by formulating and solving an optimization objective, so that compared with the traditional method, the method has stronger adaptability to scenes.
The invention comprises the following steps:
therefore, the invention aims to provide a high-value quick and orderly recovery method for disaster recovery data based on file heat and multi-objective optimization so as to realize efficient recovery of power metering disaster recovery data.
In order to achieve the above purpose, the present invention provides the following technical solutions:
according to the method, aiming at a power metering data disaster recovery scene, data recovery priority is formulated by calculating file heat, a disaster recovery scheme is designed by adopting a multi-objective optimization algorithm, recovery cost is considered while the problem of quick recovery of disaster recovery data is solved, and the method is suitable for the service requirements of a complex environment of an actual disaster recovery system. The scheme specifically comprises the following steps:
s1: acquiring state information of power metering data, and establishing a disaster recovery data backup metadata attribute group;
s2: calculating the heat value of the disaster recovery data file, and designing a recovery priority ordering method;
s3: designing a disaster recovery data recovery scheme based on a multi-objective optimization algorithm;
s4: the specific scheme flow is briefly described;
in step S1, a metadata attribute set of the disaster recovery data file copy is constructed by counting information such as the disaster recovery data access frequency, the data copy size, the access time, etc.
Further, in the step S2, the heat value of the data file is calculated according to the meta attribute group, and the priority of disaster recovery data is ordered based on the descending method of the heat value.
Further, in the step S3, a disaster recovery data recovery method based on a multi-objective optimization algorithm is designed, the disaster recovery data recovery strategy is dynamically prepared according to the average load rate of the server cluster, and when the load rate is smaller than a preset threshold, recovery tasks of the data copies are sequentially completed according to the file heat value; when the load rate is larger than a preset threshold, a data recovery scheme is formulated by taking minimized load rate, service transmission delay and service data loss as optimization targets.
Further, in the step S4, the specific scheme flow includes four parts, namely node status determination, data priority ordering, disaster recovery policy formulation, and data recovery. In the process of disaster recovery strategy formulation, a Chebyshev weight aggregation method is adopted to decompose a multi-objective optimization problem into a plurality of objective sub-problems, the solution process is fast in convergence, the calculation complexity is low, and high-value, rapid and orderly recovery of disaster recovery data can be ensured.
Compared with the prior art, the invention has the following advantages:
1) According to the invention, the metadata attribute group of the disaster recovery data file copy is constructed, the power metering disaster recovery data is analyzed, the file heat value is calculated according to the indexes such as the file size, the read-write rate and the like, and the disaster recovery data recovery priority is formulated according to the obtained result, so that the data with high value in the power system can be recovered preferentially. And the characteristics of the heat value of the data copy are further considered, the data is subjected to grading treatment, the management efficiency of disaster recovery data is effectively improved, and a support is provided for high-value rapid and orderly recovery of the disaster recovery data.
2) The invention provides a disaster recovery data high-value rapid and orderly recovery method based on multi-objective optimization, which takes minimized load rate, service transmission delay and service data loss as optimization objectives and can formulate a disaster recovery data recovery scheme comprehensively considering multiple factors. Further considering the average load rate of the server cluster, and formulating different disaster recovery strategies under the condition of different load rates, the throughput of the system in the process of recovering disaster recovery data can be improved, the response time of user access is reduced, the data service capacity of the system is improved, and the maximization of the service value of the power metering disaster recovery data is realized.
Description of the drawings:
FIG. 1 is a flow chart of a method for high-value and rapid and orderly recovery of disaster recovery data based on multi-objective optimization according to an embodiment of the present invention;
Detailed Description
The invention will now be described in detail with reference to the drawings and specific examples. The embodiment is implemented on the premise of the technical scheme of the invention, and detailed implementation modes and specific operation processes are given. It should be noted that, without conflict, the embodiments of the present invention and features of the embodiments may be combined with each other.
Fig. 1 is a flow chart of a method for high-value and rapid and orderly recovery of disaster recovery data based on multi-objective optimization according to the present embodiment, which specifically includes the following steps:
1) Node state determination
The invention adopts the heartbeat communication mode to monitor the state of the server node, and judges that the server fails and needs disaster recovery data when the heartbeat of the main server node cannot be monitored for a long time.
2) Disaster recovery data recovery priority determining method
The invention provides a disaster recovery data recovery priority determining method based on file heat. The method further carries out priority formulation by constructing metadata attribute groups of disaster recovery data file copies, and ensures orderly recovery of disaster recovery data, and is specifically introduced as follows:
metadata group construction: metadata property group for constructing disaster tolerant data file copy expressed as
G＝{S,U n ,T l ,T n ,H * ,F}
Wherein S represents a size set of files, U n For a set of number of users accessed over a period of time, T l For the last accessed time set of the file, T n H is the current time of the system * For the set of the heat value of the file in the previous stage, the initial heat value of the file is 0,F, which is the set of the read-write rate of the disaster recovery data file, and the higher the read-write rate is, the more times of accessing the file is indicated, and the larger the corresponding heat value of the file is.
And (5) calculating file heat: and according to the metadata attribute group of the disaster recovery data files, counting the information such as the access frequency, the data volume, the number of access users and the like of each disaster recovery data file in the current system time period, so as to calculate the heat value of the file and evaluate the recovery priority of the file. In the invention, the calculation formula of the file heat is as follows
Wherein, C is the file heat value evaluation index.
Grading disaster recovery: considering that the difference between the heat value of the data copies stored in different servers is possibly smaller, the data recovery needs to be carried out simultaneously under the condition of higher system load rate, so that the data recovery efficiency is further improved, the safety of the system is ensured, the disaster recovery data to be recovered can be subjected to grading treatment at the moment, and the data priority is divided according to grading results. By normalizing the heat valueThe data can be divided into key data to be recovered,Important data to be recovered and general data to be recovered. Wherein critical data to be restored->Important data to be recoveredGeneral data to be restored->
3) Disaster recovery strategy formulation
And calculating the average load rate of the current server cluster. If the average load rate is lower, the concurrent access amount of the system is low, the disaster recovery pressure is low, and the recovery task of the data copy can be completed in sequence according to the file heat value; when the average load rate is larger than a preset threshold omega, a data recovery scheme is formulated according to the disaster recovery data grading result. The calculation formula of the average load rate is as follows
Wherein M is the number of servers in the cluster, S i Is the size of a single data copy, S s The threshold ω=0.6 is preset for the maximum storage amount of the server.
4) Specific scheme flow
The disaster recovery data high-value rapid and orderly recovery scheme flow based on the multi-objective optimization method comprises an initialization stage, a genetic evolution stage and a termination stage. The specific steps are as follows:
an initialization stage: initializing the population size as K, setting an optimization target as minimizing the Load rate Load, the transmission Delay and the service data Loss of the server, and restricting the storage margin of the server as A. The overall optimization function is expressed as
Genetic evolution stage: the invention firstly solves the problem of multiple optimization targetsDecomposing into K single-objective optimization sub-questions, and distributing initial weight vector to each sub-questionThe vector dimension corresponds to the number of optimization objective functions, and then each sub-problem is subjected to cross mutation in the neighbor individuals to generate new server individuals.
Because the servers are in discrete distribution in the three-dimensional space, after a new generation individual is obtained, the Euclidean distance between the position of the new generation individual and the position of each real server is calculated, and the real server closest to the new generation individual is used as the new individual to continue iteration.
Aggregating all new individuals by using Chebyshev weight aggregation method, updating parent population according to aggregation result, and the aggregation formula is that
Wherein x is population individual, z * Is a set of reference solutions.
Termination phase: and when the maximum iteration times are reached, the loop is exited, a disaster recovery data recovery server node list is output, and a recovery task of the disaster recovery data is executed.
Although specific implementations of the invention and the accompanying drawings are disclosed for illustrative purposes only, and are presented to aid in understanding the invention and its implementation, it will be appreciated by those skilled in the art that: various alternatives, variations and modifications are possible without departing from the spirit and scope of the invention and the appended claims. Therefore, the present invention should not be limited to the preferred embodiments and the disclosure of the drawings, but the scope of the invention is defined by the appended claims.
Claims (5)
1. The high-value quick and orderly recovery method for disaster recovery data based on file heat and multi-objective optimization is characterized in that the method aims at a disaster recovery scene of electric power metering data, the data recovery priority is formulated by calculating the file heat, a disaster recovery data recovery scheme is designed by adopting a multi-objective optimization algorithm, recovery cost is considered while the problem of quick recovery of disaster recovery data is solved, and the method is suitable for service requirements of a complex environment of an actual disaster recovery system.
2. The method for high-value quick and orderly recovery of disaster recovery data based on file heat and multi-objective optimization according to claim 1, wherein the scheme specifically comprises the following steps:
s1: acquiring state information of power metering data, and establishing a disaster recovery data backup metadata attribute group;
s2: calculating the heat value of the disaster recovery data file, and designing a recovery priority ordering method;
s3: designing a disaster recovery data recovery scheme based on a multi-objective optimization algorithm;
s4: the specific scheme flow is briefly described.
3. The method for sorting the recovery priorities according to claim 2, wherein the metadata attribute group of the disaster recovery data file copy is constructed by counting information such as the disaster recovery data access frequency, the data copy size, the access time, and the like, the heat value of the data file is calculated according to the attribute group, and the method for sorting the recovery priorities of the disaster recovery data is realized based on the descending order of the heat value.
4. The disaster recovery data recovery method based on the multi-objective optimization algorithm according to claim 2, wherein the disaster recovery data recovery method based on the multi-objective optimization algorithm is designed, a disaster recovery data recovery strategy is dynamically prepared according to the average load rate of the server cluster, and when the load rate is smaller than a threshold value, recovery tasks of data copies are sequentially completed according to the file heat value; when the load rate is greater than the threshold, a data recovery scheme is formulated with the minimum load rate, service transmission delay and service data loss as optimization targets.
5. The disaster recovery data recovery method based on multi-objective optimization algorithm according to claim 2, wherein the specific scheme flow comprises an initialization phase, a genetic evolution phase and a termination phase. In the genetic evolution stage, a Chebyshev weight aggregation method is adopted to decompose the multi-objective optimization problem into a plurality of objective quantum problems, the solving process is fast in convergence, the calculation complexity is low, and high-value, rapid and orderly recovery of disaster recovery data can be ensured.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310855044.1A CN117033071A (en) | 2023-07-12 | 2023-07-12 | Disaster recovery data high-value quick and orderly recovery method based on file heat and multi-objective optimization |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310855044.1A CN117033071A (en) | 2023-07-12 | 2023-07-12 | Disaster recovery data high-value quick and orderly recovery method based on file heat and multi-objective optimization |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117033071A true CN117033071A (en) | 2023-11-10 |
Family
ID=88638089
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310855044.1A Pending CN117033071A (en) | 2023-07-12 | 2023-07-12 | Disaster recovery data high-value quick and orderly recovery method based on file heat and multi-objective optimization |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117033071A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117931830A (en) * | 2024-03-22 | 2024-04-26 | 平凯星辰(北京)科技有限公司 | Data recovery method, device, electronic equipment, storage medium and program product |
-
2023
- 2023-07-12 CN CN202310855044.1A patent/CN117033071A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117931830A (en) * | 2024-03-22 | 2024-04-26 | 平凯星辰(北京)科技有限公司 | Data recovery method, device, electronic equipment, storage medium and program product |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111027736B (en) | Micro-service combined deployment and scheduling method under multi-objective optimization | |
| CN110225074B (en) | Communication message distribution system and method based on equipment address domain | |
| Šikšnys et al. | Aggregating and disaggregating flexibility objects | |
| CN103179052B (en) | A kind of based on the central virtual resource allocation method and system of the degree of approach | |
| WO2016165392A1 (en) | Genetic algorithm-based cloud computing resource scheduling method | |
| CN107330056A (en) | Wind power plant SCADA system and its operation method based on big data cloud computing platform | |
| CN104995870A (en) | Multi-objective server placement determination | |
| CN117033071A (en) | Disaster recovery data high-value quick and orderly recovery method based on file heat and multi-objective optimization | |
| CN111258980B (en) | Dynamic file placement method based on combined prediction in cloud storage system | |
| Liu et al. | A data placement strategy for scientific workflow in hybrid cloud | |
| CN110968920A (en) | Method for placing chain type service entity in edge computing and edge computing equipment | |
| CN105681052A (en) | Energy-saving method for data center distributed file storage | |
| Karthikeyan et al. | Saldeft: Self-adaptive learning differential evolution based optimal physical machine selection for fault tolerance problem in cloud | |
| El Gaily et al. | Constrained quantum optimization for resource distribution management | |
| CN110689175A (en) | Energy consumption optimization method for distributed green cloud data center with chaotic multiple universes | |
| Mohd. Zin et al. | Replication techniques in data grid environments | |
| Rasool et al. | Replica placement in multi-tier data grid | |
| Spivak et al. | Storage tier-aware replicative data reorganization with prioritization for efficient workload processing | |
| Cai et al. | Cluster-based federated learning framework for intrusion detection | |
| CN112417527B (en) | Energy internet data reliable storage system considering attack | |
| Papapetrou et al. | Text clustering for peer-to-peer networks with probabilistic guarantees | |
| Tian et al. | Recovery mechanism of large-scale damaged edge computing net-work in industrial internet of things | |
| CN111813542A (en) | Load balancing method and device for parallel processing of large-scale graph analysis tasks | |
| Mansouri et al. | A review of replica replacement techniques in grid computing and cloud computing | |
| Chen et al. | Dynamic retrieval model of quantitative data of power grid resources based on 3D geographic information systems (GIS) |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication |