CN111752756B - Method for setting database backup strategy through autonomous learning - Google Patents
Method for setting database backup strategy through autonomous learning Download PDFInfo
- Publication number
- CN111752756B CN111752756B CN202010587772.5A CN202010587772A CN111752756B CN 111752756 B CN111752756 B CN 111752756B CN 202010587772 A CN202010587772 A CN 202010587772A CN 111752756 B CN111752756 B CN 111752756B
- Authority
- CN
- China
- Prior art keywords
- backup
- strategy
- database
- load
- client program
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1446—Point-in-time backing up or restoration of persistent data
- G06F11/1448—Management of the data involved in backup or backup restore
Abstract
The invention relates to a method for setting a database backup strategy by autonomous learning, which comprises the following steps: s1) the operation and maintenance platform pushes the backup client program and the initial backup strategy to the target equipment; a backup client program is deployed on target equipment, and an initial backup strategy is used as a default backup strategy for database backup; s2) the backup client program collects the monitoring data of the load parameters of the target device, calculates the load parameter control values of different time points through a rule engine, then obtains the optimal backup time point and the load parameter control value thereof in a backup period, further optimizes the backup strategy, and takes the backup strategy as the default backup strategy for the backup of the next backup time point database; s3) the backup client program uploads the optimized backup strategy and the real-time monitoring data to the operation and maintenance platform so as to provide reference basis for the database administrator to intervene the backup strategy manually. The method is beneficial to automatically adjusting and optimizing the database backup strategy according to different target devices.
Description
Technical Field
The invention belongs to the technical field of database backup, and particularly relates to a method for setting a database backup strategy through autonomous learning.
Background
The database stores massive core data of a company and generally needs to be backed up regularly. Backup of a database generally serves two purposes: firstly, in consideration of the safety of a database system, when the database system fails, backup data is used for recovery; and secondly, historical data is archived, and original data is provided for subsequent data analysis or audit requirements.
At present, the types of related databases are more, there are common relational databases, cache databases and many non-relational databases, such as document databases, time-series databases and the like, and the backup modes used by each type of database are also different, so that the complexity of database backup is increased, and the requirements on operators are higher.
The backup strategies of each type of database are different, most of the backup strategies can be set only according to the experience of operation and maintenance personnel, the technical requirements on the operation and maintenance personnel are high, the operation is complex, and the applicability is poor.
Disclosure of Invention
The invention aims to provide a method for setting a database backup strategy through autonomous learning, which is beneficial to automatically adjusting and optimizing the database backup strategy according to different target devices.
In order to achieve the purpose, the invention adopts the technical scheme that: a method for setting a database backup strategy through autonomous learning comprises the following steps:
s1) backup policy initialization: the operation and maintenance platform pushes a backup client program and an initial backup strategy to the target equipment; a backup client program is deployed on target equipment, and an initial backup strategy is used as a default backup strategy for database backup;
s2) backup strategy autonomous learning and optimization: the backup client program collects and stores monitoring data of load parameters of target equipment according to set frequency, calculates load parameter control values of different time points through a rule engine based on the collected monitoring data of the load parameters, then obtains an optimal backup time point and a load parameter control value thereof in a backup period, further optimizes a backup strategy, and takes the optimal backup strategy as a default backup strategy for backup of a next backup time point database;
s3) backup policy passback: after the backup strategy is optimized each time, the backup client program uploads the optimized backup strategy to the operation and maintenance platform, and simultaneously collects real-time monitoring data of the load parameters of the target equipment in the backup process and uploads the real-time monitoring data to the operation and maintenance platform so as to provide a reference basis for a database administrator to manually intervene the backup strategy.
Further, in step S1, when the operation and maintenance platform needs to push the backup client program and the initial backup policy to the multiple target devices, the operation and maintenance platform records corresponding tasks, and then calculates a random time point to push the tasks, so as to avoid abnormal pushing caused by an excessive load on the operation and maintenance platform due to a large number of pushed tasks.
Further, the random time point Ti+1The calculation method comprises the following steps:
let current timestamp be TiLet Ti+1=Ti+A^(Ti-B)p(Ti-C)1-p^N(0,D)，Wherein p =1 or 0, 0<A<=1，15<B<=30，1<=C<=15, N (0, D) represents a random number generated from a gaussian distribution with mean 0 and variance D, where 0 is<D<=10, a takes real numbers between 0 and 1, B, C, D takes integers; then T is addedi+1Constrained to the interval [ C, B]Inner, i.e. if Ti+1If greater than B, then T isi+1Set to B, if Ti+1If less than C, then T isi+1Is set as C.
Further, after the backup client program is deployed successfully on the target device, the backup start time of the initial backup strategy is set, and the read-write speed of the disk during backup is controlled.
Further, the backup client program realizes backup strategy autonomous learning and optimization through a system monitoring module, a rule engine and an autonomous learning module; after a backup client program is deployed successfully on target equipment, the system monitoring module collects monitoring data of various load parameters of the target equipment according to a set frequency and stores the monitoring data in a local database, wherein the various load parameters comprise the CPU utilization rate and the memory utilization rate of the target equipment, the IOPS of a disk and the QPS of a database instance;
when a CPU of the target equipment is in an idle state, calling a rule engine by a backup client program to perform autonomous learning, periodically reading monitoring data of each load parameter in a local database by the rule engine at different time points, and calculating each load parameter control value at different time points based on a control algorithm of each load parameter;
the autonomous learning module selects the time point with the optimal load parameter control value and the corresponding load parameter control value as the optimal backup time point and the load parameter control value in a backup period based on the load parameter control values at different time points, so as to optimize a backup strategy.
Further, the rule engine mainly includes a rule engine execution interface module bakrule executing, a rule element defining module bakrule list and a rule engine executor bakrule runner, where the bakrule executing module is used to define an action mode of rule configuration, the bakrule list module is used to define a rule element, and the bakrule runner is used to execute a specific rule code and trigger a corresponding event judgment; the rule elements defined by the BakRuleList module are stored in a local database; after the rule engine starts to work, the BakRuleExecutor module is connected to a local database to load rule elements, cache loading is completed, meanwhile, the BakRuleExecutor module defines an action mode of rule configuration, and preposed data loading work is done for the subsequent work of the BakRuleRunner module; the BakRuleRunner module is a core module of the rule engine, and loads the monitoring data of the collected load parameters in the local database after the rule elements are loaded, so as to perform autonomous learning.
Further, the time interval of the periodic reading at different time points is every hour, and the one backup period is one day, i.e. 24 hours.
Further, the method for calculating the control values of the load parameters at different time points by the rule engine based on the control algorithm of the load parameters is as follows:
setting the average value of all monitoring data of a load parameter from the last time point to the current time point as Ck iLet Ck i+1=Ck i+Ak*(Ck i-Bk)^(Ck i-Mk)^(Ck i-Nk) Wherein i represents the ith time point, k represents the kth item load parameter, k =1, 2, …, q, q represents the total q items load parameter; c is to bek i+1Constrained to the interval [ M, N]In, i.e. if Ck i+1If greater than N, then C is addedk i+1Is set to N, if Ck i+1Less than M, then Ck i+1Setting the load parameter as M, and controlling the load parameter in the backup process between the set ranges by the method;
after obtaining the control values of the load parameters, setting the control values of the load parameters into an optimized backup strategy as the control values when the next backup is started; when the backup client program starts to work, the backup client program reads the control values of the load parameters in the optimized backup strategy, and the load parameters are limited within the control value range in the backup process.
Further, 0<Ak<=1，20<Bk<= 50; for the target device's CPU utilization, k =1, 1<=M1<=30，30<N1<= 60; for memory utilization, k =2, 1<=M2<=10，10<N2<= 30; for IOPS of disk, k =3, 1<=M3<=10，10<N3<= 30; for QPS of database instance, k =4, 1<=M4<=10，30<N4<=30。
Further, the method for the autonomous learning module to select the time point at which the control values of the load parameters are optimal is as follows:
and summing the load parameter control values at different time points, sequencing the sum values, and taking the value from all the sum values according to a set method, wherein the corresponding time point is the time point with the optimal load parameter control value.
Compared with the prior art, the invention has the following beneficial effects: the method can automatically adjust and optimize the database backup strategy according to the system load condition of target equipment, solves the problem that the backup strategy can only be set by relying on DBA experience, greatly relieves the technical requirements of database backup on operation and maintenance personnel, is easy to deploy and realize, simplifies the setting process of the user backup strategy, improves the flexibility of the database backup, avoids the problems of system resource occupation and the like caused by backup, improves the stability of the system, and is suitable for application occasions needing database backup, so the method has strong practicability and wide application prospect.
Drawings
Fig. 1 is a schematic flow chart of a method implementation of the embodiment of the present invention.
Detailed Description
The invention is described in further detail below with reference to the figures and the embodiments.
The invention provides a method for setting a database backup strategy through autonomous learning, which comprises the following steps as shown in figure 1:
s1) backup policy initialization: the operation and maintenance platform pushes a backup client program and an initial backup strategy to the target equipment; and the backup client program is deployed on the target equipment, and the initial backup strategy is used as a default backup strategy for database backup.
The initial backup policy files are as follows:
[backup]
gid = 61dkvrfx # group number, operation and maintenance platform identification
start _ at = 00:01 # backup default start time
cpu _ limit = 35 # cpu usage
memory _ limit = 45 # memory usage
diskio _ limit = 55 # disk iops usage
QPS _ limit = 65 # QPS usage
dbtype = MySQL # database type, mySQL, Redis, MongoDB, etc
IP = 172.18.1.120 # database instance IP
Port = 3306 # database instance port
backup dir =/data/dbbackup # backup directory
dacycle = 7 # backup file retention period
mode = physics # backup mode, physical backup physics, logical backup logic
dbname = -A # backup database
hosttype =1 # host type 1 virtual machine, 0 physical machine
After a platform administrator initiates a task of pushing a backup client program, the operation and maintenance platform needs to push the backup client program and an initial backup strategy to a plurality of target devices, records corresponding tasks, and calculates a random time point within 30 seconds to push the tasks in order to prevent the exception of a download server caused by a large number of pushed tasks, so that the phenomenon that the pushing exception is caused by the large number of pushed tasks due to the overlarge load of the operation and maintenance platform is avoided.
The randomTime point Ti+1The calculation method comprises the following steps:
let current timestamp be TiLet Ti+1=Ti+A^(Ti-B)p(Ti-C)1-p^ N (0, D), wherein p =1 or 0, 0<A<=1，15<B<=30，1<=C<=15, N (0, D) represents a random number generated from a gaussian distribution with mean 0 and variance D, where 0 is<D<=10, a takes real numbers between 0 and 1, B, C, D takes integers; then T is addedi+1Constrained to the interval [ C, B]Inner, i.e. if Ti+1If greater than B, then T isi+1Set to B, if Ti+1If less than C, then T isi+1Is set as C.
When the backup client program is deployed successfully on the target equipment, the backup start time of the initial backup strategy is set, a random time point between 0 point and 3 points in the morning is controlled, and the disk read-write speed during backup is controlled within 100 MB/s.
S2) backup strategy autonomous learning and optimization: the backup client program collects and stores monitoring data of load parameters of the target equipment according to the set frequency, calculates load parameter control values of different time points through a rule engine based on the collected monitoring data of the load parameters, then obtains an optimal backup time point and a load parameter control value thereof in a backup period, further optimizes a backup strategy, and takes the backup strategy as a default backup strategy for backup of a next backup time point database.
The backup client program realizes backup strategy autonomous learning and optimization through a system monitoring module, a rule engine and an autonomous learning module.
And when the backup client program is deployed successfully on the target equipment, the system monitoring module acquires monitoring data of each load parameter of the target equipment according to the set frequency and stores the monitoring data in the local sqlite database. The load parameters include the CPU utilization rate and the memory utilization rate of the target device, the IOPS of the disk, the QPS of the database instance, and other indexes that may affect the normal operation of the system.
When the CPU of the target device is in an idle state (in this embodiment, the determination is based on the CPU usage rate being lower than 20%), the backup client program invokes the rule engine to perform autonomous learning, the rule engine periodically reads the monitoring data of each load parameter in the local database at different time points, and calculates each load parameter control value at different time points based on the control algorithm of each load parameter.
In this embodiment, the frequency of monitoring data acquisition is once every 5 minutes. Meanwhile, in order to prevent various problems caused by the overlarge data file, the data retention period is defaulted to 7 days. The time interval of the periodic reading at different time points is every hour, and the one backup period is one day, namely 24 hours.
The rule engine mainly comprises the following modules:
the rule engine executes an interface module BakRuleExecutor and is used for defining the action mode of rule configuration;
a rule element definition module, BakRuleList, for defining rule elements,
and the rule engine executor BakRuleRunner is used for executing specific rule codes and triggering corresponding event judgment.
The rule elements defined by the BakRuleList module are stored in a local sqlite database, and the table structure is as follows:
create table bak_rule_list (
id int(11) not null auto_increment primary key ,
rule_no varchar(20) not null default ‘’,
rule_context text);
where, rule no is a rule number, and rule context is a rule content.
class BakRuleList:
rule_no = ''
rule_context = ''
Definitions of
def __init__(self,no,context):
self.rule_no= n
self.rule_context= a
def get(self):
return {
self.role_no,self.
rule_context
}
def set(self,key,value):
If key == self.rule_no:
self.rule_context = value
return true
return false
After the rule engine starts to work, the BakRuleExecutor module is connected to a local database to load rule elements, cache loading is completed, meanwhile, the BakRuleExecutor module defines an action mode of rule configuration, and preposed data loading work is done for the subsequent work of the BakRuleRunner module. The BakRuleRunner module is a core module of the rule engine, and loads the monitoring data of the collected load parameters in the local database after the rule elements are loaded, so as to perform autonomous learning.
In this embodiment, the method for calculating the control values of each load parameter at different time points by the rule engine based on the control algorithm of each load parameter (the CPU utilization of the target device, the memory utilization, the IOPS of the disk, the QPS of the database instance, and the like) includes:
setting the average value of all monitoring data of a load parameter from the last time point to the current time point as Ck iLet Ck i+1=Ck i+Ak*(Ck i-Bk)^(Ck i-Mk)^(Ck i-Nk) Wherein i represents the ith time point, k represents the kth item load parameter, k =1, 2, …, q, q represents the total q items load parameter; c is to bek i+1Constrained to the interval [ M, N]In, i.e. if Ck i+1If greater than N, then C is addedk i+1Is set to N, if Ck i+1Less than M, then Ck i+1And setting to be M.
Wherein, 0<Ak<=1，20<Bk<= 50; CPU utilization for target deviceRate, k =1, 1<=M1<=30，30<N1<= 60; for memory utilization, k =2, 1<=M2<=10，10<N2<= 30; for IOPS of disk, k =3, 1<=M3<=10，10<N3<= 30; for QPS of database instance, k =4, 1<=M4<=10，30<N4<= 30. By the method, various load parameters in the backup process are controlled in a set range, and the method effectively controls the following conditions caused by a backup client program: the problems of database system instability caused by CPU surge, database instance OOM caused by memory surge, slow response of database instance caused by disk IOPS surge, slow response of database instance caused by database instance QPS surge and the like are solved. The method specifically comprises the following steps:
method of controlling CPU utilization (P2):
setting the average CPU utilization to CiLet Ci+1 = Ci + A*（Ci-B）^（Ci-M）^（Ci-N) in which 0<A<=1,20<B<=50,1<=M<=30,30<N<=60, mixing Ci+1Constrained to the interval [ M, N]In, i.e. if Ci+1If greater than N, C is addedi+1Set to N, if Ci+1When less than M, then C is addedi+1And setting to be M. By the method, the CPU utilization rate of the backup program is controlled to be 30-60%, and the instability of the database system caused by the violent CPU caused by the backup client program is effectively controlled.
Method of controlling memory utilization (P3):
setting the average memory utilization rate as CiLet Ci+1 = Ci + A*（Ci-B）^（Ci-M）|（Ci-N）^（Ci-M) in which 0<A<=1,20<B<=50,1<=M<=10,10<N<=30, mixing Ci+1Constrained to the interval [ M, N]In, i.e. if Ci+1If greater than N, C is addedi+1Set to N, if Ci+1When the average molecular weight is less than 5M, C is addedi+1And setting to be M. By the method, the memory utilization rate of the backup program is controlled to be higher than 10-30% of the average memory utilization rate, and the high-index memory caused by the backup client program is effectively controlledProblem of the database instance OOM.
Method of controlling disk IOPS (P4):
setting the current average disk IOPS to CiLet Ci+1 = Ci + A*（Ci-B）^（Ci-M）^（Ci-N) in which 0<A<=1,20<B<=50,1<=M<=10,10<N<=30, mixing Ci+1Constrained to the interval [ M, N]In, i.e. if Ci+1If greater than N, C is addedi+1Set to N, if Ci+1When less than M, then C is addedi+1And setting to be M. By the method, the disk IOPS of the backup program is controlled to be between 10% and 30% of the average disk IOPS, so that the problem of slow response of the database instance caused by the fact that the backup client program causes the disk IOPS to be high is effectively controlled.
Method of controlling database instance QPS (P5):
setting QPS of the average database to CiLet Ci+1 = Ci + A*（Ci-B）^（Ci-M）^（Ci-N) in which 0<A<=1,20<B<=50,1<=M<=10,10<N<=30, mixing Ci+1Constrained to the interval [ M, N]In, i.e. if Ci+1If greater than N, C is addedi+1Set to N, if Ci+1When less than M, then C is addedi+1And setting to be M. By the method, the database instance QPS of the backup program controls the average QPS of the database instance to be between 10% and 30%, so that the problem of slow response of the database instance caused by the violent high QPS of the database instance caused by the backup client program is effectively controlled.
After obtaining the control values of the load parameters, setting the control values of the load parameters into an optimized backup strategy as the control values when the next backup is started; when the backup client program starts to work, the backup client program reads the control values of the load parameters in the optimized backup strategy, and the load parameters are limited in a set value range taking the control values as the center in the backup process.
According to the control algorithm, the rule engine executor adds the calculated new rule elements to a local database through a method defined by BakRuleList, so that the cache data of the subsequent rule engine can be loaded.
The autonomous learning module calculates each load parameter control value of each hour based on each load parameter control value of different time points, namely according to the rule engine actuator, and selects the time point with the optimal load parameter control value and each corresponding load parameter control value as the optimal backup time point and the load parameter control value in one day so as to optimize the backup strategy. And the optimized backup strategy is used as the default backup strategy of the database backup at the next backup time point, namely the next backup time point in tomorrow backup.
In this embodiment, the method for the autonomous learning module to select the time point at which each load parameter control value is optimal includes: the load parameter control values at different time points are summed (in this embodiment, P2+ P3+ P4+ P5), and the summed values are sorted, so that the summed value is the smallest or the summed values are taken from all the summed values according to a set method, for example, several values with the largest summed value and the smallest summed value are removed, and a value is randomly taken from the summed values in the middle, and the corresponding time point is used as the time point at which the load parameter control values are optimal.
The optimized backup policy file is as follows:
[backup]
gid = 61dkvrfx # group number, operation and maintenance platform identification
start _ at = P1 # backup default start time
cpu _ limit = P2 # cpu usage
memory _ limit = P3 # memory usage rate
diskio _ limit = P4 # disk iops usage rate
QPS _ limit = P5 # QPS usage
dbtype = MySQL # database type, mySQL, Redis, MongoDB, etc
IP = 172.18.1.120 # database instance IP
Port = 3306 # database instance port
backup dir =/data/dbbackup # backup directory
dacycle = 7 # backup file retention period
mode = physics # backup mode, physical backup physics, logical backup logic
dbname = -A # backup database
hosttype =1 # host type 1 virtual machine, 0 physical machine
The autonomous learning and optimization is an iterative process, and in an iterative period (default 7 days), the database backup strategy is continuously adjusted to an optimal strategy along with the change of the system load. And in the next iteration cycle, the optimal backup strategy in the previous iteration cycle can be the initial default backup strategy. Therefore, an efficient balance point is obtained in the aspects of ensuring the stability of the system and optimizing the database backup.
S3) backup policy passback: after optimizing the backup strategy each time, the backup client program uploads the optimized backup strategy to the operation and maintenance platform, and simultaneously collects real-time monitoring data of the load parameters of the target equipment in the backup process and uploads the real-time monitoring data to the operation and maintenance platform so as to provide a reference basis for a database administrator DBA to manually intervene in the backup strategy.
Because the time for autonomous learning of each backup client program is different, the autonomous learning tasks are completed under the condition that the target machine CPU is controlled to be relatively idle, in order to prevent a great number of return tasks from causing downtime of an operation and maintenance platform, the return time is backed up by adopting a random time point, and the specific scheme can refer to a random time point calculation method for pushing an initial backup strategy.
The above are preferred embodiments of the present invention, and all changes made according to the technical scheme of the present invention that produce functional effects do not exceed the scope of the technical scheme of the present invention belong to the protection scope of the present invention.
Claims (9)
1. A method for setting a database backup strategy through autonomous learning is characterized by comprising the following steps:
s1) backup policy initialization: the operation and maintenance platform pushes a backup client program and an initial backup strategy to the target equipment; a backup client program is deployed on target equipment, and an initial backup strategy is used as a default backup strategy for database backup;
s2) backup strategy autonomous learning and optimization: the backup client program collects and stores monitoring data of load parameters of target equipment according to set frequency, calculates load parameter control values of different time points through a rule engine based on the collected monitoring data of the load parameters, then obtains an optimal backup time point and a load parameter control value thereof in a backup period, further optimizes a backup strategy, and takes the optimal backup strategy as a default backup strategy for backup of a next backup time point database;
s3) backup policy passback: after optimizing the backup strategy each time, uploading the optimized backup strategy to the operation and maintenance platform by the backup client program, and simultaneously, collecting real-time monitoring data of the load parameters of the target equipment in the backup process and uploading the real-time monitoring data to the operation and maintenance platform to provide a reference basis for a database administrator to manually intervene in the backup strategy;
the backup client program realizes backup strategy autonomous learning and optimization through a system monitoring module, a rule engine and an autonomous learning module; after a backup client program is deployed successfully on target equipment, the system monitoring module collects monitoring data of various load parameters of the target equipment according to a set frequency and stores the monitoring data in a local database, wherein the various load parameters comprise the CPU utilization rate and the memory utilization rate of the target equipment, the IOPS of a disk and the QPS of a database instance;
when a CPU of the target equipment is in an idle state, calling a rule engine by a backup client program to perform autonomous learning, periodically reading monitoring data of each load parameter in a local database by the rule engine at different time points, and calculating each load parameter control value at different time points based on a control algorithm of each load parameter;
the autonomous learning module selects the time point with the optimal load parameter control value and the corresponding load parameter control value as the optimal backup time point and the load parameter control value in a backup period based on the load parameter control values at different time points, so as to optimize a backup strategy.
2. The method according to claim 1, wherein in step S1, when the operation and maintenance platform needs to push the backup client program and the initial backup policy to a plurality of target devices, the operation and maintenance platform records corresponding tasks, and then calculates a random time point to push the tasks, so as to avoid abnormal pushing caused by an excessive load on the operation and maintenance platform due to a large number of pushed tasks.
3. The method for setting up database backup strategy for autonomous learning according to claim 2, wherein said random time point T isi+1The calculation method comprises the following steps:
let current timestamp be TiLet Ti+1=Ti+A^(Ti-B)p(Ti-C)1-p^ N (0, D), wherein p =1 or 0, 0<A<=1，15<B<=30，1<=C<=15, N (0, D) represents a random number generated from a gaussian distribution with mean 0 and variance D, where 0 is<D<=10, a takes real numbers between 0 and 1, B, C, D takes integers; then T is addedi+1Constrained to the interval [ C, B]Inner, i.e. if Ti+1If greater than B, then T isi+1Set to B, if Ti+1If less than C, then T isi+1Is set as C.
4. The method of claim 1, wherein after the backup client program is deployed successfully on the target device, the backup start time of the initial backup policy is set, and the disk read-write speed during backup is controlled.
5. The method for setting up database backup strategy through autonomous learning according to claim 1, wherein the rule engine mainly includes a rule engine execution interface module BakRuleExecutor, a rule element definition module BakRuleList and a rule engine executor BakRuleRunner, the BakRuleExecutor module is used for defining action mode of rule configuration, the BakRuleList module is used for defining rule elements, and the BakRuleRunner is used for executing specific rule codes and triggering corresponding event judgment; the rule elements defined by the BakRuleList module are stored in a local database; after the rule engine starts to work, the BakRuleExecutor module is connected to a local database to load rule elements, cache loading is completed, meanwhile, the BakRuleExecutor module defines an action mode of rule configuration, and preposed data loading work is done for the subsequent work of the BakRuleRunner module; the BakRuleRunner module is a core module of the rule engine, and loads the monitoring data of the collected load parameters in the local database after the rule elements are loaded, so as to perform autonomous learning.
6. The method for setting up a database backup strategy according to claim 1, wherein the time interval for reading periodically at different time points is every hour, and the backup period is one day, i.e. 24 hours.
7. The method for setting up the database backup strategy through autonomous learning according to claim 1, wherein the rule engine calculates the control values of the load parameters at different time points based on the control algorithms of the load parameters by:
setting the average value of all monitoring data of a load parameter from the last time point to the current time point as Ck iLet Ck i+1=Ck i+Ak*(Ck i-Bk)^(Ck i-Mk)^(Ck i-Nk) Wherein i represents the ith time point, k represents the kth item load parameter, k =1, 2, …, q, q represents the total q items load parameter; c is to bek i+1Constrained to the interval [ M, N]In, i.e. if Ck i+1If greater than N, then C is addedk i+1Is set to N, if Ck i+1Less than M, then Ck i+1Setting the load parameter as M, and controlling the load parameter in the backup process between the set ranges by the method;
after obtaining the control values of the load parameters, setting the control values of the load parameters into an optimized backup strategy as the control values when the next backup is started; when the backup client program starts to work, the backup client program reads the control values of the load parameters in the optimized backup strategy, and the load parameters are limited within the control value range in the backup process.
8. The method for setting up database backup strategy for autonomous learning of claim 7 wherein 0<Ak<=1，20<Bk<= 50; for the target device's CPU utilization, k =1, 1<=M1<=30，30<N1<= 60; for memory utilization, k =2, 1<=M2<=10，10<N2<= 30; for IOPS of disk, k =3, 1<=M3<=10，10<N3<= 30; for QPS of database instance, k =4, 1<=M4<=10，30<N4<=30。
9. The method for autonomously learning and setting database backup strategy according to claim 1, wherein the method for autonomously learning module to select the time point with optimal control value of each load parameter is as follows:
and summing the load parameter control values at different time points, sequencing the sum values, and taking the value from all the sum values according to a set method, wherein the corresponding time point is the time point with the optimal load parameter control value.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010587772.5A CN111752756B (en) | 2020-06-24 | 2020-06-24 | Method for setting database backup strategy through autonomous learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010587772.5A CN111752756B (en) | 2020-06-24 | 2020-06-24 | Method for setting database backup strategy through autonomous learning |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111752756A CN111752756A (en) | 2020-10-09 |
| CN111752756B true CN111752756B (en) | 2021-02-19 |
Family
ID=72677079
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010587772.5A Active CN111752756B (en) | 2020-06-24 | 2020-06-24 | Method for setting database backup strategy through autonomous learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111752756B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112214358A (en) * | 2020-10-30 | 2021-01-12 | 上海爱数信息技术股份有限公司 | Backup and recovery system and method for GaussDB distributed database |
| CN114546714B (en) * | 2020-11-27 | 2023-07-21 | 中国移动通信集团四川有限公司 | Data backup method, system, device and storage medium |
| CN113806121A (en) * | 2021-02-09 | 2021-12-17 | 京东科技控股股份有限公司 | Instance backup abnormity detection method and device, electronic equipment and storage medium |
Family Cites Families (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101788938B (en) * | 2010-03-26 | 2011-12-14 | 北京邮电大学 | Data backup method based on user storing actions |
| CN102104496B (en) * | 2010-12-23 | 2013-08-14 | 北京航空航天大学 | Fault tolerance optimizing method of intermediate data in cloud computing environment |
| WO2014133502A1 (en) * | 2013-02-27 | 2014-09-04 | Hewlett-Packard Development Company, L.P. | Sending a request to a management service |
| DE102013102195A1 (en) * | 2013-03-06 | 2014-09-11 | Techem Energy Services Gmbh | Method for operating a battery-powered device and circuitry arranged therefor |
| CN103617098B (en) * | 2013-12-03 | 2017-06-16 | 上海爱数信息技术股份有限公司 | Intelligent backup method and intelligent backup system based on data variation |
| CN104182302A (en) * | 2014-09-15 | 2014-12-03 | 北京国双科技有限公司 | Database backup method and device |
| CN105677507B (en) * | 2014-11-21 | 2019-01-01 | 中国移动通信集团公司 | A kind of business data cloud standby system and method |
| CN108241553B (en) * | 2016-12-23 | 2022-04-08 | 中科星图股份有限公司 | Data backup control method |
| CN108733508B (en) * | 2017-04-17 | 2022-03-11 | 伊姆西Ip控股有限责任公司 | Method and system for controlling data backup |
| CN108958968B (en) * | 2017-05-17 | 2023-12-22 | 中兴通讯股份有限公司 | File processing method and device |
| CN108255641B (en) * | 2017-12-25 | 2020-08-18 | 南京壹进制信息科技有限公司 | CDP disaster recovery method based on cloud platform |
| CN109901952B (en) * | 2019-03-06 | 2021-11-26 | 山东申启信息技术有限公司 | Data backup method, system and server |
| CN110321253A (en) * | 2019-07-09 | 2019-10-11 | 北京百度网讯科技有限公司 | Method and apparatus for Backup Data |
| CN110473435B (en) * | 2019-07-30 | 2021-07-20 | 北京捷足先登教育科技有限公司 | Word auxiliary learning system and method with quantification of learning period |
| CN110968458B (en) * | 2019-11-26 | 2022-03-29 | 山东大学 | Backup system and method based on reinforcement learning and oriented to nonvolatile processor |
-
2020
- 2020-06-24 CN CN202010587772.5A patent/CN111752756B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN111752756A (en) | 2020-10-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111752756B (en) | Method for setting database backup strategy through autonomous learning | |
| US8260750B1 (en) | Intelligent backup escalation system | |
| US20090193064A1 (en) | Method and system for access-rate-based storage management of continuously stored data | |
| US6044335A (en) | Productivity metrics for application software systems | |
| CN106202444A (en) | A kind of implementation method of data base's O&M monitoring | |
| CN101692226B (en) | Storage method of mass filing stream data | |
| CN112433919B (en) | Information warning method, equipment and storage medium | |
| CN103970851A (en) | Method for directly issuing large enterprise group head office financial statement according to mass of voucher data | |
| CN105656698A (en) | Intelligent monitoring structure and method for network application system | |
| CN101937474A (en) | Mass data query method and device | |
| CN104246787A (en) | Parameter adjustment for pattern discovery | |
| CN110363381B (en) | Information processing method and device | |
| CN111506271A (en) | Data segment unit transmission differentiation management method and distributed block storage system | |
| CN112527381A (en) | Multi-platform software version management and updating system | |
| US20210004675A1 (en) | Predictive apparatus and method for predicting workload group metrics of a workload management system of a database system | |
| CN113328920B (en) | Method for collecting and storing equipment data | |
| CN109871359A (en) | File watching system and method | |
| CN103597473A (en) | Systems and methods for merging partially aggregated query results | |
| CN114610234B (en) | Storage system parameter recommendation method and related device | |
| CN115981562A (en) | Data processing method and device | |
| CN109189611A (en) | A kind of method, apparatus and system of data backup and resume | |
| Hu et al. | A practical RBF framework for database load balancing prediction | |
| CN113886503A (en) | Distributed storage method and system for electric power acquisition data | |
| CN108765140A (en) | Batch service abnormality monitoring method and the first electronic equipment | |
| CN110851486A (en) | Data storage method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |