CN113778649A - Multi-backup task dynamic scheduling method, device, equipment and medium - Google Patents
Multi-backup task dynamic scheduling method, device, equipment and medium Download PDFInfo
- Publication number
- CN113778649A CN113778649A CN202111071929.XA CN202111071929A CN113778649A CN 113778649 A CN113778649 A CN 113778649A CN 202111071929 A CN202111071929 A CN 202111071929A CN 113778649 A CN113778649 A CN 113778649A
- Authority
- CN
- China
- Prior art keywords
- backup
- tasks
- backup tasks
- target end
- task
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 41
- 238000012163 sequencing technique Methods 0.000 claims abstract description 12
- 230000005540 biological transmission Effects 0.000 claims description 25
- 238000004590 computer program Methods 0.000 claims description 12
- 238000004422 calculation algorithm Methods 0.000 claims description 4
- 238000004364 calculation method Methods 0.000 claims description 2
- 230000003044 adaptive effect Effects 0.000 abstract description 3
- 101150027068 DEGS1 gene Proteins 0.000 description 6
- 102100037416 Sphingolipid delta(4)-desaturase DES1 Human genes 0.000 description 6
- 238000010586 diagram Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 101100194363 Schizosaccharomyces pombe (strain 972 / ATCC 24843) res2 gene Proteins 0.000 description 2
- 230000001186 cumulative effect Effects 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000014759 maintenance of location Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 239000002699 waste material Substances 0.000 description 1
- 230000003442 weekly effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/48—Program initiating; Program switching, e.g. by interrupt
- G06F9/4806—Task transfer initiation or dispatching
- G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
- G06F9/4881—Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1446—Point-in-time backing up or restoration of persistent data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
- G06F9/5038—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals considering the execution order of a plurality of tasks, e.g. taking priority or time dependency constraints into consideration
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Quality & Reliability (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a method, a device, equipment and a medium for dynamically scheduling multiple backup tasks, which comprises the following steps: (1) receiving backup tasks of a plurality of source ends; (2) collecting the service peak time of a server where a database is located; (3) collecting the data volume of database backup, calculating backup time, and setting server attribute; (4) sequencing the backup tasks and arranging the backup tasks in a non-service peak period; (5) and judging the target ends of one or more backup tasks, and distributing the tasks according to the different target ends and the storage capacity and the network speed limit. Receiving and collecting data quantity needing backup by a plurality of backup tasks of source ends; corresponding backup strategies are adopted according to different quantities of source ends and different data volumes, and a backup scheme adaptive to the set parameters is adopted according to different specific target ends of the backup task, so that the backup task can be completed most efficiently in the same time, and the experience of a user is improved.
Description
Technical Field
The invention belongs to the technical field of databases, and particularly relates to a method, a device, equipment and a medium for dynamically scheduling multiple backup tasks.
Background
In the prior art, database backup tasks have a great influence on the performance of online services, at present, conventional database backup tasks need to be scheduled and executed at the low peak of the online services, and the high peak of each online service is different and is influenced by factors such as the characteristics of the service, whether marketing activities exist, the use habits of users and the like; the execution time of the backup task is related to the data volume of the service, but in the prior art, the target end capability of each backup task is inconsistent, the maximum concurrent task number capable of being simultaneously borne is limited, the backup task must be completed in the same day, and the prior processing scheme has the problems of low processing efficiency and incapability of better scheduling the backup task and further causing the server to crash.
Disclosure of Invention
In view of this, an object of the present invention is to provide a method, an apparatus, a device and a medium for dynamically scheduling multiple backup tasks, which can improve the user experience of a user and improve the efficiency of backup tasks in the process of using a large-scale database. The specific scheme is as follows: in a first aspect, the present application discloses a method for dynamically scheduling multiple backup tasks, comprising the following steps:
(1) receiving backup tasks of a plurality of source ends;
(2) collecting the service peak time of a server where a database is located;
(3) acquiring the data quantity to be backed up of each database, calculating expected backup time according to the transmission speed and the backup speed, and setting the backup period, the backup frequency and the backup capacity of the server;
(4) sequencing the backup speed of the backup tasks from small to large, and arranging the backup tasks in a time period of a non-service peak period by combining the backup days specified by the source end task;
(5) when the backup tasks are arranged, the target ends of one or more backup tasks are judged: if the backup tasks are different at the target end, the backup tasks are arranged according to the time sequence in the time period of the non-service peak period; if the backup tasks with the same target end are the same, adjusting according to the storage capacity and the network speed limit, wherein the adjustment conditions are that the storage space of the backup tasks with the same target end does not exceed the total space size of the target end, the backup speeds of a plurality of tasks do not exceed the network speed limit of the target end, and the backup tasks are distributed after the two conditions are met until all the backup tasks are distributed.
Preferably, in step (2), the peak traffic period is a period of time in minutes during which the cpu usage or IOPS is greater than a set maximum value.
Preferably, step (4) further comprises: the backup speeds are ordered sequentially from small to large by using a greedy algorithm.
Preferably, in the step (5), if the total size of the target end space is exceeded, the user needs to be reminded to expand the storage space or to modify the backup task.
Preferably, in the step (5), the calculating process that the backup speed of the plurality of tasks does not exceed the network speed limit of the target end includes: if the sum of the backup speeds of the plurality of backup tasks in the same time period is less than or equal to the network speed limit, directly transmitting the backup tasks in an overlapping manner; if the sum of the backup tasks of the plurality of backup tasks in the same time period is greater than the network speed limit, combining and arranging the sum of the transmission speeds of all the backup tasks, arranging the backup tasks according to a rule that the sum of the transmission speeds is less than the idle network speed limit and the combined transmission speed is the most preferential, and transmitting the backup tasks after arranging.
Preferably, the method further comprises the following steps: and (4) re-collecting and updating the database service peak period and the backup data volume every period T, repeating the steps (1) to (5), and dynamically adjusting the backup starting time and the backup ending time according to the feedback result of the backup task.
In a second aspect, the present application discloses a multi-backup task dynamic scheduling apparatus, including:
the receiving module is used for receiving backup tasks of a plurality of source ends;
the acquisition module is used for acquiring the service peak period of the server where the database is located;
the setting module is used for collecting the data quantity of each database to be backed up, calculating expected backup time according to the transmission speed and the backup speed, and setting the backup period, the backup frequency and the backup capacity of the server;
the sequencing module is used for sequencing the backup speed of the backup tasks from small to large, and arranging the backup tasks in a time period of non-service peak period by combining the backup days specified by the source end task;
the judging module is used for judging the target ends of one or more backup tasks when the backup tasks are arranged: if the backup tasks are different at the target end, the backup tasks are arranged according to the time sequence in the time period of the non-service peak period; if the backup tasks with the same target end are the same, adjusting according to the storage capacity and the network speed limit, wherein the adjustment conditions are that the storage space of the backup tasks with the same target end does not exceed the total space size of the target end, the backup speeds of a plurality of tasks do not exceed the network speed limit of the target end, and the backup tasks are distributed after the two conditions are met until all the backup tasks are distributed.
In a third aspect, the present application discloses a multi-backup task dynamic scheduling device, including: a memory for storing a computer program; and the processor is used for executing the computer program to realize the multi-backup task dynamic scheduling method.
In a fourth aspect, the present application discloses a computer readable storage medium for storing a computer program; wherein the computer program when executed by the processor implements the aforementioned multi-backup task dynamic scheduling method.
Receiving backup tasks of a plurality of source ends; collecting the service peak time of a server where a database is located; acquiring the data quantity to be backed up of each database, calculating expected backup time according to the transmission speed and the backup speed, and setting the backup period, the backup frequency and the backup capacity of the server; sequencing the backup speed of the backup tasks from small to large, and arranging the backup tasks in a time period of a non-service peak period by combining the backup days specified by the source end task; when the backup tasks are arranged, the target ends of one or more backup tasks are judged: if the backup tasks are different at the target end, the backup tasks are arranged according to the time sequence in the time period of the non-service peak period; if the backup tasks with the same target end are the same, adjusting according to the storage capacity and the network speed limit, wherein the adjustment conditions are that the storage space of the backup tasks with the same target end does not exceed the total space size of the target end, the backup speeds of a plurality of tasks do not exceed the network speed limit of the target end, and the backup tasks are distributed after the two conditions are met until all the backup tasks are distributed. Therefore, the backup tasks of a plurality of source ends are received and the data quantity to be backed up is collected, and the backup time is calculated according to the transmission speed and the backup speed, so that the server parameters can be conveniently set; corresponding backup strategies can be adopted according to different quantities of source ends and different data volumes, and then a backup scheme adaptive to the set parameters is adopted according to different specific target ends of the backup task, so that the backup task can be efficiently carried out, the backup task can be completed most efficiently in the same time, and the experience of a user is improved.
In addition, the application also provides a method, a device, equipment and a medium for dynamically scheduling the multiple backup tasks, and the technical effect of the method, the device, the equipment and the medium corresponds to the technical effect of the system, and the details are not repeated here.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the provided drawings without creative efforts.
Fig. 1 is a flowchart of a method for dynamically scheduling multiple backup tasks according to the present application;
fig. 2 is a structural diagram of a multi-backup task dynamic scheduling device provided in the present application;
fig. 3 is a structural diagram of a multi-backup task dynamic scheduling device provided in the present application.
Detailed Description
In order that those skilled in the art will better understand the disclosure, the following detailed description will be given with reference to the accompanying drawings. It is to be understood that the embodiments described are only a few embodiments of the present application and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
The core of the application is to provide a multi-backup task dynamic scheduling method, which is used for improving the use experience of a user and enabling a backup task to be carried out efficiently. The core of the application is also to provide a method, a device, equipment and a medium for dynamically scheduling the multiple backup tasks.
In order that those skilled in the art will better understand the disclosure, the following detailed description will be given with reference to the accompanying drawings.
Fig. 1 is a flowchart of a method for dynamically scheduling multiple backup tasks according to the present application, and as shown in fig. 1, the method includes the following steps:
(1) receiving backup tasks of a plurality of source ends;
in the embodiment of the application, the server can receive backup tasks of a plurality of source ends, the source ends are equivalent to a plurality of users, and a backup request sent by the source ends is received in the server for the backup tasks, so that a backup relationship is established.
(2) Collecting the service peak time of a server where a database is located;
in the embodiment of the present application, the service peak period is a time period when the cpu usage rate or the IOPS is greater than a set maximum value, and is measured in minutes. For example, in 7 × 24 × 60 minutes per week, the cpu utilization rate is higher than that of cpu _ peak _ pct1 or the IOPS is greater than IOPS _ peak _ pct1, and the period is determined as the peak traffic period.
(3) Acquiring the data quantity to be backed up of each database, calculating expected backup time according to the transmission speed and the backup speed, and setting the backup period, the backup frequency and the backup capacity of the server;
in the embodiment of the application, an expected backup time can be calculated according to the collected data amount needing to be backed up, and the operation attribute of the server is set according to the backup time, so that unnecessary waste caused by excessive resource occupation during the backup task of the server can be reduced, for example, the data amount data1 needing to be backed up of each database, the network transmission speeds net1 of a source end and a target end, the speed back1 of backup data generation, the expected backup speed is backup _ v1 min (back1, net1), and the expected backup time is T1 data1/v1 min. Collecting backup retention time keep _ t1 of each database, time _ in _ week1 of weekly full backup and minimum value min _ intvl1 of days between two backup intervals, the occupied space capacity of backup at the target end is not more than
(4) Sequencing the backup speed of the backup tasks from small to large, and arranging the backup tasks in a time period of a non-service peak period by combining the backup days specified by the source end task;
in the embodiment of the application, the backup speed of the backup tasks is sequentially ordered from small to large and the backup timeliness is obtained, so that the backup tasks are prevented from being arranged in the time period of the business peak. The backup speed ordering is used for ordering the backup speeds from small to large by using a greedy algorithm, for example, the backup time of the first backup task s1 is t 1-15 minutes, the specified backup times are Monday, Monday and Saturday, the business peak period is 8:05 to 10:45 in the morning of Monday to Friday and 14:30 to 17:25 in the afternoon, 11:45 in Friday to 1:05 in Saturday, the backup time of the first backup task is 0: 00-0: 15 in Monday, 0: 00-0: 15 in Wednesday and 1: 05-1: 20 in Saturday.
(5) When the backup tasks are arranged, the target ends of one or more backup tasks are judged: if the backup tasks are different at the target end, the backup tasks are arranged according to the time sequence in the time period of the non-service peak period; if the backup tasks with the same target end are the same, adjusting according to the storage capacity and the network speed limit, wherein the adjustment conditions are that the storage space of the backup tasks with the same target end does not exceed the total space size of the target end, the backup speeds of a plurality of tasks do not exceed the network speed limit of the target end, and the backup tasks are distributed after the two conditions are met until all the backup tasks are distributed.
Specifically, in order to ensure that the backup task can be performed smoothly, the amount of the backup task needs to be determined before backup, and if the backup task exceeds the total space of the target end, a user needs to be reminded to expand the storage space or modify the backup task, and if the backup task does not exceed the total space of the target end, the backup task can be continued. For example, the target ends of the 1 st, 5 th, 8 th, and 15 th tasks are Des1, the storage capacity of Des1 is that stop ep 1 is 50T, the maximum space occupied by the backups of the 1 st, 5 th, 8 th, and 15 th tasks is max _ Des _ space1 is 15T, max _ Des _ space5 is 5T, max _ Des _ space8 is 3T, max _ Des _ space15 is 4T, the cumulative capacity (max _ Des _ space1+ max _ Des _ space5+ max _ Des _ space8+ max _ Des _ space15) stop ep 1, the capacity of Des1 can satisfy the backup requirements of multiple backup tasks, otherwise, the user is reminded to expand the storage space of Des1 or the user is reminded to modify the backup tasks.
Specifically, the calculation process that the backup speed of the plurality of tasks does not exceed the network speed limit of the target end includes: if the sum of the backup speeds of the plurality of backup tasks in the same time period is less than or equal to the network speed limit, directly transmitting the backup tasks in an overlapping manner; if the sum of the backup tasks of the plurality of backup tasks in the same time period is greater than the network speed limit, combining and arranging the sum of the transmission speeds of all the backup tasks, arranging the backup tasks according to a rule that the sum of the transmission speeds is less than the idle network speed limit and the combined transmission speed is the most preferential, and transmitting the backup tasks after arranging.
In the embodiment of the present application, it is further required to consider the problem of network speeds of the target end, that is, the server and the source end, during the setting of the network speed of the target end, the network speed is basically higher than that of a single backup task, and the transmission speeds of a plurality of backup tasks can be accommodated, but for normal operation, it is still required to determine whether the network speeds of the plurality of backup tasks exceed the network speed of the target end, and how to efficiently operate the plurality of backup tasks in the limited network speed of the target end is solved, for example, the backup network speed limit of Des1 is 1Gb/s, the backup speeds of the 1 st, 5 th, 8 th, and 15 th tasks are respectively backup _ v1 — 500Mb/s, backup _ v5 — 600Mb/s, backup _ v8 — 400Mb/s, and backup _ v15 — 800 Mb/s; the starting time and the ending time of the 1 st task backup are 0: 00-0: 15 of Monday, the starting time and the ending time of the 5 th task backup are 0: 00-0: 20 of Monday, the starting time and the ending time of the 8 th task backup are 0: 00-0: 30 of Monday, and the starting time and the ending time of the 15 th task backup are 1: 00-0: 20 of Monday; because the sum of the backup speeds of the four backup tasks is greater than the backup network speed limit of Des1, the four backup tasks are respectively arranged in different backup time periods according to the backup speed sequence, and the sum of the backup speeds of the multiple backup tasks at the same time is ensured not to exceed the backup speed. The 8 th, 1 st, 15 th and 5 th tasks are arranged in the sequence after the sorting, the corresponding 8 th task 400Mb/s is arranged at 0: 00-0: 30 of Monday, the 1 st task 500Mb/s < (1Gb-400Mb/s) is arranged at 0: 00-0: 15 of Monday; the 15 th task does not conflict with the time of other backup tasks, and the 5 th backup task 600Mb/s < (1Gb-400Mb/s) is arranged at 0: 15-0: 35 of Monday.
In this embodiment of the present application, after the completion of the backup scheduling process, the data of the source end needs to be subsequently acquired and adjusted in real time, which further includes the following steps: and (4) re-collecting and updating the database service peak period and the backup data volume every period T, repeating the steps (1) to (5), and dynamically adjusting the backup starting time and the backup ending time according to the feedback result of the backup task. Through subsequent real-time data acquisition, parameters can be adjusted according to the changing requirements of users and the backup task amount, the backup tasks are effectively operated and monitored, and the server is prevented from being down, so that backup services are influenced.
In the foregoing embodiments, the multi-backup task dynamic method is described in detail, and the present application also provides embodiments corresponding to the multi-backup task dynamic device. It should be noted that the present application describes the embodiments of the apparatus portion from two perspectives, one from the perspective of the function module and the other from the perspective of the hardware.
Therefore, the backup tasks of a plurality of source ends are received and the data quantity to be backed up is collected, and the backup time is calculated according to the transmission speed and the backup speed, so that the server parameters can be conveniently set; corresponding backup strategies can be adopted according to different quantities of source ends and different data volumes, and then a backup scheme adaptive to the set parameters is adopted according to different specific target ends of the backup task, so that the backup task can be efficiently carried out, the backup task can be completed most efficiently in the same time, and the experience of a user is improved.
Correspondingly, an embodiment of the present application further discloses a device for dynamically scheduling multiple backup tasks, as shown in fig. 2, based on the angle of the functional module, the device includes:
a receiving module 11, configured to receive backup tasks of multiple source ends;
the acquisition module 12 is used for acquiring the service peak period of the server where the database is located;
the setting module 13 is used for acquiring the data quantity to be backed up of each database, calculating expected backup time according to the transmission speed and the backup speed, and setting the backup period, the backup frequency and the backup capacity of the server;
the sequencing module 14 is configured to sequence the backup speeds of the backup tasks from small to large, and arrange the backup tasks in a time period of a non-service peak period in combination with the number of backup days specified by the source-end task;
the judging module 15, when the backup tasks are arranged, judges the target end of one or more backup tasks: if the backup tasks are different at the target end, the backup tasks are arranged according to the time sequence in the time period of the non-service peak period; if the backup tasks with the same target end are the same, adjusting according to the storage capacity and the network speed limit, wherein the adjustment conditions are that the storage space of the backup tasks with the same target end does not exceed the total space size of the target end, the backup speeds of a plurality of tasks do not exceed the network speed limit of the target end, and the backup tasks are distributed after the two conditions are met until all the backup tasks are distributed.
Since the embodiments of the apparatus portion and the method portion correspond to each other, please refer to the description of the embodiments of the method portion for the embodiments of the apparatus portion, which is not repeated here.
In some embodiments, the peak traffic period in the acquisition module 12 is a period of time, in minutes, when the cpu usage rate or IOPS is greater than a set maximum value.
In some specific embodiments, the sorting module 14 further includes: the backup speeds are ordered sequentially from small to large by using a greedy algorithm.
In some embodiments, in the determining module 15, if the total size of the target end space is exceeded, the user needs to be reminded to expand the storage space or to modify the backup task.
In some embodiments, in the determining module 15, the calculating process that the backup speed of the plurality of tasks does not exceed the network speed limit of the target end includes: if the sum of the backup speeds of the plurality of backup tasks in the same time period is less than or equal to the network speed limit, directly transmitting the backup tasks in an overlapping manner; if the sum of the backup tasks of the plurality of backup tasks in the same time period is greater than the network speed limit, combining and arranging the sum of the transmission speeds of all the backup tasks, arranging the backup tasks according to a rule that the sum of the transmission speeds is less than the idle network speed limit and the combined transmission speed is the most preferential, and transmitting the backup tasks after arranging.
In some embodiments, the method further comprises: and (4) re-collecting and updating the database service peak period and the backup data volume every period T, repeating the steps (1) to (5), and dynamically adjusting the backup starting time and the backup ending time according to the feedback result of the backup task.
An embodiment of the present application further provides an electronic device, as shown in fig. 3, which shows a schematic structural diagram of a multi-backup task dynamic scheduling device provided in an embodiment of the present application, and includes:
a memory 21 for storing a computer program;
the processor 22, configured to execute the computer program, may implement the following steps:
receiving backup tasks of a plurality of source ends; collecting the service peak time of a server where a database is located; acquiring the data quantity to be backed up of each database, calculating expected backup time according to the transmission speed and the backup speed, and setting the backup period, the backup frequency and the backup capacity of the server; sequencing the backup speed of the backup tasks from small to large, and arranging the backup tasks in a time period of a non-service peak period by combining the backup days specified by the source end task; when the backup tasks are arranged, the target ends of one or more backup tasks are judged: if the backup tasks are different at the target end, the backup tasks are arranged according to the time sequence in the time period of the non-service peak period; if the backup tasks with the same target end are the same, adjusting according to the storage capacity and the network speed limit, wherein the adjustment conditions are that the storage space of the backup tasks with the same target end does not exceed the total space size of the target end, the backup speeds of a plurality of tasks do not exceed the network speed limit of the target end, and the backup tasks are distributed after the two conditions are met until all the backup tasks are distributed.
The embodiment of the application also provides a computer readable storage medium for storing a computer program; wherein the computer program when executed by the processor is operable to perform the steps of:
receiving backup tasks of a plurality of source ends; collecting the service peak time of a server where a database is located; acquiring the data quantity to be backed up of each database, calculating expected backup time according to the transmission speed and the backup speed, and setting the backup period, the backup frequency and the backup capacity of the server; sequencing the backup speed of the backup tasks from small to large, and arranging the backup tasks in a time period of a non-service peak period by combining the backup days specified by the source end task; when the backup tasks are arranged, the target ends of one or more backup tasks are judged: if the backup tasks are different at the target end, the backup tasks are arranged according to the time sequence in the time period of the non-service peak period; if the backup tasks with the same target end are the same, adjusting according to the storage capacity and the network speed limit, wherein the adjustment conditions are that the storage space of the backup tasks with the same target end does not exceed the total space size of the target end, the backup speeds of a plurality of tasks do not exceed the network speed limit of the target end, and the backup tasks are distributed after the two conditions are met until all the backup tasks are distributed.
The computer-readable storage medium may include: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.
For a description of a relevant part in a multi-backup task dynamic scheduling method, an apparatus, a device, and a medium provided in the embodiments of the present application, reference may be made to a detailed description of a corresponding part in the multi-backup task dynamic scheduling method provided in the embodiments of the present application, and details are not repeated here.
It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Furthermore, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include elements inherent in the list. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element. In addition, parts of the above technical solutions provided in the embodiments of the present application, which are consistent with the implementation principles of corresponding technical solutions in the prior art, are not described in detail so as to avoid redundant description.
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the application. Thus, the present application is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (9)
1. A multi-backup task dynamic scheduling method is characterized by comprising the following steps:
(1) receiving backup tasks of a plurality of source ends;
(2) collecting the service peak time of a server where a database is located;
(3) acquiring the data quantity to be backed up of each database, calculating expected backup time according to the transmission speed and the backup speed, and setting the backup period, the backup frequency and the backup capacity of the server;
(4) sequencing the backup speed of the backup tasks from small to large, and arranging the backup tasks in a time period of a non-service peak period by combining the backup days specified by the source end task;
(5) when the backup tasks are arranged, the target ends of one or more backup tasks are judged: if the backup tasks are different at the target end, the backup tasks are arranged according to the time sequence in the time period of the non-service peak period; if the backup tasks with the same target end are the same, adjusting according to the storage capacity and the network speed limit, wherein the adjustment conditions are that the storage space of the backup tasks with the same target end does not exceed the total space size of the target end, the backup speeds of a plurality of tasks do not exceed the network speed limit of the target end, and the backup tasks are distributed after the two conditions are met until all the backup tasks are distributed.
2. The method for dynamically scheduling multiple backup tasks according to claim 1, wherein: in the step (2), the service peak period is a time period when the cpu utilization rate or the IOPS is greater than a set maximum value, and is in units of minutes.
3. The method for dynamically scheduling multiple backup tasks according to claim 1, wherein: in the step (4), the method further comprises the following steps: the backup speeds are ordered sequentially from small to large by using a greedy algorithm.
4. The method for dynamically scheduling multiple backup tasks according to claim 1, wherein: in the step (5), if the total space size of the target end is exceeded, the user needs to be reminded to expand the storage space or to modify the backup task.
5. The method for dynamically scheduling multiple backup tasks according to claim 1, wherein: in the step (5), the calculation process that the backup speed of the plurality of tasks does not exceed the network speed limit of the target end comprises the following steps: if the sum of the backup speeds of the plurality of backup tasks in the same time period is less than or equal to the network speed limit, directly transmitting the backup tasks in an overlapping manner; if the sum of the backup tasks of the plurality of backup tasks in the same time period is greater than the network speed limit, combining and arranging the sum of the transmission speeds of all the backup tasks, arranging the backup tasks according to a rule that the sum of the transmission speeds is less than the idle network speed limit and the combined transmission speed is the most preferential, and transmitting the backup tasks after arranging.
6. The method for dynamically scheduling multiple backup tasks according to claim 1, wherein: further comprising the steps of: and (4) re-collecting and updating the database service peak period and the backup data volume every period T, repeating the steps (1) to (5), and dynamically adjusting the backup starting time and the backup ending time according to the feedback result of the backup task.
7. A multi-backup task dynamic scheduling apparatus, comprising:
the receiving module is used for receiving backup tasks of a plurality of source ends;
the acquisition module is used for acquiring the service peak period of the server where the database is located;
the setting module is used for collecting the data quantity of each database to be backed up, calculating expected backup time according to the transmission speed and the backup speed, and setting the backup period, the backup frequency and the backup capacity of the server;
the sequencing module is used for sequencing the backup speed of the backup tasks from small to large, and arranging the backup tasks in a time period of non-service peak period by combining the backup days specified by the source end task;
the judging module is used for judging the target ends of one or more backup tasks when the backup tasks are arranged: if the backup tasks are different at the target end, the backup tasks are arranged according to the time sequence in the time period of the non-service peak period; if the backup tasks with the same target end are the same, adjusting according to the storage capacity and the network speed limit, wherein the adjustment conditions are that the storage space of the backup tasks with the same target end does not exceed the total space size of the target end, the backup speeds of a plurality of tasks do not exceed the network speed limit of the target end, and the backup tasks are distributed after the two conditions are met until all the backup tasks are distributed.
8. A multi-backup task dynamic scheduling device, comprising: a memory for storing a computer program; a processor for executing said computer program to implement the method for dynamic scheduling of multiple backup tasks according to any of claims 1 to 6.
9. A computer-readable storage medium for storing a computer program; wherein the computer program when executed by the processor implements a method for dynamic scheduling of multiple backup tasks according to any of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111071929.XA CN113778649A (en) | 2021-09-14 | 2021-09-14 | Multi-backup task dynamic scheduling method, device, equipment and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111071929.XA CN113778649A (en) | 2021-09-14 | 2021-09-14 | Multi-backup task dynamic scheduling method, device, equipment and medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113778649A true CN113778649A (en) | 2021-12-10 |
Family
ID=78843428
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111071929.XA Pending CN113778649A (en) | 2021-09-14 | 2021-09-14 | Multi-backup task dynamic scheduling method, device, equipment and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113778649A (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105183582A (en) * | 2015-07-30 | 2015-12-23 | 北京奇虎科技有限公司 | Data backup method and device |
| US20170364415A1 (en) * | 2016-06-21 | 2017-12-21 | International Business Machines Corporation | Optimizing data backup schedules |
| CN111966449A (en) * | 2020-07-17 | 2020-11-20 | 苏州浪潮智能科技有限公司 | Virtual machine backup management method, system, terminal and storage medium |
| CN112527559A (en) * | 2020-12-09 | 2021-03-19 | 广州技象科技有限公司 | Internet of things data backup method and device |
| CN113032185A (en) * | 2021-04-29 | 2021-06-25 | 中国工商银行股份有限公司 | Backup task management method, device, equipment and storage medium |
| CN113282437A (en) * | 2021-05-27 | 2021-08-20 | 济南浪潮数据技术有限公司 | Dynamic adjustment method, system and device for virtual machine backup speed limit |
-
2021
- 2021-09-14 CN CN202111071929.XA patent/CN113778649A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105183582A (en) * | 2015-07-30 | 2015-12-23 | 北京奇虎科技有限公司 | Data backup method and device |
| US20170364415A1 (en) * | 2016-06-21 | 2017-12-21 | International Business Machines Corporation | Optimizing data backup schedules |
| CN111966449A (en) * | 2020-07-17 | 2020-11-20 | 苏州浪潮智能科技有限公司 | Virtual machine backup management method, system, terminal and storage medium |
| CN112527559A (en) * | 2020-12-09 | 2021-03-19 | 广州技象科技有限公司 | Internet of things data backup method and device |
| CN113032185A (en) * | 2021-04-29 | 2021-06-25 | 中国工商银行股份有限公司 | Backup task management method, device, equipment and storage medium |
| CN113282437A (en) * | 2021-05-27 | 2021-08-20 | 济南浪潮数据技术有限公司 | Dynamic adjustment method, system and device for virtual machine backup speed limit |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9917782B2 (en) | Equitable distribution of excess shared-resource throughput capacity | |
| EP3920490B1 (en) | Adaptive resource allocation method and apparatus | |
| US9477513B2 (en) | Batch scheduling | |
| US10445208B2 (en) | Tunable, efficient monitoring of capacity usage in distributed storage systems | |
| US9385956B2 (en) | Compound token buckets for burst-mode admission control | |
| CA2915996C (en) | Burst mode control | |
| CN106407190A (en) | Event record querying method and device | |
| CN110888714A (en) | Container scheduling method, device and computer-readable storage medium | |
| CN111190696A (en) | Docker container deployment method, system, device and storage medium | |
| CN113094183B (en) | Training task creating method, device, system and medium of AI (Artificial Intelligence) training platform | |
| CN113626199B (en) | Management method and device of idle cloud computing resources, electronic equipment and storage medium | |
| US20120324466A1 (en) | Scheduling Execution Requests to Allow Partial Results | |
| CN113010315A (en) | Resource allocation method, resource allocation device and computer-readable storage medium | |
| CN113326285B (en) | Database table query method and device | |
| CN110990366A (en) | Index allocation method and device for improving performance of log system based on ES | |
| CN113778649A (en) | Multi-backup task dynamic scheduling method, device, equipment and medium | |
| CN108664322A (en) | Data processing method and system | |
| US9652766B1 (en) | Managing data stored in memory locations having size limitations | |
| CN115981852A (en) | Tenant-oriented server cluster resource quota adjusting method, device and equipment | |
| CN114647701A (en) | Load balancing method and device for distributed database, electronic equipment and medium | |
| CN117827453B (en) | Storage resource ordering method, device, equipment and storage medium | |
| CN115328666B (en) | Device scheduling method, system, electronic device and computer readable storage medium | |
| US9928469B1 (en) | Techniques for administrating finite life instances | |
| CN111030858B (en) | Data management method, system and related device for distributed multi-cluster system | |
| CN112598447A (en) | Order information processing method and device, electronic equipment and processor |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |