CN116627646A - Backup task self-load balancing method and device - Google Patents
Backup task self-load balancing method and device Download PDFInfo
- Publication number
- CN116627646A CN116627646A CN202310601081.XA CN202310601081A CN116627646A CN 116627646 A CN116627646 A CN 116627646A CN 202310601081 A CN202310601081 A CN 202310601081A CN 116627646 A CN116627646 A CN 116627646A
- Authority
- CN
- China
- Prior art keywords
- load
- backup
- list
- backup task
- media server
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 58
- 238000012544 monitoring process Methods 0.000 claims abstract description 57
- 238000004590 computer program Methods 0.000 claims description 20
- 230000015654 memory Effects 0.000 claims description 18
- 238000010586 diagram Methods 0.000 description 12
- 238000004891 communication Methods 0.000 description 8
- 230000006870 function Effects 0.000 description 7
- 238000012423 maintenance Methods 0.000 description 7
- 238000005516 engineering process Methods 0.000 description 6
- 238000003032 molecular docking Methods 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 230000003287 optical effect Effects 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 2
- 230000014509 gene expression Effects 0.000 description 2
- 230000007246 mechanism Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000010606 normalization Methods 0.000 description 2
- 238000011084 recovery Methods 0.000 description 2
- 230000002829 reductive effect Effects 0.000 description 2
- 230000003111 delayed effect Effects 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 230000000670 limiting effect Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 230000008520 organization Effects 0.000 description 1
- 238000004806 packaging method and process Methods 0.000 description 1
- 230000036961 partial effect Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 230000002441 reversible effect Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 239000000758 substrate Substances 0.000 description 1
- 238000010200 validation analysis Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
- G06F9/505—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals considering the load
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1446—Point-in-time backing up or restoration of persistent data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3003—Monitoring arrangements specially adapted to the computing system or computing system component being monitored
- G06F11/3006—Monitoring arrangements specially adapted to the computing system or computing system component being monitored where the computing system is distributed, e.g. networked systems, clusters, multiprocessor systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3051—Monitoring arrangements for monitoring the configuration of the computing system or of the computing system component, e.g. monitoring the presence of processing resources, peripherals, I/O links, software programs
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5083—Techniques for rebalancing the load in a distributed system
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- Quality & Reliability (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The disclosure provides a self-load balancing method for backup tasks, relates to the technical field of cloud computing, and can be applied to the technical field of finance. The method comprises the following steps: acquiring monitoring index data of each media server at regular time; determining the load capacity of each media server according to the monitoring index data and a preset threshold set; generating a near load list and a normal load list according to the load quantity; and load balancing the backup task according to the near load list and the normal load list. The disclosure also provides a backup task self-load balancing device, equipment, a storage medium and a program product.
Description
Technical Field
The present disclosure relates to the field of cloud computing technologies, and in particular, to the field of load balancing technologies, and more particularly, to a backup task self-load balancing method, device, apparatus, storage medium, and program product.
Background
In the current large service organization, netbackup (hereinafter referred to as NBU) is used for backup, and the backup architecture is generally that 1 Master Server management Server and a plurality of Media Server Media servers are added for client backup, where the Master Server management Server is used as a management and control center, and each Media Server bears the backup requirement of part of clients. Because the current NBU lacks a load balancing mechanism, when backup tasks are greatly increased, resources of individual Media Server Media servers are tensed and even exhausted due to uneven distribution of client backup strategies, so that partial backup strategies are delayed in execution and even beyond a time window, backup cannot be executed, and a large business risk exists.
It should be noted that the information disclosed in the above background section is only for enhancing understanding of the background of the present disclosure and thus may include information that does not constitute prior art known to those of ordinary skill in the art.
Disclosure of Invention
In view of the foregoing, the present disclosure provides a backup task self-load balancing method, apparatus, device, storage medium and program product that improve operation and maintenance efficiency.
According to a first aspect of the present disclosure, there is provided a backup task self-load balancing method, the method comprising:
acquiring monitoring index data of each media server at regular time;
determining the load capacity of each media server according to the monitoring index data and a preset threshold set;
generating a near load list and a normal load list according to the load quantity; and
and carrying out load balancing on the backup task according to the near load list and the normal load list.
According to an embodiment of the present disclosure, the generating the near load list and the normal load list according to the load amount includes:
determining the media server with the load capacity larger than a first preset threshold as a near-load list; and
and determining the medium server with the load capacity smaller than or equal to a first preset threshold value as a normal load list.
According to an embodiment of the present disclosure, the load balancing the backup task according to the near load list and the normal load list includes:
determining a target backup task of the near-load list according to a target keyword field;
performing backup strategy adjustment according to the normal load list and the target backup task; and
and executing the backup task according to the adjusted backup strategy.
According to an embodiment of the present disclosure, the determining the target backup task of the near-load list according to the target key field includes:
acquiring a backup task identifier to be operated according to the first keyword field; and
and acquiring the backup task identifier in the latest operation according to the second keyword field.
According to an embodiment of the present disclosure, the performing backup policy adjustment according to the normal load list and the target backup task includes:
determining a target media server of the target backup task, wherein the target media server is randomly selected from the normal load list; and
and adjusting the backup strategy according to the target media server.
According to an embodiment of the disclosure, the monitoring indexes include a disk read-write usage rate, a processor usage rate, a memory usage rate, a network read-write usage rate, and a tape drive usage rate, each of the monitoring indexes is provided with a preset threshold, and determining the load capacity of each media server according to the monitoring index data and a preset threshold set includes:
carrying out load judgment on the monitoring index data and a corresponding preset threshold value to determine the load capacity of each monitoring index; and
and determining the load capacity of the media server according to the load capacity of each monitoring index.
A second aspect of the present disclosure provides a backup task self-load balancing apparatus, the apparatus comprising:
the acquisition module is used for acquiring the monitoring index data of each medium server at regular time;
the load capacity determining module is used for determining the load capacity of each media server according to the monitoring index data and a preset threshold set;
the generation module is used for generating a near load list and a normal load list according to the load quantity; and
and the load balancing module is used for carrying out load balancing on the backup task according to the near load list and the normal load list.
According to an embodiment of the present disclosure, the generating module includes: a first determination sub-module and a second determination sub-module.
The first determining submodule is used for determining that the medium server with the load capacity larger than a first preset threshold value is a near-load list; and
and the second determining submodule is used for determining that the medium server with the load capacity smaller than or equal to the first preset threshold value is a normal load list.
According to an embodiment of the present disclosure, a load balancing module includes: the system comprises a third determination sub-module, a backup strategy adjustment sub-module and an execution sub-module.
A third determining submodule, configured to determine a target backup task of the near-load list according to a target keyword field;
the backup strategy adjustment sub-module is used for carrying out backup strategy adjustment according to the normal load list and the target backup task; and
and the execution sub-module is used for executing the backup task according to the adjusted backup strategy.
According to an embodiment of the present disclosure, the third determination submodule includes a first acquisition unit and a second acquisition unit.
The first acquisition unit is used for acquiring a backup task identifier to be operated according to the first keyword field; and
and the second acquisition unit is used for acquiring the latest running backup task identifier according to the second keyword field.
According to an embodiment of the present disclosure, a backup policy adjustment submodule includes: a determining unit and a backup strategy adjusting unit.
A determining unit, configured to determine a target media server of the target backup task, where the target media server is randomly selected in the normal load list; and
and the backup strategy adjusting unit is used for adjusting the backup strategy according to the target media server.
According to the embodiment of the disclosure, the monitoring indexes include disk read-write utilization rate, processor utilization rate, memory utilization rate, network read-write utilization rate and tape drive utilization rate, each monitoring index is provided with a preset threshold value, and an acquisition sub-module is used for acquiring the latest uploaded project document text according to a document archiving path; the load amount determining module includes: a fourth determination sub-module and a fifth determination sub-module.
A fourth determining submodule, configured to determine a load of each monitoring indicator by performing load determination on the monitoring indicator data and a corresponding preset threshold; and
and a fifth determining submodule, configured to determine the load capacity of the media server according to the load capacity of each monitoring indicator.
A third aspect of the present disclosure provides an electronic device, comprising: one or more processors; and a memory for storing one or more programs, wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to perform the backup task self-load balancing method described above.
A fourth aspect of the present disclosure also provides a computer-readable storage medium having stored thereon executable instructions that, when executed by a processor, cause the processor to perform the above-described backup task self-load balancing method.
A fifth aspect of the present disclosure also provides a computer program product comprising a computer program which, when executed by a processor, implements the above-described backup task self-load balancing method.
According to the self-load balancing method for the backup task, monitoring index data of each media server are obtained through timing; determining the load capacity of each media server according to the monitoring index data and a preset threshold set; generating a near load list and a normal load list according to the load quantity; and load balancing the backup task according to the near load list and the normal load list. By monitoring the load condition of the medium server at regular time, the automatic load balance of the backup task is realized, the operation and maintenance efficiency is improved, and the operation and maintenance cost is reduced.
Drawings
The foregoing and other objects, features and advantages of the disclosure will be more apparent from the following description of embodiments of the disclosure with reference to the accompanying drawings, in which:
FIG. 1 schematically illustrates an application scenario diagram of a backup task self-load balancing method, apparatus, device, storage medium, and program product according to an embodiment of the present disclosure;
FIG. 2 schematically illustrates an architecture diagram of a backup task self-load balancing apparatus provided in accordance with an embodiment of the present disclosure;
FIG. 3 schematically illustrates a flow chart of a backup task self-load balancing method provided in accordance with an embodiment of the present disclosure;
FIG. 4 schematically illustrates a flow chart of a backup task self-load balancing method provided in accordance with another embodiment of the present disclosure;
FIG. 5 schematically illustrates a flow chart of a backup task self-load balancing method provided in accordance with yet another embodiment of the present disclosure;
FIG. 6 schematically illustrates a block diagram of a backup task self-load balancing apparatus in accordance with an embodiment of the present disclosure; and
fig. 7 schematically illustrates a block diagram of an electronic device adapted to implement a backup task self-load balancing method in accordance with an embodiment of the present disclosure.
Detailed Description
Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. It should be understood that the description is only exemplary and is not intended to limit the scope of the present disclosure. In the following detailed description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the embodiments of the present disclosure. It may be evident, however, that one or more embodiments may be practiced without these specific details. In addition, in the following description, descriptions of well-known structures and techniques are omitted so as not to unnecessarily obscure the concepts of the present disclosure.
The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. The terms "comprises," "comprising," and/or the like, as used herein, specify the presence of stated features, steps, operations, and/or components, but do not preclude the presence or addition of one or more other features, steps, operations, or components.
All terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art unless otherwise defined. It should be noted that the terms used herein should be construed to have meanings consistent with the context of the present specification and should not be construed in an idealized or overly formal manner.
Where expressions like at least one of "A, B and C, etc. are used, the expressions should generally be interpreted in accordance with the meaning as commonly understood by those skilled in the art (e.g.," a system having at least one of A, B and C "shall include, but not be limited to, a system having a alone, B alone, C alone, a and B together, a and C together, B and C together, and/or A, B, C together, etc.).
The terms appearing in the embodiments of the present disclosure will be explained first:
master Server: the main management server of the NBU completes centralized control of the NBU backup system and can complete backup, recovery, unified strategy scheduling and other operation tasks.
Media Server: the media server of the NBU is a main body which is actually responsible for backup, data recovery and data transmission between the NBU and the tape library.
Anstable: the automatic operation and maintenance tool is an automatic operation and maintenance tool used for a Linux system, is connected with a client host based on an SSH protocol, and achieves functions of batch system configuration, batch program deployment, batch operation commands and the like.
Backup strategy: the backup method responsible for defining the backup host includes which directories or files to backup, when to backup, media Server to perform the backup, etc.
Backup tasks: an actually running backup process of the backup strategy is performed.
Based on the technical problems, an embodiment of the present disclosure provides a backup task self-load balancing method, where the method includes: acquiring monitoring index data of each media server at regular time; determining the load capacity of each media server according to the monitoring index data and a preset threshold set; generating a near load list and a normal load list according to the load quantity; and load balancing the backup task according to the near load list and the normal load list.
Fig. 1 schematically illustrates an application scenario diagram of a backup task self-load balancing method, apparatus, device, storage medium and program product according to an embodiment of the present disclosure.
As shown in fig. 1, an application scenario 100 according to this embodiment may include a backup task self-load balancing scenario. The network 104 is used as a medium to provide communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, among others.
A user may interact with the server 105 via the network 104 using the terminal devices 101, 102, 203 to receive or send messages or the like. Various communication client applications, such as shopping class applications, web browser applications, search class applications, instant messaging tools, mailbox clients, social platform software, etc. (by way of example only) may be installed on the terminal devices 101, 102, 103.
The terminal devices 101, 102, 103 may be a variety of electronic devices having a display screen and supporting web browsing, including but not limited to smartphones, tablets, laptop and desktop computers, and the like.
The server 105 may be a backup task self-load balancing server, in which the backup task self-load balancing method provided by the embodiment of the present disclosure is executed, a target project document is obtained in response to an uploading operation of the project document, and static scanning is performed on the target project document based on a text recognition algorithm to determine a target field of the target project document; and checking the normalization of the target project document according to the target field and a preset checking rule to determine the normalization of the target project document.
It should be noted that, the backup task self-load balancing method provided by the embodiments of the present disclosure may be generally performed by the server 105. Accordingly, the backup task self-load balancing apparatus provided by the embodiments of the present disclosure may be generally disposed in the server 105. The backup task self-load balancing method provided by the embodiments of the present disclosure may also be performed by a server or server cluster that is different from the server 105 and is capable of communicating with the terminal devices 101, 102, 103 and/or the server 105. Accordingly, the backup task self-load balancing apparatus provided by the embodiments of the present disclosure may also be provided in a server or a server cluster different from the server 105 and capable of communicating with the terminal devices 101, 102, 103 and/or the server 105.
It should be understood that the number of terminal devices, networks and servers in fig. 1 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.
It should be noted that, the backup task self-load balancing method and device determined by the embodiment of the present disclosure may be used in the technical field of cloud computing, or may be used in the technical field of finance, or may be used in any field other than the financial field, and the application field of the backup task self-load balancing method and device determined by the embodiment of the present disclosure is not limited.
Fig. 2 schematically illustrates a schematic diagram of a backup task self-load balancing apparatus provided according to an embodiment of the present disclosure. As shown in FIG. 2, the system mainly comprises a Master Server management Server, a Media Server, a backup host and a load balancing loading module. The management Server is responsible for managing a platform of the Media Server, the backup host, the backup strategy and the backup task, and can perform unified scheduling operations such as host data backup, strategy management, task management and the like; the media Server is responsible for executing the backup tasks of the backup host, and completing the transmission of all backup tasks scheduled to the media Server by the Master Server management Server. The media server is provided with a backup strategy, the backup strategy is used for defining a backup method of a backup host, the backup method comprises the steps of backing up which catalogues or files, backing up at what time, executing the backup media server and the like, and the backup host is managed by the media server.
The load balancing module in the embodiment of the disclosure realizes the docking of a Master Server and a Media Server based on an allowable automation tool, monitors all Media servers in an NBU platform based on technologies such as system monitoring and bpdbjobs, bpbackup, automatically adjusts backup tasks of the Media servers with heavier loads based on monitoring conditions, and realizes the load balancing of whole resources and the average distribution of backup tasks. The system specifically comprises a platform docking sub-module, a monitoring data acquisition sub-module, a load judging sub-module and a load balancing sub-module. The platform docking sub-module is used for docking the Master Server and the Media Server of the NBU platform in the device according to the user name and the password, and acquiring the operation authority of the operating system. The monitoring data acquisition sub-module is based on a dynamic polling mechanism, and periodically logs in to the operating system terminals of all Media servers when the device starts to run, and acquires the current values of all monitoring indexes based on nmon or other different technical means. The load judging sub-module is used for judging whether the current media server is overloaded. And the load balancing sub-module automatically balances the backup tasks in the overload medium server to other normal servers to complete the automatic adjustment of the backup strategy.
The backup task self-load balancing method according to the embodiments of the present disclosure will be described in detail below with reference to fig. 3 to 5 based on the application scenario described in fig. 1 and the system architecture described in fig. 2.
Fig. 3 schematically illustrates a flowchart of a backup task self-load balancing method provided according to an embodiment of the present disclosure. As shown in fig. 3, the backup task self-load balancing method of this embodiment includes operations S210 to S240, which may be performed by a server or other computing device.
In operation S210, monitoring index data of each media server is acquired at regular time.
According to an embodiment of the disclosure, the monitoring index includes a disk read-write usage, a processor usage, a memory usage, a network read-write usage, and a tape drive usage.
In one example, the data acquisition module dynamically polls the operating system terminals logged on all the media servers to acquire various monitoring index data. The monitoring index comprises the following five items:
a. disk IO case: based on iostat, the read-write condition and IO usage rate of the disk are checked.
CPU utilization: based on the us field of top, cpus(s), view CPU usage
c. Memory utilization rate: based on the top, the used field of Mem, the memory usage is checked.
d. Network IO case: based on iftop, network bandwidth is viewed.
e. Tape unit rate of usage: based on vmoprcmd, tape drive usage is viewed.
In operation S220, the capacity of each media server is determined according to the monitoring index data and a preset threshold set.
According to an embodiment of the disclosure, each of the monitoring indicators is provided with a preset threshold.
In one example, each monitoring index is correspondingly provided with a threshold value, and all preset threshold values form a preset threshold value set. And comparing the monitoring index data collected in the operation S210 with a corresponding preset threshold value to determine the load capacity of each media server. The load represents the actual running situation of the device at the current moment, the more the load is loaded by the server to bear the backup task, the greater the load is, and the specific process of determining the load can be seen from operation S221 and operation S222 shown in fig. 4.
In operation S230, a near load list and a normal load list are generated according to the load amount.
According to the embodiment of the disclosure, determining that the media server with the load capacity greater than the first preset threshold is a near-load list; and determining the medium server with the load capacity smaller than or equal to a first preset threshold value as a normal load list.
In one example, assuming that the initial load_count of each media server is 0, when the load_count is greater than a first preset threshold, determining that the server is near-loaded, that is, represents that the device is near-loaded, and should not bear the backup task any more; when the load_count is smaller than 2, the normal load is determined, that is, the current load state of the device is smaller, and the backup task can be carried continuously. Generating a near load list by near load equipment with load capacity larger than a first preset threshold value, and generating a normal load list by normal load equipment with load capacity smaller than or equal to the first preset threshold value.
In operation S240, load balancing is performed on the backup task according to the near load list and the normal load list.
In one example, to relieve backup pressure of media servers in the near load list, a backup policy is adjusted to load balance a portion of backup tasks on each media server in the near load list to the media servers in the normal load list, so as to achieve maximum utilization of backup resources.
According to the self-load balancing method for the backup task, monitoring index data of each media server are obtained through timing; determining the load capacity of each media server according to the monitoring index data and a preset threshold set; generating a near load list and a normal load list according to the load quantity; and load balancing the backup task according to the near load list and the normal load list. By monitoring the load condition of the medium server at regular time, the automatic load balance of the backup task is realized, the operation and maintenance efficiency is improved, and the operation and maintenance cost is reduced.
Fig. 4 schematically illustrates a flowchart of a backup task self-load balancing method according to another embodiment of the present disclosure. As shown in fig. 4, operation S220 includes operation S221 and operation S222.
In operation S221, load determination is performed on the monitoring index data and the corresponding preset threshold value, so as to determine the load amount of each monitoring index.
In operation S222, the load of the media server is determined according to the load of each monitoring index.
In one example, for backup execution, three indexes of disk IO, network IO and tape drive utilization rate are more depended, and when the three indexes reach the bottleneck, it can be determined that the Media Server is heavy in load, and the backup task cannot be increased. Judging each index of the Media Server medium Server, and adding one operation to the load_count when the cpu and memory usage index exceeds the preset threshold of the index; when the utilization rates of the disk IO, the network IO and the tape machine exceed the corresponding preset thresholds, adding two operations to the load_count; if the index is below the threshold, load_count is not modified. The load_count is reset to 0 after each poll is completed.
The process of backup task load balancing is described below in conjunction with fig. 5, where fig. 5 schematically illustrates a flowchart of a backup task self-load balancing method provided in accordance with a further embodiment of the present disclosure. As shown in fig. 5, operation S240 includes operations S241 to S243.
In operation S241, a target backup task of the near-load list is determined according to a target key field.
According to the embodiment of the disclosure, a backup task identifier to be operated is obtained according to a first keyword field; and acquiring the backup task identifier in the latest operation according to the second keyword field.
In one example, a target backup task of the near load list is first determined, the target backup task including a backup task currently waiting to run and a backup task in the latest run. Specifically, the operation authority of a Master Server operating system is obtained. Reading a near-load list, and for a near-load Media Server, using a bpbdjobs technology to acquire a backup task of the Media Server currently waiting to run according to a 'Stat' field; and acquiring the JobID of the latest backup task operated by the Media Server according to the field 'Started'.
In operation S242, a backup policy adjustment is performed according to the normal load list and the target backup task.
According to an embodiment of the present disclosure, determining a target media server of the target backup task, the target media server being randomly selected in the normal load list; and adjusting the backup strategy according to the target media server.
In operation S243, a backup task is performed according to the adjusted backup policy.
In one example, based on bpdbjobs and the obtained target backup task ID, the latest backup task is canceled. As the task is just executed, the influence of cancellation execution on backup is not great, and the current load condition of the Media Server can be further lightened. And carrying out backup strategy adjustment on the cancelled backup tasks to be executed, modifying a contrast field in the backup strategy into a Media Server name in a normal load list, specifically selecting the Media Server from the normal load list in a random_int random number form, and randomly carrying out each task once to avoid overhigh load caused by the fact that the tasks are distributed to the same Media Server, wherein other information is not modified. Saving the validation, thereby adjusting the Media Server executing the backup task. Based on bpbackup technology, the setting immediately executes backup for the adjusted backup policy. After load balancing, based on bpbdjobs technology, whether the state of the backup task is normal or not and whether the backup task is executed normally or not can be judged according to the strategy name.
Based on the backup task self-load balancing method, the disclosure also provides a backup task self-load balancing device. The device will be described in detail below in connection with fig. 6.
Fig. 6 schematically illustrates a block diagram of a backup task self-load balancing apparatus according to an embodiment of the present disclosure.
As shown in fig. 6, the backup task self-load balancing apparatus 600 of this embodiment includes an acquisition module 610, a load determination module 620, a generation module 630, and a load balancing module 640.
The acquisition module 610 is configured to acquire the monitor indicator data of each media server at regular time. In an embodiment, the obtaining module 610 may be configured to perform the operation S210 described above, which is not described herein.
The load determining module 620 is configured to determine a load of each media server according to the monitoring index data and a preset threshold set. In an embodiment, the load determining module 620 may be configured to perform the operation S220 described above, which is not described herein.
The generating module 630 is configured to generate a near load list and a normal load list according to the load amount. In an embodiment, the generating module 630 may be configured to perform the operation S230 described above, which is not described herein.
The load balancing module 640 is configured to load balance the backup task according to the near load list and the normal load list. In an embodiment, the load balancing module 640 may be configured to perform the operation S240 described above, which is not described herein.
According to an embodiment of the present disclosure, the generating module 630 includes: a first determination sub-module and a second determination sub-module.
And the first determining submodule is used for determining that the media server with the load capacity larger than a first preset threshold value is a near-load list. In an embodiment, the first determining sub-module may be used to perform the operation S230 described above, which is not described herein.
And the second determining submodule is used for determining that the medium server with the load capacity smaller than or equal to the first preset threshold value is a normal load list. In an embodiment, the second determining sub-module may be used to perform the operation S230 described above, which is not described herein.
According to an embodiment of the present disclosure, the load balancing module 640 includes: the system comprises a third determination sub-module, a backup strategy adjustment sub-module and an execution sub-module.
And the third determination submodule is used for determining the target backup task of the near-load list according to the target keyword field. In an embodiment, the third determining sub-module may be used to perform the operation S241 described above, which is not described herein.
And the backup strategy adjustment sub-module is used for carrying out backup strategy adjustment according to the normal load list and the target backup task. In an embodiment, the backup policy adjustment sub-module may be used to perform operation S242 described above, which is not described herein.
And the execution sub-module is used for executing the backup task according to the adjusted backup strategy. In an embodiment, the execution sub-module may be configured to execute the operation S243 described above, which is not described herein.
According to an embodiment of the present disclosure, the third determination submodule includes a first acquisition unit and a second acquisition unit.
And the first acquisition unit is used for acquiring the backup task identifier to be operated according to the first keyword field. In an embodiment, the first obtaining unit may be configured to perform the operation S241 described above, which is not described herein.
And the second acquisition unit is used for acquiring the latest running backup task identifier according to the second keyword field. In an embodiment, the second obtaining unit may be configured to perform the operation S242 described above, which is not described herein.
According to an embodiment of the present disclosure, a backup policy adjustment submodule includes: a determining unit and a backup strategy adjusting unit.
And the determining unit is used for determining a target media server of the target backup task, wherein the target media server is randomly selected from the normal load list. In an embodiment, the determining unit may be configured to perform the operation S242 described above, which is not described herein.
And the backup strategy adjusting unit is used for adjusting the backup strategy according to the target media server. In an embodiment, the backup policy adjustment unit may be used to perform the operation S242 described above, which is not described herein.
According to the embodiment of the disclosure, the monitoring indexes include disk read-write utilization rate, processor utilization rate, memory utilization rate, network read-write utilization rate and tape drive utilization rate, each monitoring index is provided with a preset threshold value, and an acquisition sub-module is used for acquiring the latest uploaded project document text according to a document archiving path; the load amount determining module includes: a fourth determination sub-module and a fifth determination sub-module.
And the fourth determining submodule is used for carrying out load judgment on the monitoring index data and the corresponding preset threshold value so as to determine the load capacity of each monitoring index. In an embodiment, the fourth determining sub-module may be used to perform the operation S221 described above, which is not described herein.
And a fifth determining submodule, configured to determine the load capacity of the media server according to the load capacity of each monitoring indicator. In an embodiment, the fifth determining sub-module may be used to perform the operation S222 described above, which is not described herein.
Any of the acquisition module 610, the load determination module 620, the generation module 630, and the load balancing module 640 may be combined in one module to be implemented, or any of them may be split into multiple modules, according to embodiments of the present disclosure. Alternatively, at least some of the functionality of one or more of the modules may be combined with at least some of the functionality of other modules and implemented in one module. According to embodiments of the present disclosure, at least one of the acquisition module 610, the load determination module 620, the generation module 630, and the load balancing module 640 may be implemented at least in part as hardware circuitry, such as a Field Programmable Gate Array (FPGA), a Programmable Logic Array (PLA), a system on a chip, a system on a substrate, a system on a package, an Application Specific Integrated Circuit (ASIC), or may be implemented in hardware or firmware in any other reasonable way of integrating or packaging the circuitry, or in any one of or a suitable combination of any of the three. Alternatively, at least one of the acquisition module 610, the load determination module 620, the generation module 630, and the load balancing module 640 may be at least partially implemented as a computer program module that, when executed, may perform the corresponding functions.
Fig. 7 schematically illustrates a block diagram of an electronic device adapted to implement a backup task self-load balancing method in accordance with an embodiment of the present disclosure.
As shown in fig. 7, an electronic device 900 according to an embodiment of the present disclosure includes a processor 901 that can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM) 902 or a program loaded from a storage portion 908 into a Random Access Memory (RAM) 903. The processor 901 may include, for example, a general purpose microprocessor (e.g., a CPU), an instruction set processor and/or an associated chipset and/or a special purpose microprocessor (e.g., an Application Specific Integrated Circuit (ASIC)), or the like. Processor 901 may also include on-board memory for caching purposes. Processor 901 may include a single processing unit or multiple processing units for performing the different actions of the method flows according to embodiments of the present disclosure.
In the RAM903, various programs and data necessary for the operation of the electronic device 900 are stored. The processor 901, the ROM902, and the RAM903 are connected to each other by a bus 904. The processor 901 performs various operations of the method flow according to the embodiments of the present disclosure by executing programs in the ROM902 and/or the RAM 903. Note that the program may be stored in one or more memories other than the ROM902 and the RAM 903. The processor 901 may also perform various operations of the method flow according to embodiments of the present disclosure by executing programs stored in the one or more memories.
According to an embodiment of the disclosure, the electronic device 900 may also include an input/output (I/O) interface 905, the input/output (I/O) interface 905 also being connected to the bus 904. The electronic device 900 may also include one or more of the following components connected to the I/O interface 905: an input section 906 including a keyboard, a mouse, and the like; an output portion 907 including a display such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and a speaker; a storage portion 908 including a hard disk or the like; and a communication section 909 including a network interface card such as a LAN card, a modem, or the like. The communication section 909 performs communication processing via a network such as the internet. The drive 910 is also connected to the I/O interface 905 as needed. A removable medium 911 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is installed as needed on the drive 910 so that a computer program read out therefrom is installed into the storage section 908 as needed.
The present disclosure also provides a computer-readable storage medium that may be embodied in the apparatus/device/system described in the above embodiments; or may exist alone without being assembled into the apparatus/device/system. The computer-readable storage medium carries one or more programs that, when executed, implement a backup task self-load balancing method according to an embodiment of the present disclosure.
According to embodiments of the present disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium, which may include, for example, but is not limited to: a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this disclosure, a computer-readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. For example, according to embodiments of the present disclosure, the computer-readable storage medium may include ROM902 and/or RAM903 and/or one or more memories other than ROM902 and RAM903 described above.
Embodiments of the present disclosure also include a computer program product comprising a computer program containing program code for performing the methods shown in the flowcharts. When the computer program product runs in a computer system, the program code is used for enabling the computer system to realize the backup task self-load balancing method provided by the embodiment of the disclosure.
The above-described functions defined in the system/apparatus of the embodiments of the present disclosure are performed when the computer program is executed by the processor 901. The systems, apparatus, modules, units, etc. described above may be implemented by computer program modules according to embodiments of the disclosure.
In one embodiment, the computer program may be based on a tangible storage medium such as an optical storage device, a magnetic storage device, or the like. In another embodiment, the computer program may also be transmitted, distributed, and downloaded and installed in the form of a signal on a network medium, via communication portion 909, and/or installed from removable medium 911. The computer program may include program code that may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the foregoing.
In such an embodiment, the computer program may be downloaded and installed from the network via the communication portion 909 and/or installed from the removable medium 911. The above-described functions defined in the system of the embodiments of the present disclosure are performed when the computer program is executed by the processor 901. The systems, devices, apparatus, modules, units, etc. described above may be implemented by computer program modules according to embodiments of the disclosure.
According to embodiments of the present disclosure, program code for performing computer programs provided by embodiments of the present disclosure may be written in any combination of one or more programming languages, and in particular, such computer programs may be implemented in high-level procedural and/or object-oriented programming languages, and/or assembly/machine languages. Programming languages include, but are not limited to, such as Java, c++, python, "C" or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, partly on a remote computing device, or entirely on the remote computing device or server. In the case of remote computing devices, the remote computing device may be connected to the user computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computing device (e.g., connected via the Internet using an Internet service provider).
The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
Those skilled in the art will appreciate that the features recited in the various embodiments of the disclosure and/or in the claims may be provided in a variety of combinations and/or combinations, even if such combinations or combinations are not explicitly recited in the disclosure. In particular, the features recited in the various embodiments of the present disclosure and/or the claims may be variously combined and/or combined without departing from the spirit and teachings of the present disclosure. All such combinations and/or combinations fall within the scope of the present disclosure.
The embodiments of the present disclosure are described above. However, these examples are for illustrative purposes only and are not intended to limit the scope of the present disclosure. Although the embodiments are described above separately, this does not mean that the measures in the embodiments cannot be used advantageously in combination. The scope of the disclosure is defined by the appended claims and equivalents thereof. Various alternatives and modifications can be made by those skilled in the art without departing from the scope of the disclosure, and such alternatives and modifications are intended to fall within the scope of the disclosure.
Claims (10)
1. A backup task self-load balancing method, the method comprising:
acquiring monitoring index data of each media server at regular time;
determining the load capacity of each media server according to the monitoring index data and a preset threshold set;
generating a near load list and a normal load list according to the load quantity; and
and carrying out load balancing on the backup task according to the near load list and the normal load list.
2. The method of claim 1, wherein the generating a near load list and a normal load list from the load amount comprises:
determining the media server with the load capacity larger than a first preset threshold as a near-load list; and
and determining the medium server with the load capacity smaller than or equal to a first preset threshold value as a normal load list.
3. The method of claim 1, wherein load balancing backup tasks according to the near load list and the normal load list comprises:
determining a target backup task of the near-load list according to a target keyword field;
performing backup strategy adjustment according to the normal load list and the target backup task; and
and executing the backup task according to the adjusted backup strategy.
4. The method of claim 3, wherein the determining the target backup task of the near load list based on the target key field comprises:
acquiring a backup task identifier to be operated according to the first keyword field; and
and acquiring the backup task identifier in the latest operation according to the second keyword field.
5. The method of claim 4, wherein said performing backup policy adjustment based on said normal load list and said target backup task comprises:
determining a target media server of the target backup task, wherein the target media server is randomly selected from the normal load list; and
and adjusting the backup strategy according to the target media server.
6. The method of any one of claims 1 to 5, wherein the monitoring metrics include disk read-write usage, processor usage, memory usage, network read-write usage, and tape drive usage, each of the monitoring metrics is provided with a preset threshold, and determining the capacity of each media server based on the monitoring metrics data and a preset threshold set includes:
carrying out load judgment on the monitoring index data and a corresponding preset threshold value to determine the load capacity of each monitoring index; and
and determining the load capacity of the media server according to the load capacity of each monitoring index.
7. A backup task self-load balancing apparatus, the apparatus comprising:
the acquisition module is used for acquiring the monitoring index data of each medium server at regular time;
the load capacity determining module is used for determining the load capacity of each media server according to the monitoring index data and a preset threshold set;
the generation module is used for generating a near load list and a normal load list according to the load quantity; and
and the load balancing module is used for carrying out load balancing on the backup task according to the near load list and the normal load list.
8. An electronic device, comprising:
one or more processors;
storage means for storing one or more programs,
wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to perform the backup task self-load balancing method of any of claims 1-6.
9. A computer readable storage medium having stored thereon executable instructions which when executed by a processor cause the processor to perform the backup task self-load balancing method of any one of claims 1 to 6.
10. A computer program product comprising a computer program which when executed by a processor implements a backup task self-load balancing method according to any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310601081.XA CN116627646A (en) | 2023-05-25 | 2023-05-25 | Backup task self-load balancing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310601081.XA CN116627646A (en) | 2023-05-25 | 2023-05-25 | Backup task self-load balancing method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116627646A true CN116627646A (en) | 2023-08-22 |
Family
ID=87637772
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310601081.XA Pending CN116627646A (en) | 2023-05-25 | 2023-05-25 | Backup task self-load balancing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116627646A (en) |
-
2023
- 2023-05-25 CN CN202310601081.XA patent/CN116627646A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11487562B2 (en) | Rolling resource credits for scheduling of virtual computer resources | |
| US8443363B1 (en) | Coordinated virtualization activities | |
| US8661447B1 (en) | Method and apparatus for managing execution of a plurality of computer tasks based on availability of computer resources | |
| US9473413B1 (en) | Dynamic throttle of network traffic | |
| US9870269B1 (en) | Job allocation in a clustered environment | |
| CN102541460B (en) | Multiple disc management method and equipment | |
| US11507417B2 (en) | Job scheduling based on job execution history | |
| CN104937584A (en) | Providing optimized quality of service to prioritized virtual machines and applications based on quality of shared resources | |
| US10572319B2 (en) | Optimization of message oriented middleware monitoring in heterogenenous computing environments | |
| US8539192B2 (en) | Execution of dataflow jobs | |
| US10917459B2 (en) | Server load management for data migration | |
| US9588813B1 (en) | Determining cost of service call | |
| US10250480B2 (en) | Polling parameter adjustment | |
| US11126506B2 (en) | Systems and methods for predictive data protection | |
| US20170250918A1 (en) | Optimized resource provisioning | |
| US11418583B2 (en) | Transaction process management by dynamic transaction aggregation | |
| Liao et al. | A resource provision strategy for software-as-a-service in cloud computing | |
| US10740143B2 (en) | Controlling asynchronous tasks | |
| CN109800060B (en) | Cloud platform system, management method, device and storage medium | |
| CN116627646A (en) | Backup task self-load balancing method and device | |
| US20230185631A1 (en) | Embedded capacity-computer module for microservice load balancing and distribution | |
| CN113076175B (en) | Memory sharing method and device for virtual machine | |
| US20220188167A1 (en) | System and method to adapt memory usage of containerized workspaces | |
| US20180322564A1 (en) | System and Method for Accessing and Evaluating Orders in an Order Processing and Fulfillment System | |
| CN113138772B (en) | Construction method and device of data processing platform, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |