CN108270837B - Distributed task scheduling method and system using idle resources - Google Patents
Distributed task scheduling method and system using idle resources Download PDFInfo
- Publication number
- CN108270837B CN108270837B CN201710003887.3A CN201710003887A CN108270837B CN 108270837 B CN108270837 B CN 108270837B CN 201710003887 A CN201710003887 A CN 201710003887A CN 108270837 B CN108270837 B CN 108270837B
- Authority
- CN
- China
- Prior art keywords
- task
- target
- equipment
- time
- resource scheduling
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/60—Scheduling or organising the servicing of application requests, e.g. requests for application data transmissions using the analysis and optimisation of the required network resources
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/54—Presence management, e.g. monitoring or registration for receipt of user log-on information, or the connection status of the users
Abstract
The invention provides a distributed task scheduling method and a distributed task scheduling system using idle resources, wherein the method comprises the following steps: acquiring equipment information of target equipment and judging the online state of the target equipment; initializing target equipment, and generating an available equipment list according to the online state of the target equipment; according to the predicted execution time of each task to be distributed and the idle time of each target device in the available device list, each task is sent to each target device with the matched time according to a task distribution strategy; the invention can accurately match the target task and the execution equipment according to the task execution time consumption estimation method, realizes the utilization of idle equipment resources at different time periods, greatly saves the operation cost, effectively realizes the automatic task execution under the condition that a research and development team does not need to purchase a large amount of special equipment, and collects the execution result.
Description
Technical Field
The invention relates to the technical field of computers, in particular to a distributed task scheduling method and system utilizing idle resources.
Background
During the development of software, many automation tasks (e.g., compiling and packaging in continuous integration, static code inspection, automatic testing, etc.) require a lot of related supporting resources to complete execution, and thus a lot of equipment needs to be purchased to support. However, in reality, many devices are idle during idle time, and therefore how to fully utilize available resources during idle time becomes a problem to be solved.
The existing distributed equipment scheduling scheme is mostly composed of a main control machine and an execution machine. The main control computer schedules the task execution of the execution machine by sending the execution agent instruction to the execution machine and executing the agent instruction.
As shown in fig. 1, the existing automated task machine scheduling method:
(1) the main control computer is responsible for task and machine scheduling, and all the execution machines are responsible for task execution;
(2) the executive machine is used as a Slave of the master control machine, and is controlled by the master control machine in the whole time period;
(3) the tasks are initiated by the main control computer actively according to needs, and the executive machine receives and executes the tasks.
However, the existing automatic task machine scheduling method has the following problems:
the main control computer and the execution machine must be bound together to provide services for users, the execution machine must be completely controlled, once idle resources are used for supporting the scheduling of the main control computer, the autonomy is lost to a certain extent, and some kinds of idle resources (such as temporary idle resources, personal computer resources and the like) are not suitable for receiving the complete scheduling of the main control computer.
Disclosure of Invention
In view of the above, the present invention provides a distributed task scheduling method and system using idle resources, which distribute tasks according to idle time of devices, effectively improve the utilization rate of device resources, and save the operation cost.
The technical scheme of the invention is to provide a distributed task scheduling method using idle resources, wherein the method comprises the following steps:
acquiring equipment information of target equipment and judging the online state of the target equipment;
initializing the target equipment, and generating an available equipment list according to the online state of the target equipment;
distributing each task to each target device with matched time according to a task distribution strategy according to the expected execution time of each task to be distributed and the idle time of each target device in the available device list, wherein the task distribution strategy is as follows:
the idle time of the target device and the number of executable atomic tasks in the unit time of the target device are more than or equal to the estimated execution time of the tasks to be distributed/the execution time of the atomic tasks,
the atomic task is a unit preset task of the task, the execution time of the atomic task represents the time for executing the unit preset task in unit time, and the idle time of the target equipment represents the idle time of the target equipment;
the respective target devices automatically execute the distributed tasks.
Optionally, the device information includes:
the IP address, the device type and the device holder information of the target device.
Optionally, initializing the target device includes:
and detecting the connectivity of each target device, authenticating and accessing the target devices according to the device information, and installing a resource scheduling client on each target device.
Optionally, the determining the presence information of the target device includes:
and the resource scheduling engine sends detection information to the resource scheduling client regularly, and judges the online state of the target equipment according to the response of the resource scheduling client.
Optionally, the determining the online status of the target device further includes:
the resource scheduling engine and the resource scheduling client are connected in a long way by establishing TCP, and regularly send detection information to the resource scheduling client;
if the resource scheduling engine can normally receive a response, the resource scheduling client is in an online state;
and if the resource scheduling engine cannot normally receive the response, the resource scheduling client is in an offline state.
Optionally, initializing the target device, and generating an available device list according to the online state of the target device, further including:
and updating the available equipment list in real time according to the online state of the target equipment.
Optionally, the task distribution policy further includes:
if the task state is the execution failure state, entering a failure task processing flow, wherein the failure task processing flow comprises the following steps:
judging whether the frequency of the failed tasks exceeds a preset threshold value or not;
if the frequency of the failed task is greater than or equal to the preset threshold value, the failed task is moved to an abnormal task list;
and if the number of times of the failed task is less than the preset threshold value, distributing the task again.
The invention also provides a distributed task scheduling system using idle resources, comprising:
the information acquisition device is used for acquiring the equipment information of the target equipment and judging the online state of the target equipment;
the available information device is used for initializing the target equipment and generating an available equipment list according to the online state of the target equipment;
the task distribution device sends each task to each target device with matched time according to a task distribution strategy according to the expected execution time of each task to be distributed and the idle time of each target device in the available device list, wherein the task distribution strategy is as follows:
the idle time of the target device and the number of executable atomic tasks in the unit time of the target device are more than or equal to the estimated execution time of the tasks to be distributed/the execution time of the atomic tasks,
the atomic task is a unit preset task of the task, the execution time of the atomic task represents the time for executing the unit preset task in unit time, and the idle time of the target equipment represents the idle time of the target equipment;
and the task execution device automatically executes the distributed tasks by the target equipment.
Optionally, the device information includes:
the IP address, the device type and the device holder information of the target device.
Optionally, the available information means is for:
and detecting the connectivity of each target device, authenticating and accessing the target devices according to the device information, and installing a resource scheduling client on each target device.
Optionally, the information acquiring apparatus is configured to:
and the resource scheduling engine sends detection information to the resource scheduling client regularly, and judges the online state of the target equipment according to the response of the resource scheduling client.
Optionally, the information acquiring apparatus is further configured to:
the resource scheduling engine and the resource scheduling client are connected in a long way by establishing TCP, and regularly send detection information to the resource scheduling client;
if the resource scheduling engine can normally receive a response, the resource scheduling client is in an online state;
and if the resource scheduling engine cannot normally receive the response, the resource scheduling client is in an offline state.
Optionally, the available information apparatus further includes:
and updating the available equipment list in real time according to the online state of the target equipment.
Optionally, the task distribution policy further includes:
if the task state is the execution failure state, entering a failure task processing flow, wherein the failure task processing flow comprises the following steps:
judging whether the frequency of the failed tasks exceeds a preset threshold value or not;
if the frequency of the failed task is larger than or equal to the threshold value, the failed task is moved to an abnormal task list;
and if the number of times of the failed task is less than the preset threshold value, distributing the task again.
The invention also provides a distributed task scheduling system using idle resources, comprising: a memory to store instructions and a processor to execute the instructions to:
acquiring equipment information of target equipment and judging the online state of the target equipment;
initializing the target equipment, and generating an available equipment list according to the online state of the target equipment;
distributing each task to each target device with matched time according to a task distribution strategy according to the expected execution time of each task to be distributed and the idle time of each target device in the available device list, wherein the task distribution strategy is as follows:
the idle time of the target device and the number of executable atomic tasks in the unit time of the target device are more than or equal to the estimated execution time of the tasks to be distributed/the execution time of the atomic tasks,
the atomic task is a unit preset task of the task, the execution time of the atomic task represents the time for executing the unit preset task in unit time, and the idle time of the target equipment represents the idle time of the target equipment;
the respective target devices automatically execute the distributed tasks.
To achieve the above object, according to still another aspect of embodiments of the present invention, there is provided an electronic apparatus including: one or more processors; the storage device is used for storing one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors realize the distributed task scheduling method utilizing idle resources provided by the embodiment of the invention.
To achieve the above object, according to another aspect of the embodiments of the present invention, there is provided a computer readable medium having a computer program stored thereon, where the computer program is used to implement a method for distributed task scheduling using idle resources according to the embodiments of the present invention when the computer program is executed by a processor.
By the distributed task scheduling method and system using the idle resources, the target task and the execution equipment can be accurately matched according to a task execution time consumption estimation method, the task can be accurately issued to the target equipment in an activated state through automatic detection of the equipment state, the idle equipment resources can be utilized in different time periods, the operation cost is greatly saved, and a research and development team can effectively realize automatic task execution (for example, compiling and packaging in continuous integration, static code inspection and automatic test) and collect execution results under the condition that a large amount of special equipment is not required to be purchased.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts. In the drawings:
FIG. 1 is a schematic diagram illustrating an application scenario of an automated task machine scheduling method in the prior art;
FIG. 2 is a schematic diagram of an application scenario of the distributed task scheduling method and system using idle resources according to the present invention;
FIG. 3 is a flowchart illustrating a distributed task scheduling method using idle resources according to an embodiment of the present invention;
FIG. 4 is a diagram illustrating a distributed task scheduling system using idle resources according to an embodiment of the present invention;
FIG. 5 is a diagram illustrating a distributed task scheduling system using idle resources according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the embodiments of the present invention are further described in detail below with reference to the accompanying drawings. The exemplary embodiments and descriptions of the present invention are provided to explain the present invention, but not to limit the present invention.
As will be appreciated by one skilled in the art, embodiments of the present invention may be embodied as a system, apparatus, device, method, or computer program product. Accordingly, the present disclosure may be embodied in the form of: entirely hardware, entirely software (including firmware, resident software, micro-code, etc.), or a combination of hardware and software.
Application scenarios
Reference is first made to fig. 2, which illustrates an application scenario in which embodiments of the present invention may be implemented. The scenario shown in fig. 2 includes an idle resource registration interface 21 of the target device, a resource initialization apparatus 22, a database 23, a resource scheduling engine 24, and a resource scheduling client 25. The idle resource registration interface 21 of the target device is a Web interface interacting with the user, and is used for the user to enter the IP, the device type, and the device holder information of the target device. And can display the initialized state of the equipment which has been entered, the resource initialization device 22 receives the IP of the target equipment, the equipment type, the equipment holder information which are entered by the user through the idle resource registration interface, and stores the information to the database 23, after the target resource device completes the initialization process, the initialization status information is pushed to the idle resource registration interface 21, the resource scheduling client 25 is a module installed on the target device, and is used as an entry for entering the idle period of the device, and feeds back the online status information of the target device, executing an automation task on the target device, collecting task result data, the resource scheduling engine 24 is the core module of the overall system, and automatically matching the task with the target equipment according to the initialization state information, the equipment online state information and the idle time information of the idle resource equipment, and executing task distribution operation.
Exemplary method
In the following, a picture display method according to an exemplary embodiment of the present invention is described with reference to fig. 3 in conjunction with the application scenario of fig. 2. It should be noted that the above-described application scenarios are merely illustrative for the convenience of understanding the spirit and principles of the present invention,
the embodiments of the invention are not limited in any way in this respect. Rather, embodiments of the present invention may be applied to any scenario where applicable.
For example, referring to fig. 3, a flowchart of a distributed task scheduling method using idle resources according to an embodiment of the present invention is shown.
As shown, the method includes:
step S301: acquiring equipment information of target equipment and judging the online state of the target equipment;
step S302: initializing the target equipment, and generating an available equipment list according to the online state of the target equipment;
step S303: distributing each task to each target device with matched time according to a task distribution strategy according to the expected execution time of each task to be distributed and the idle time of each target device in the available device list, wherein the task distribution strategy is as follows:
the idle time of the target device and the number of executable atomic tasks in the unit time of the target device are more than or equal to the estimated execution time of the tasks to be distributed/the execution time of the atomic tasks,
the atomic task is a unit preset task of the task, the execution time of the atomic task represents the time for executing the unit preset task in unit time, and the idle time of the target equipment represents the idle time of the target equipment;
step S304: the respective target devices automatically execute the distributed tasks.
Optionally, the device information includes:
the IP address, the device type and the device holder information of the target device.
Optionally, initializing the target device includes:
and detecting the connectivity of each target device, authenticating and accessing the target devices according to the device information, and installing a resource scheduling client on each target device.
Optionally, the determining the presence information of the target device includes:
and the resource scheduling engine sends detection information to the resource scheduling client regularly, and judges the online state of the target equipment according to the response of the resource scheduling client.
Optionally, the determining the online status of the target device further includes:
the resource scheduling engine and the resource scheduling client are connected in a long way by establishing TCP, and regularly send detection information to the resource scheduling client;
if the resource scheduling engine can normally receive a response, the resource scheduling client is in an online state;
and if the resource scheduling engine cannot normally receive the response, the resource scheduling client is in an offline state.
Optionally, initializing the target device, and generating an available device list according to the online state of the target device, further including: and updating the available equipment list in real time according to the online state of the target equipment.
Optionally, the task distribution policy further includes:
if the task state is the execution failure state, entering a failure task processing flow, wherein the failure task processing flow comprises the following steps:
judging whether the frequency of the failed tasks exceeds a preset threshold value or not;
if the frequency of the failed task is greater than or equal to the preset threshold value, the failed task is moved to an abnormal task list;
and if the number of times of the failed task is less than the preset threshold value, distributing the task again.
Examples
The present invention will be described in detail with reference to a specific embodiment, but it should be noted that the specific embodiment is only for better describing the present invention and should not be construed as limiting the present invention.
Fig. 3 is a schematic flow chart of a distributed task scheduling method using idle resources according to an embodiment of the present invention, where the method includes:
step S301: and acquiring the equipment information of the target equipment and judging the online state of the target equipment.
Specifically, the research and development personnel can enter the IP address, the equipment type and the equipment holder information of the target equipment through the idle resource registration interface.
Step S302: initializing the target equipment, and generating an available equipment list according to the online state of the target equipment.
In an embodiment of the present invention, initializing the target device by using the resource initialization apparatus includes:
and detecting the connectivity of each target device, authenticating and accessing the target devices according to the device information, and installing a resource scheduling client on each target device. In an embodiment of the present invention, the initialized state of the target device that has been entered may be displayed through the idle resource registration interface.
After the resource scheduling client is installed on each target device, the online state information of the target device needs to be judged. Specifically, before the task is distributed, the resource scheduling engine screens out target devices meeting the task distribution condition from the database according to the device online state information, generates an available device list and stores the device online state information in the database in real time. In addition, the available device list is updated in real time according to the online state of the target device.
Specifically, the resource scheduling engine sends detection information to the resource scheduling client at regular time, and judges the online state of the target device according to the response of the resource scheduling client. In particular, each target device may adopt a manner of closing the resource scheduling client in a time-sharing manner to disconnect the association between the target device and the master controller. And under the state that the target equipment closes the resource scheduling client, the resource scheduling engine cannot distribute the tasks to the target equipment. In addition, a TCP long connection is established between the resource scheduling engine and the resource scheduling client, and the resource scheduling engine sends detection information to the resource scheduling client at regular time; if the resource scheduling engine can normally receive the response, the resource scheduling client is in an online state; and if the resource scheduling engine cannot normally receive the response, the resource scheduling client is in an offline state.
Step S303: and according to the predicted execution time of each task to be distributed and the idle time of each target device in the available device list, sending each task to each target device with the matched time according to a task distribution strategy.
In the batch task scheduling, a plurality of tasks are distributed according to a simultaneous execution mode, that is, after one task is distributed and before the execution of the task is finished, the next task is distributed to different target devices.
Specifically, automatic matching is performed according to the estimated task execution time and the idle time of the target device, and then task distribution is performed on the finally matched target device. When the resource scheduling engine allocates and schedules each task to be distributed in batches, the execution state of each task to be distributed is checked firstly:
(1) if the task to be distributed is in a task ending state, directly switching to the next task;
(2) if the task to be distributed is in the 'execution failure state', entering a failure task processing flow;
specifically, firstly, judging whether the frequency of the failed task exceeds a preset threshold value N, and if the frequency of the failed task exceeds the preset threshold value N, moving the failed task to an abnormal task list;
otherwise, if the failure times are smaller than the threshold value, distributing the task to the matched target equipment and executing the task, and after the task is executed, recording the execution result;
(3) if the task to be distributed is in the 'un-started state', distributing the task to the matched target equipment and executing the task, and after the task is executed, recording an execution result;
more specifically, the condition of matching needs to satisfy the following condition:
the idle time of the target device and the number of executable atomic tasks in the unit time of the target device are more than or equal to the estimated execution time/the atomic task execution time of the tasks to be distributed, wherein,
the atomic task is a unit preset task of the task;
the execution time of the atomic task represents the duration of executing a unit preset task in unit time;
the target device idle time represents an idle duration of the target device.
The following will be described in detail by taking "code scanning" as an example of the task:
assuming that in the code scanning task, 1 minute is needed for scanning 1 ten thousand lines of codes, so that the task (i.e. scanning 1 ten thousand lines of codes) can be set as an atomic task;
setting the idle time of any target equipment as 60 minutes in terms of target equipment idle time; the number of executable atomic tasks in unit time (1 minute) of the target equipment is 1;
as can be seen from the above, for one "code scan" task to be distributed, which is a total of 50 ten thousand lines of tasks, the expected execution time is 50 minutes;
then, according to the above formula, it can be known that: 60 × 1> -50/1, so that the target device meets the availability conditions and task distribution is possible.
Step S304: the respective target devices automatically execute the distributed tasks.
And finally, after all the batch tasks are executed, generating a task execution report. Wherein the task execution report includes: task number, task name, task type, start time, end time, execution machine, execution state.
As shown in fig. 4, a schematic diagram of a distributed task scheduling system using idle resources according to an embodiment of the present invention is shown, where the system 4 includes:
an information obtaining device 41, configured to obtain device information of a target device, and determine an online state of the target device;
an available information device 42, configured to initialize the target device, and generate an available device list according to an online state of the target device;
the task distributing device 43 sends each task to each target device with the matched time according to a task distribution policy according to the expected execution time of each task to be distributed and the idle time of each target device in the available device list, where the task distribution policy is:
the idle time of the target device and the number of executable atomic tasks in the unit time of the target device are more than or equal to the estimated execution time of the tasks to be distributed/the execution time of the atomic tasks,
the atomic task is a unit preset task of the task, the execution time of the atomic task represents the time for executing the unit preset task in unit time, and the idle time of the target equipment represents the idle time of the target equipment;
and a task executing device 44 for controlling each target device to automatically execute the distributed task.
Optionally, the device information includes:
the IP address, the device type and the device holder information of the target device.
Optionally, the available information device 42 is configured to:
and detecting the connectivity of each target device, authenticating and accessing the target devices according to the device information, and installing a resource scheduling client on each target device.
Optionally, the information acquiring device 41 is configured to:
and the resource scheduling engine sends detection information to the resource scheduling client regularly, and judges the online state of the target equipment according to the response of the resource scheduling client.
Optionally, the information acquiring device 41 is further configured to:
the resource scheduling engine and the resource scheduling client are connected in a long way by establishing TCP, and regularly send detection information to the resource scheduling client;
if the resource scheduling engine can normally receive a response, the resource scheduling client is in an online state;
and if the resource scheduling engine cannot normally receive the response, the resource scheduling client is in an offline state.
Optionally, the available information device 42 further includes:
and updating the available equipment list in real time according to the online state of the target equipment.
Optionally, the task distribution policy further includes:
if the task state is the execution failure state, entering a failure task processing flow, wherein the failure task processing flow comprises the following steps:
judging whether the frequency of the failed tasks exceeds a preset threshold value or not;
if the frequency of the failed task is larger than or equal to the threshold value, the failed task is moved to an abnormal task list;
and if the number of times of the failed task is less than the preset threshold value, distributing the task again.
The distributed task scheduling system using idle resources provided by the present invention is a system corresponding to the above method, and therefore, is not described herein again.
Fig. 5 is a schematic diagram of a distributed task scheduling system using idle resources according to an embodiment of the present invention, where the system 5 includes: a memory 51 and a processor 52, wherein the memory 51 is configured to store instructions, and the processor 52 is configured to execute the instructions to:
acquiring equipment information of target equipment and judging the online state of the target equipment;
initializing the target equipment, and generating an available equipment list according to the online state of the target equipment;
distributing each task to each target device with matched time according to a task distribution strategy according to the expected execution time of each task to be distributed and the idle time of each target device in the available device list, wherein the task distribution strategy is as follows:
the idle time of the target device and the number of executable atomic tasks in the unit time of the target device are more than or equal to the estimated execution time of the tasks to be distributed/the execution time of the atomic tasks,
the atomic task is a unit preset task of the task, the execution time of the atomic task represents the time for executing the unit preset task in unit time, and the idle time of the target equipment represents the idle time of the target equipment;
the respective target devices automatically execute the distributed tasks.
By the distributed task scheduling method and system using the idle resources, the target task and the execution equipment can be accurately matched according to a task execution time consumption estimation method, the task can be accurately issued to the target equipment in an activated state through automatic detection of the equipment state, the idle equipment resources can be utilized in different time periods, the operation cost is greatly saved, and a research and development team can effectively realize automatic task execution (for example, compiling and packaging in continuous integration, static code inspection and automatic test) and collect execution results under the condition that a large amount of special equipment is not required to be purchased.
Moreover, while the operations of the method of the invention are depicted in the drawings in a particular order, this does not require or imply that all of the depicted operations must be performed to achieve desirable results. Additionally or alternatively, certain steps may be omitted, multiple steps combined into one step execution, and/or one step broken down into multiple step executions.
The above-mentioned embodiments are intended to illustrate the objects, technical solutions and advantages of the present invention in further detail, and it should be understood that the above-mentioned embodiments are only exemplary embodiments of the present invention, and are not intended to limit the scope of the present invention, and any modifications, equivalent substitutions, improvements and the like made within the spirit and principle of the present invention should be included in the scope of the present invention.
Claims (17)
1. A distributed task scheduling method using idle resources, comprising:
acquiring equipment information of target equipment and judging the online state of the target equipment;
initializing the target equipment, and generating an available equipment list according to the online state of the target equipment;
distributing each task to each target device with matched time according to a task distribution strategy according to the expected execution time of each task to be distributed and the idle time of each target device in the available device list, wherein the task distribution strategy is as follows:
the idle time of the target device and the number of executable atomic tasks in the unit time of the target device are more than or equal to the estimated execution time of the tasks to be distributed/the execution time of the atomic tasks,
the atomic task is a unit preset task of the task, the execution time of the atomic task represents the time for executing the unit preset task in unit time, and the idle time of the target equipment represents the idle time of the target equipment;
the respective target devices automatically execute the distributed tasks.
2. The method of claim 1, wherein the device information comprises:
the IP address, the device type and the device holder information of the target device.
3. The method of claim 1, wherein initializing the target device comprises:
and detecting the connectivity of each target device, authenticating and accessing the target devices according to the device information, and installing a resource scheduling client on each target device.
4. The method of claim 1, wherein determining presence information of the target device comprises:
and the resource scheduling engine sends detection information to the resource scheduling client regularly, and judges the online state of the target equipment according to the response of the resource scheduling client.
5. The method of claim 4, wherein determining the presence status of the target device further comprises:
the resource scheduling engine and the resource scheduling client are connected in a long way by establishing TCP, and regularly send detection information to the resource scheduling client;
if the resource scheduling engine can normally receive a response, the resource scheduling client is in an online state;
and if the resource scheduling engine cannot normally receive the response, the resource scheduling client is in an offline state.
6. The method of claim 1, wherein initializing the target device and generating a list of available devices based on the presence status of the target device, further comprises:
and updating the available equipment list in real time according to the online state of the target equipment.
7. The method of claim 1, wherein the task distribution policy further comprises:
if the task state is the execution failure state, entering a failure task processing flow, wherein the failure task processing flow comprises the following steps:
judging whether the frequency of the failed tasks exceeds a preset threshold value or not;
if the frequency of the failed task is greater than or equal to the preset threshold value, the failed task is moved to an abnormal task list;
and if the number of times of the failed task is less than the preset threshold value, distributing the task again.
8. A distributed task scheduling system that utilizes idle resources, comprising:
the information acquisition device is used for acquiring the equipment information of the target equipment and judging the online state of the target equipment;
the available information device is used for initializing the target equipment and generating an available equipment list according to the online state of the target equipment;
the task distribution device sends each task to each target device with matched time according to a task distribution strategy according to the expected execution time of each task to be distributed and the idle time of each target device in the available device list, wherein the task distribution strategy is as follows:
the idle time of the target device and the number of executable atomic tasks in the unit time of the target device are more than or equal to the estimated execution time of the tasks to be distributed/the execution time of the atomic tasks,
the atomic task is a unit preset task of the task, the execution time of the atomic task represents the time for executing the unit preset task in unit time, and the idle time of the target equipment represents the idle time of the target equipment;
and the task execution device is used for controlling each target device and automatically executing the distributed task.
9. The system of claim 8, wherein the device information comprises:
the IP address, the device type and the device holder information of the target device.
10. The system of claim 8, wherein the available information means is configured to:
and detecting the connectivity of each target device, authenticating and accessing the target devices according to the device information, and installing a resource scheduling client on each target device.
11. The system according to claim 9, wherein the information acquisition device is configured to:
and the resource scheduling engine sends detection information to the resource scheduling client regularly, and judges the online state of the target equipment according to the response of the resource scheduling client.
12. The system of claim 11, wherein the information obtaining means is further configured to:
the resource scheduling engine and the resource scheduling client are connected in a long way by establishing TCP, and regularly send detection information to the resource scheduling client;
if the resource scheduling engine can normally receive a response, the resource scheduling client is in an online state;
and if the resource scheduling engine cannot normally receive the response, the resource scheduling client is in an offline state.
13. The system of claim 8, wherein the available information means further comprises:
and updating the available equipment list in real time according to the online state of the target equipment.
14. The system of claim 8, wherein the task distribution policy further comprises:
if the task state is the execution failure state, entering a failure task processing flow, wherein the failure task processing flow comprises the following steps:
judging whether the frequency of the failed tasks exceeds a preset threshold value or not;
if the frequency of the failed task is larger than or equal to the threshold value, the failed task is moved to an abnormal task list;
and if the number of times of the failed task is less than the preset threshold value, distributing the task again.
15. A distributed task scheduling system that utilizes idle resources, comprising: a memory to store instructions and a processor to execute the instructions to:
acquiring equipment information of target equipment and judging the online state of the target equipment;
initializing the target equipment, and generating an available equipment list according to the online state of the target equipment;
distributing each task to each target device with matched time according to a task distribution strategy according to the expected execution time of each task to be distributed and the idle time of each target device in the available device list, wherein the task distribution strategy is as follows:
the idle time of the target device and the number of executable atomic tasks in the unit time of the target device are more than or equal to the estimated execution time of the tasks to be distributed/the execution time of the atomic tasks,
the atomic task is a unit preset task of the task, the execution time of the atomic task represents the time for executing the unit preset task in unit time, and the idle time of the target equipment represents the idle time of the target equipment;
the respective target devices automatically execute the distributed tasks.
16. An electronic device, comprising:
one or more processors;
a storage device for storing one or more programs,
when executed by the one or more processors, cause the one or more processors to implement the method of any one of claims 1-7.
17. A computer-readable medium, on which a computer program is stored, which, when being executed by a processor, carries out the method according to any one of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710003887.3A CN108270837B (en) | 2017-01-04 | 2017-01-04 | Distributed task scheduling method and system using idle resources |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710003887.3A CN108270837B (en) | 2017-01-04 | 2017-01-04 | Distributed task scheduling method and system using idle resources |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108270837A CN108270837A (en) | 2018-07-10 |
| CN108270837B true CN108270837B (en) | 2021-04-30 |
Family
ID=62770800
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710003887.3A Active CN108270837B (en) | 2017-01-04 | 2017-01-04 | Distributed task scheduling method and system using idle resources |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108270837B (en) |
Families Citing this family (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111487421B (en) * | 2019-01-28 | 2024-02-02 | 深圳市帝迈生物技术有限公司 | Scheduling method, storage medium and sample analyzer |
| CN110087144A (en) * | 2019-05-15 | 2019-08-02 | 深圳市商汤科技有限公司 | Video file processing method, device, electronic equipment and computer storage medium |
| US11842225B2 (en) * | 2019-10-03 | 2023-12-12 | Steering Solutions Ip Holding Corporation | Systems and methods for decentralized-distributed processing of vehicle data |
| CN110908782B (en) * | 2019-11-01 | 2022-08-19 | 湖北省楚天云有限公司 | Genetic algorithm optimization-based packaging type distributed job task scheduling method and system |
| CN111611074A (en) * | 2020-05-14 | 2020-09-01 | 北京达佳互联信息技术有限公司 | Method and device for scheduling cluster resources |
| CN113296945A (en) * | 2021-05-20 | 2021-08-24 | 山东云海国创云计算装备产业创新中心有限公司 | Continuous integration method, system, equipment and computer readable storage medium |
| CN113946431B (en) * | 2021-12-22 | 2022-03-04 | 北京瑞莱智慧科技有限公司 | Resource scheduling method, system, medium and computing device |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102143563A (en) * | 2010-07-30 | 2011-08-03 | 华为技术有限公司 | Control method, equipment and systems of short message center cluster |
| CN104702664A (en) * | 2014-12-29 | 2015-06-10 | 惠州Tcl移动通信有限公司 | Method and system for controlling task server |
| CN104731663A (en) * | 2015-03-31 | 2015-06-24 | 北京奇艺世纪科技有限公司 | Task processing method and system |
| CN105320570A (en) * | 2015-11-09 | 2016-02-10 | 深圳市万普拉斯科技有限公司 | Resource management method and system |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| TWI623205B (en) * | 2012-09-27 | 2018-05-01 | 內數位專利控股公司 | End-to-end architecture, api framework, discovery, and access in a virtualized network |
-
2017
- 2017-01-04 CN CN201710003887.3A patent/CN108270837B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102143563A (en) * | 2010-07-30 | 2011-08-03 | 华为技术有限公司 | Control method, equipment and systems of short message center cluster |
| CN104702664A (en) * | 2014-12-29 | 2015-06-10 | 惠州Tcl移动通信有限公司 | Method and system for controlling task server |
| CN104731663A (en) * | 2015-03-31 | 2015-06-24 | 北京奇艺世纪科技有限公司 | Task processing method and system |
| CN105320570A (en) * | 2015-11-09 | 2016-02-10 | 深圳市万普拉斯科技有限公司 | Resource management method and system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108270837A (en) | 2018-07-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108270837B (en) | Distributed task scheduling method and system using idle resources | |
| US8954579B2 (en) | Transaction-level health monitoring of online services | |
| US9329983B2 (en) | Computer program testing | |
| CN105164644B (en) | Hook frame | |
| CN103201724B (en) | Providing application high availability in highly-available virtual machine environments | |
| CN103620561B (en) | The method and system of interoperability debugging when running for n road | |
| US20150358392A1 (en) | Method and system of virtual desktop infrastructure deployment studio | |
| US20150100829A1 (en) | Method and system for selecting and executing test scripts | |
| US20150100832A1 (en) | Method and system for selecting and executing test scripts | |
| US9483314B2 (en) | Systems and methods for fault tolerant batch processing in a virtual environment | |
| CN108228256B (en) | Code synchronization method, device, computer readable medium and terminal | |
| US20150100830A1 (en) | Method and system for selecting and executing test scripts | |
| US20150100831A1 (en) | Method and system for selecting and executing test scripts | |
| CN108984179B (en) | Linux compiling processing method and device | |
| CN107045475B (en) | Test method and device | |
| CN109324962B (en) | Method and equipment for testing application program based on cloud Internet of things technology | |
| CN111258913A (en) | Automatic algorithm testing method and device, computer system and readable storage medium | |
| WO2014138980A1 (en) | Computer system using in-service software upgrade | |
| CN110268378A (en) | The method for creating the data backup of virtual automation solution, the computer program of implementation method and the virtual server run by method | |
| US20200310828A1 (en) | Method, function manager and arrangement for handling function calls | |
| US10120744B1 (en) | Graph execution system and method for a computing environment | |
| CN110727575B (en) | Information processing method, system, device and storage medium | |
| RU2696299C2 (en) | Control when initiating elementary tasks on server platform | |
| CN114006815B (en) | Automatic deployment method and device for cloud platform nodes, nodes and storage medium | |
| CN108243205B (en) | Method, equipment and system for controlling resource allocation of cloud platform |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |