CN111324428B - Task allocation method, device, equipment and computer readable storage medium - Google Patents
Task allocation method, device, equipment and computer readable storage medium Download PDFInfo
- Publication number
- CN111324428B CN111324428B CN201910895064.5A CN201910895064A CN111324428B CN 111324428 B CN111324428 B CN 111324428B CN 201910895064 A CN201910895064 A CN 201910895064A CN 111324428 B CN111324428 B CN 111324428B
- Authority
- CN
- China
- Prior art keywords
- buffer pool
- nodes
- task
- injection
- preset
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/48—Program initiating; Program switching, e.g. by interrupt
- G06F9/4806—Task transfer initiation or dispatching
- G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
- G06F9/4881—Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
- G06F9/5038—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals considering the execution order of a plurality of tasks, e.g. taking priority or time dependency constraints into consideration
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Abstract
The embodiment of the application provides a task allocation method, device and equipment and a computer readable storage medium, relates to the technical field of computers, and can improve task allocation efficiency. The task allocation method comprises the following steps: acquiring the number of task nodes in a buffer pool; determining whether the number of task nodes in the buffer pool is smaller than an injection threshold, if so, performing one-time traversal injection on the buffer pool, and distributing the tasks injected into the buffer pool to an executor, wherein the performing one-time traversal injection on the buffer pool comprises: traversing each node in the buffer pool, and injecting tasks in the task pool into empty nodes to enable the empty nodes to be changed into task nodes; when a task execution result of the executor is obtained, a corresponding node of the executed task in the buffer pool is changed into an empty node. The scheme of the application is mainly used for task scheduling.
Description
Technical Field
The present application relates to the field of computer technologies, and in particular, to a task allocation method, apparatus, device, and computer readable storage medium.
Background
The task system based on the network environment comprises an executor for executing tasks by utilizing resources and a task pool for storing the tasks, wherein the task pool is provided with a fixed storage space for storing the tasks, the tasks are periodically injected into the task pool and issued to the executor to be executed by the executor, however, the task distribution efficiency in the current task distribution mode is lower.
Disclosure of Invention
The application provides a task allocation method, a device, equipment and a computer readable storage medium, which can improve task allocation efficiency.
In a first aspect, an embodiment of the present application provides a task allocation method, including:
acquiring the number of task nodes in a buffer pool;
determining whether the number of task nodes in the buffer pool is smaller than an injection threshold, if so, performing one-time traversal injection on the buffer pool, and distributing the tasks injected into the buffer pool to an executor, wherein the performing one-time traversal injection on the buffer pool comprises: traversing each node in the buffer pool, and injecting tasks in the task pool into empty nodes to enable the empty nodes to be changed into task nodes;
when a task execution result of the executor is obtained, a corresponding node of the executed task in the buffer pool is changed into an empty node.
Optionally, the method further comprises:
periodically obtaining the times of traversing injection of the buffer pool in a first preset time period;
determining whether the times of performing traversal injection on the buffer pool within the first preset duration meets preset conditions, and if yes, adjusting the node number of the buffer pool.
Optionally, the injection threshold is positively correlated with the number of nodes of the buffer pool.
Optionally, the determining whether the number of times of performing traversal injection on the buffer pool within the first preset duration meets a preset condition, and if yes, the adjusting the number of nodes of the buffer pool includes:
determining whether the number of times of traversal injection on the buffer pool in the first preset time period is larger than a first preset number of times, if so, increasing the number of nodes of the buffer pool;
determining whether the number of times of traversal injection on the buffer pool in the first preset time period is smaller than a second preset time period, and if yes, reducing the number of nodes of the buffer pool.
Optionally, the first preset number of times is positively correlated with the number of nodes of the buffer pool, and the second preset number of times is positively correlated with the number of nodes of the buffer pool.
Optionally, the magnitude of the change in the number of nodes of the buffer pool each time is positively correlated with the value before the change in the number of nodes of the buffer pool each time.
Optionally, the process of periodically obtaining the number of times of performing traversal injection on the buffer pool in the first preset duration includes:
periodically acquiring the times of traversing injection and the reference injection times of the buffer pool in a first preset time period, wherein the reference injection times are positively correlated with the node number of the buffer pool;
obtaining the first preset times and the second preset times according to the reference injection times, wherein the first preset times are larger than the reference injection times and are positively correlated with the reference injection times, and the second preset times are smaller than the reference injection times and are positively correlated with the reference injection times;
after the node number of the buffer pool is increased each time, the ratio of the node number of the buffer pool after the current increase to the node number of the buffer pool before the current increase is equal to the ratio of the first preset times to the reference injection times;
after the number of nodes of the buffer pool is reduced each time, the ratio of the number of nodes of the buffer pool after the current reduction to the number of nodes of the buffer pool before the current reduction is equal to the ratio of the second preset times to the reference injection times.
Optionally, the process of periodically obtaining the number of times of performing traversal injection on the buffer pool in the first preset duration includes:
Periodically acquiring the times N of traversing injection on the buffer pool, the number m of nodes in the buffer pool and the number p of task nodes in a first preset time period;
determining whether N, m and p have no change in a second preset time period, and if so, resetting the buffer pool;
the resetting the buffer pool comprises:
the task allocation relation between tasks corresponding to all task nodes in the buffer pool and the executor is cancelled, the tasks corresponding to all task nodes in the buffer pool are injected into the task pool, the task nodes become empty nodes, and the buffer pool is emptied;
and injecting the tasks in the task pool into the emptied buffer pool, enabling all empty nodes in the buffer pool to be changed into task nodes, and distributing the tasks injected into the buffer pool to an executor.
On the other hand, the embodiment of the application also provides a task allocation device, which comprises:
the first acquisition module is used for acquiring the number of task nodes in the buffer pool;
the injection module is configured to determine whether the number of task nodes in the buffer pool is smaller than an injection threshold, if yes, perform a traversal injection on the buffer pool, and allocate the tasks injected into the buffer pool to an executor, where performing the traversal injection on the buffer pool includes: traversing each node in the buffer pool, and injecting tasks in the task pool into empty nodes to enable the empty nodes to be changed into task nodes;
And the execution result processing module is used for enabling corresponding nodes of the executed task in the buffer pool to become empty nodes when the task execution result of the executor is acquired.
Optionally, the apparatus further includes:
the second acquisition module is used for periodically acquiring the times of traversal injection of the buffer pool in a first preset duration;
the node quantity adjusting module is used for determining whether the times of performing traversal injection on the buffer pool in the first preset duration meet preset conditions or not, and if yes, adjusting the node quantity of the buffer pool.
Optionally, the injection threshold is positively correlated with the number of nodes of the buffer pool.
Optionally, the node number adjusting module is specifically configured to:
determining whether the number of times of traversal injection on the buffer pool in the first preset time period is larger than a first preset number of times, if so, increasing the number of nodes of the buffer pool;
determining whether the number of times of traversal injection on the buffer pool in the first preset time period is smaller than a second preset time period, and if yes, reducing the number of nodes of the buffer pool.
Optionally, the first preset number of times is positively correlated with the number of nodes of the buffer pool, and the second preset number of times is positively correlated with the number of nodes of the buffer pool.
Optionally, the magnitude of the change in the number of nodes of the buffer pool each time is positively correlated with the value before the change in the number of nodes of the buffer pool each time.
Optionally, the second obtaining module is specifically configured to:
periodically acquiring the times of traversing injection and the reference injection times of the buffer pool in a first preset time period, wherein the reference injection times are positively correlated with the node number of the buffer pool;
obtaining the first preset times and the second preset times according to the reference injection times, wherein the first preset times are larger than the reference injection times and are positively correlated with the reference injection times, and the second preset times are smaller than the reference injection times and are positively correlated with the reference injection times;
after the node number of the buffer pool is increased each time, the ratio of the node number of the buffer pool after the current increase to the node number of the buffer pool before the current increase is equal to the ratio of the first preset times to the reference injection times;
after the number of nodes of the buffer pool is reduced each time, the ratio of the number of nodes of the buffer pool after the current reduction to the number of nodes of the buffer pool before the current reduction is equal to the ratio of the second preset times to the reference injection times.
Optionally, the second obtaining module is specifically configured to:
periodically acquiring the times N of traversing injection on the buffer pool, the number m of nodes in the buffer pool and the number p of task nodes in a first preset time period;
determining whether N, m and p have no change in a second preset time period, and if so, resetting the buffer pool;
the resetting the buffer pool comprises:
the task allocation relation between tasks corresponding to all task nodes in the buffer pool and the executor is cancelled, the tasks corresponding to all task nodes in the buffer pool are injected into the task pool, the task nodes become empty nodes, and the buffer pool is emptied;
and injecting the tasks in the task pool into the emptied buffer pool, enabling all empty nodes in the buffer pool to be changed into task nodes, and distributing the tasks injected into the buffer pool to an executor.
On the other hand, the embodiment of the application also provides a task allocation device, which comprises:
a processor and a memory for storing at least one instruction which when loaded and executed by the processor implements the method described above.
In another aspect, embodiments of the present application also provide a computer-readable storage medium having a computer program stored therein, which when run on a computer, causes the computer to perform the above-described method.
According to the task allocation method, the device, the equipment and the computer readable storage medium, the buffer pool is arranged between the task pool and the executors, the tasks in the task pool are injected into the buffer pool according to the corresponding task quantity in the buffer pool, the tasks in the buffer pool are allocated to the corresponding executors, decoupling between the executors and the task pool is achieved in a buffer mode, the task allocation process of the executors cannot directly influence the task pool, and therefore the overall task scheduling system still keeps high task allocation and execution efficiency under the condition of uneven resource distribution.
Drawings
Fig. 1 is a schematic diagram of an application scenario in an embodiment of the present application;
FIG. 2 is a flow chart of a task allocation method according to an embodiment of the present application;
FIG. 3 is a flowchart of another task allocation method according to an embodiment of the present application;
FIG. 4 is a flowchart illustrating a portion of steps in a task allocation method according to another embodiment of the present application;
FIG. 5 is a flow chart of a refinement step of step 201 of FIG. 4;
FIG. 6 is a flow chart of another refinement step of step 201 of FIG. 4;
FIG. 7 is a flow chart of a further refinement step of step 201 of FIG. 4;
FIG. 8 is a block diagram of a task assigning apparatus according to an embodiment of the present application;
FIG. 9 is a block diagram of another task assigning apparatus according to an embodiment of the present application;
fig. 10 is a block diagram of a task allocation device according to an embodiment of the present application.
Detailed Description
The terminology used in the description of the embodiments of the application herein is for the purpose of describing particular embodiments of the application only and is not intended to be limiting of the application.
Before describing the embodiments of the present application, the present inventors firstly set forth the process of the embodiments of the present application, and found that, in the prior art, a task pool has a fixed storage space for storing tasks, and periodically makes the tasks be injected into the task pool and issued to an actuator, and this results in low task allocation efficiency due to uneven distribution of resources utilized by the actuator in time and space dimensions. For example, if a server executing a task executes the task at night, since network transmission resources are more abundant at night, the time required for executing the task at night is shorter than that at daytime, which may cause frequent access to the task pool by both the receiving and distributing parties, resulting in a low overall task distribution efficiency. In order to improve the above-described problems, the inventors have provided embodiments of the present application, which are described below.
The embodiment of the application can be applied to a task scheduling system. Fig. 1 is a schematic diagram of an application scenario in an embodiment of the present application. The task scheduling system shown in fig. 1 mainly includes a task allocation device 11, an actuator set 12, a buffer pool 13, and a task pool 14, where the task allocation device 11 and the actuator set 12 may be servers or other network devices in a network environment, the actuator set 12 includes at least one actuator 120, where the actuator 120 is configured to specifically execute a task and feed back an execution result of the task, for example, the actuator set 12 is a server, the actuator 120 is a process, the actuator 120 uses a network to execute the task, resources utilized when executing the task include network bandwidth resources, and may further include other necessary resources, the resources are unevenly distributed in time and space dimensions, performance of the task execution of the actuator set 12 is related to available resources thereof, for example, the execution efficiency of the actuator set 12 is higher in idle time such as night, and the closer the actuator set 12 is to a target position of data transmitted by the task to be executed, the execution efficiency is higher. The buffer pool 13 is used for storing tasks to be executed, the buffer pool 13 may be adaptively generated according to the executor 120 and correspond to the executor 120, so that the executor 120 and the buffer pool 13 have a one-to-one correspondence relationship, it is to be understood that in other realizable embodiments, one buffer pool may correspond to a plurality of executors, or one buffer pool corresponds to a plurality of executor sets, in the descriptions of fig. 1 and the following embodiments, only one buffer pool 13 corresponds to one executor 120 is taken as an example, the tasks in the buffer pool 13 are executed by the corresponding executor 120, the task pool 14 is used for storing tasks to be allocated, the buffer pool 13 and the task pool 14 may specifically be data storage structures in a server, the buffer pool 13 stores tasks in the form of nodes, each node corresponds to one task, and the greater the number of nodes, the greater the capacity of the buffer pool 13 can store the greater the number of tasks.
An embodiment of the present application provides a task allocation method, where a task may be a task executed by using a network, for example, a video image detecting task, and an execution body of the method may be a task allocation device 11 in a task scheduling system, as shown in fig. 2, fig. 2 is a flow chart of a task allocation method in the embodiment of the present application, and the method may correspond to a task allocation method corresponding to a buffer pool 13 in fig. 1, where the method includes:
step 101, acquiring the number of task nodes in a buffer pool 13;
step 102, determining whether the number of task nodes in the buffer pool 13 is smaller than an injection threshold, if yes, entering step 103, otherwise, entering step 101 again, and continuously monitoring the number of task nodes in the buffer pool 13;
step 103, performing one-time traversal injection on the buffer pool 13, distributing the task of injecting the buffer pool 13 to the executor 120, and then entering step 104 and step 101; wherein, performing the traversal injection to the buffer pool 13 includes: each node in the buffer pool 13 is traversed, and tasks in the task pool 14 are injected into empty nodes, so that the empty nodes become task nodes.
Specifically, after the completion of the step 103 and the step of performing the traversal injection on the buffer pool 13, the method may directly enter the steps 101 and 104, i.e. the number of task nodes in the buffer pool 13 and the task execution result are monitored in real time. In a specific traversal injection process, all nodes in the buffer pool 13 are traversed, if a node is found to be a task node, continuing to traverse the buffer pool 13 is skipped, if the node is found to be an empty node, a new task is injected from the task pool 14 to the node, so that the node becomes a task node, and until all the nodes in the buffer pool 13 are task nodes, the traversal injection is completed. It should be noted that, if the tasks in the task pool 14 are not enough to fill the buffer pool 13, the current traversal injection is completed after all the tasks in the task pool 14 are injected. The task pool 14 is used for storing all tasks to be allocated received by the task scheduling system, after the tasks in the task pool 14 are injected into the buffer pool 13, the tasks are allocated to the executor 120, when the tasks in the task pool 14 are allocated to the executor 120, position information of the tasks is sent to the executor 120, for example, poon. Key n, where poon represents the position of the buffer pool 13, key n represents the position of a node corresponding to the tasks in the buffer pool 13, so that the executor 120 finds the node where the tasks are located, and obtains corresponding specific task information, the tasks to be executed are stored in the buffer pool 13, the executor 120 executes the tasks allocated to itself in the buffer pool 13 one by one, after the executor 120 completes execution of one task, the execution result of the task corresponding to the task is fed back to the task allocation device 11, and the task execution result also includes the position information of the task, for example poon. Key n. If a node in the buffer pool 13 is injected with a task, the node becomes a task node, and if a node in the buffer pool 13 does not store information related to the task, the node becomes an empty node. The process of injecting tasks into the buffer pool 13, i.e., the process of changing hollow nodes in the buffer pool 13 into task nodes.
104, when a task execution result of the executor 120 is obtained, changing a corresponding node of the executed task in the buffer pool 13 into an empty node, and then entering step 101 to continuously monitor the number of task nodes in the buffer pool 13;
specifically, after the executor 120 performs the task, a task execution result is generated, which indicates that the task has been performed, and no allocation or execution is needed, and since the task execution result includes the position information of the task, the task allocation device 11 may find a corresponding node according to the position information, and change the task node into an empty node, so as to store other tasks.
The above steps indicate that when the number of task nodes in the buffer pool 13 falls below the injection threshold, new tasks are added from the task pool 14 and stored in the buffer pool 13 as tasks to be executed by the executor 120, the injection threshold may be a preset fixed value or a preset adjustable value, for example, for a certain buffer pool, the number of nodes in the buffer pool is 100, the injection threshold is set to be 50, the number of task nodes in the buffer pool is 49 or less than 50, that is, the number of task nodes in the buffer pool reaches the injection condition, at this time, the buffer pool 13 is subjected to one-time traversal injection, after the traversal injection, the number of task nodes in the buffer pool 13 becomes 100, during the process of gradually executing the tasks in the buffer pool 13, the number of empty nodes gradually increases, the number of task nodes gradually decreases, until the number of task nodes is less than the injection threshold again determined, and so on, since the number of task nodes in the buffer pool 14 is added to the buffer pool 13, the task nodes in the buffer pool is injected into the buffer pool 13 at first, then the task nodes in the buffer pool is allocated to the corresponding executor 120, even if the execution time of the task in the buffer pool is not directly affected by the executor 120 is not directly executed by the executor 120.
It should be noted that the above-mentioned execution sequence relationship between the steps is merely an example, and the execution sequence between the steps is not limited in the embodiment of the present application, for example, the step 104 is executed after the execution result of the task is fed back by the executor 120, and may be executed in a gap between or in synchronization with other steps.
According to the task allocation method in the embodiment of the application, the buffer pool is arranged between the task pool and the executors, the tasks in the task pool are injected into the buffer pool according to the corresponding task quantity in the buffer pool as judgment basis, the tasks in the buffer pool are allocated to the corresponding executors, decoupling between the executors and the task pool is realized in a buffer mode, the task allocation process of the executors can not directly influence the task pool, and therefore, under the condition of uneven resource distribution, the overall task scheduling system still keeps higher task allocation and execution efficiency.
Optionally, as shown in fig. 3, fig. 3 is a flow chart of another task allocation method in an embodiment of the present application, where the task allocation method further includes:
step 201, periodically obtaining the times of performing traversal injection on the buffer pool 13 within a first preset duration, and then entering step 202;
Step 202, determining whether the number of times of performing traversal injection on the buffer pool 13 within a first preset duration meets a preset condition, if yes, entering step 203, and if not, entering step 101;
step 203, adjusting the number of nodes of the buffer pool 13, and then may proceed to step 101.
Specifically, steps 201 and 202 may be executed periodically to determine whether the number of nodes in the buffer pool 13 needs to be adjusted, where the buffer pool 13 in the above steps is a certain buffer pool 13 in the task allocation system of fig. 1, that is, it is determined whether one buffer pool 13 meets the need of adjusting the number of nodes, and the task allocation method in the embodiment of the present application is the method described for the buffer pool 13, and the same task allocation manner may be applied to each buffer pool 13 in the task allocation system. In step 201, for example, the number of times of performing the traversal injection on the buffer pool 13 in x seconds is obtained every x seconds, that is, every x seconds is a period, and the number of times of performing the traversal injection on the buffer pool 13 in the previous period is periodically obtained. The preset condition may be set as required, for example, a threshold y is set, when the number of times of performing traversal injection in x seconds is greater than y, the number of nodes in the buffer pool 13 is increased, and when the number of times of performing traversal injection in x seconds is less than y, the number of nodes in the buffer pool 13 is reduced.
Alternatively, in a task allocation method such as that shown in fig. 3, the injection threshold is positively correlated with the number of nodes of the buffer pool 13.
Specifically, for example, when the number of nodes in the buffer pool 13 is 100, the corresponding injection threshold is 70, that is, when the number of task nodes in the buffer pool 13 falls below 70, new tasks are injected into the buffer pool 13 from the task pool 14; since the number of nodes in the buffer pool 13 is dynamically adjusted, the injection threshold may be changed to 300 when the number of nodes in the buffer pool 13 becomes 400, i.e., new tasks are added from the task pool 14 to the buffer pool 13 when the number of task nodes in the buffer pool 13 falls below 300; and so on. The larger the number of nodes of the buffer pool 13 is, the larger the corresponding injection threshold value is; the smaller the number of nodes of the buffer pool 13, the smaller the corresponding injection threshold. In this way, the number of task nodes stored in the buffer pool 13 that matches the number of nodes of the buffer pool 13 can be maintained without performing the injection operation too frequently. The injection threshold may be proportional to the number of nodes of the buffer pool 13, e.g. the injection threshold is set to 50% of the number of nodes of the buffer pool 13, irrespective of the change in the number of nodes of the buffer pool 13, as long as the number of task nodes therein falls below half the number of nodes, i.e. the traversal injection is performed. It will be appreciated that in other possible embodiments, the injection threshold may be a fixed preset value, for example, the number of nodes in the buffer pool 13 is divided into three steps 80, 90 and 100, and is dynamically adjusted in these three steps, but the injection threshold is 70 regardless of the number of nodes in the buffer pool 13, i.e. the traversal injection is performed as long as the number of task nodes in the buffer pool 13 is less than 70.
Optionally, as shown in fig. 4, fig. 4 is a flow chart illustrating a part of steps in another task allocation method according to an embodiment of the present application, where step 202 includes determining whether the number of times of performing traversal injection on the buffer pool 13 within a first preset duration meets a preset condition, if yes, entering step 203, and adjusting the number of nodes of the buffer pool 13 includes:
step 2021, determining whether the number of times of performing traversal injection on the buffer pool 13 within the first preset duration is greater than the first preset number of times, if yes, entering step 2031, if not, entering step 2022;
step 2031, increasing the number of nodes of the buffer pool 13, and then may enter step 101;
step 2022, determining whether the number of times of performing traversal injection on the buffer pool 13 within the first preset duration is smaller than the second preset number of times, if yes, entering step 2032, if not, entering step 101;
step 2032, reducing the number of nodes of the buffer pool.
Specifically, when the buffer pool 13 has the same number of nodes, the second preset number of times is less than or equal to the first preset number of times, for example, the number of nodes in a certain buffer pool 13 may be divided into three steps of 50, 100 and 200, the first preset duration is 10 minutes, the first preset number of times is 15, the second preset number of times is 10, and the number of times of performing traversal injection on the buffer pool 13 within the last 10 minutes is acquired every 10 minutes. The number of nodes of the buffer pool 13 is 100 at 10 minutes, and the obtained number of times of traversal injection is 20, which exceeds the first preset number of times 15, so that the step 2031 is entered to increase the number of nodes of the buffer pool 13 to 200, then the step 101 is entered, the number of task nodes in the buffer pool 13 is continuously monitored, the step 102 is entered to judge whether the injection is performed, it should be noted that steps 102, 103 and 104 are not illustrated in fig. 4, and the specific procedures of these steps may be the same as those shown in fig. 3; at 20 minutes, the number of traversals acquired in step 201 is 19, and although the number of traversals exceeds the first preset number of times 15, the number of nodes of the buffer pool 13 is unchanged because the number of nodes reaches the maximum number of nodes 200, then step 101 is performed, the number of nodes of the buffer pool 13 is continuously monitored, and injection is performed when the number of nodes falls to an injection threshold; at 30 minutes, the number of traversals obtained in step 201 is 5, which is less than the second preset number of times 10, so step 2032 is entered, the number of nodes of the buffer pool 13 is reduced to 100, and then step 101 is entered; at 40 minutes, the number of traversals obtained in step 201 is 6, which is less than the second preset number of times 10, so step 2032 is entered, the number of nodes of the buffer pool 13 is reduced to 50, and then step 101 is entered; similarly, when the number of times of performing traversal injection on the buffer pool 13 within the first preset duration is larger, the execution efficiency of the corresponding actuator 120 is higher, so that the number of nodes of the buffer pool 13 is increased, frequent injection can be avoided to reduce the allocation efficiency of the buffer pool 13 and the task pool 14, and when the number of times of performing traversal injection on the buffer pool 13 within the first preset duration is smaller, the execution efficiency of the corresponding actuator 120 is lower, so that the number of nodes of the buffer pool 13 is reduced, the node space with lower use efficiency can be released, the released nodes can be used for storing other data, and the utilization rate of the storage space is improved. Since the executor 120 continuously executes the task, the task in the buffer pool 13 gradually decreases, so that step 101 may be entered after other steps are completed, and it is determined whether the injection condition is satisfied, so as to ensure that the task can be injected into the buffer pool 13 in time. Therefore, the number of nodes in the buffer pool 13 is dynamically adjusted according to the number of times of performing traversal injection on the buffer pool 13 within the first preset duration, so that the matching degree between the number of nodes in the buffer pool 13 and the execution efficiency of the executor 120 can be improved, and the task allocation efficiency can be improved. In other possible embodiments, for example, the number of nodes in a certain buffer pool 13 defaults to 100, the number of traversal injections into the buffer pool 13 in the last 5 minutes is acquired every 10 minutes, if more than 20 times, the number of nodes in the buffer pool 13 is increased by 20, if less than 10 times, the number of nodes in the buffer pool 13 is decreased by 10.
It should be noted that, in the embodiment of the present application, the execution sequence of each step is not limited, for example, in addition to the steps shown in fig. 4, in other possible implementations, step 2022 may be executed first in each period to determine whether to reduce the number of nodes of the buffer pool 13, and if not, step 2021 is executed again to determine whether to increase the number of nodes of the buffer pool 13; it will be appreciated that step 101 may be performed first to determine whether to traverse the injection, and then step 2021 and step 2022 may be performed to determine whether to adjust the number of nodes in the buffer pool 13.
Alternatively, the first preset number of times is positively correlated with the number of nodes of the buffer pool 13, and the second preset number of times is positively correlated with the number of nodes of the buffer pool 13.
Specifically, for example, the number of nodes in a certain buffer pool 13 may be divided into three steps of 50, 100 and 200, and the first preset time period is 10 minutes. When the number of nodes in the buffer pool 13 is 50, the first preset times is 10, and the second preset times is 5; when the number of nodes of the buffer pool 13 is 100, the first preset times is 15, and the second preset times is 10; when the number of nodes of the buffer pool 13 is 200, the first preset number of times is 20, and the second preset number of times is 15. The number of times of traversal injection of the buffer pool 13 in the last 10 minutes is acquired every 10 minutes. The number of nodes of the buffer pool 13 is 100 in 10 minutes, the acquired number of times of traversal injection is 20, and the number of times exceeds the first preset number of times 15 in the gear, so that the step 2031 is entered to increase the number of nodes of the buffer pool 13 to 200, the step 101 is entered to acquire the number of task nodes in the buffer pool 13, and the step 102 is entered to judge whether to perform injection; in the 20 th minute, the obtained traversal times are 19, and the traversal times are between the first preset times 20 and the second preset times 15 in the gear, so that the number of nodes is unchanged, then the step 101 is carried out, the number of nodes of the buffer pool 13 is continuously monitored, and injection is carried out when the number of nodes is reduced to an injection threshold value; in 30 minutes, the obtained number of traversals is 5, which is less than the second preset number of traversals under the gear by 15, so that step 2032 is entered, the number of nodes of the buffer pool 13 is reduced to 100, and then step 101 is entered; by the above way, after the node number of the buffer pool 13 is adjusted, the first preset times and the second preset times are dynamically adjusted, and the first preset times and the second preset times are positively related to the node number of the buffer pool 13, so that the condition for judging whether the node number of the buffer pool 13 is adjusted each time can be adapted to the node number of the current buffer pool 13.
Alternatively, the magnitude of the change in the number of nodes of each buffer pool 13 is positively correlated with the value before the change in the number of nodes of that buffer pool 13.
Specifically, for example, the number of nodes in a certain buffer pool 13 may be divided into five steps of 25, 50, 100, 200 and 400, the number of nodes in the buffer pool 13 is adjusted step by step, for example, the number of nodes in the current buffer pool 13 is 25, the number of times of traversing injection to the buffer pool 13 in a first preset time period is determined to be greater than the first preset time period, the number of nodes in the buffer pool 13 is increased to 50, and the change amplitude of the number of nodes is 25; next time, determining that the number of times of traversing injection to the buffer pool 13 in the first preset time period is larger than the first preset time period, increasing the node number of the buffer pool 13 to 100, wherein the node number change amplitude is 50, and the value before the node number change of the buffer pool 13 is 50; the next time it is determined that the number of times of performing traversal injection on the buffer pool 13 within the first preset time period is greater than the first preset time period, the number of nodes of the buffer pool 13 is increased to 200, the change amplitude of the number of nodes is 100, and the value before the change of the number of nodes of the buffer pool 13 is 100. Each time the number of nodes in buffer pool 13 is adjusted, the magnitude of the change in the number of nodes is positively correlated with the number of nodes before the change, that is, when the number of nodes in buffer pool 13 is large, the number of nodes is increased or decreased to avoid frequent adjustment, and when the number of nodes in buffer pool 13 is small, the number of nodes is increased or decreased to avoid excessive adjustment.
Optionally, as shown in fig. 4 and fig. 5, fig. 5 is a flowchart illustrating a refinement step of step 201 in fig. 4, where the step 201 of periodically obtaining the number of times of performing traversal injection on the buffer pool 13 in the first preset duration includes:
step 2011, periodically obtaining the number of times of traversal injection and the reference injection number of times of the buffer pool 13 in the first preset duration, where the reference injection number of times is positively related to the number of nodes of the buffer pool 13, for example, when the number of nodes of the buffer pool 13 is m=100, the corresponding reference injection number of times n=10, when the number of nodes of the buffer pool 13 is m=200, the corresponding reference injection number of times n=20, and when the number of nodes of the buffer pool 13 is m=400, the corresponding reference injection number of times n=40;
step 2012, obtaining a first preset number of times and a second preset number of times according to the reference injection number, wherein the first preset number of times is greater than the reference injection number and is positively correlated with the reference injection number, and the second preset number of times is less than the reference injection number and is positively correlated with the reference injection number, e.g., a first preset number of times n 1 = (1+r) ×n, r > 0, r represents a multiple of the increase in the number of nodes, a second preset number of times n 2 = (1-R) ×n,0 < R < 1, R represents a multiple of the reduction in the number of nodes, each of which will be specifically described below by taking r=1, r=0.5 as an example, and when n=10, n 1 ＝20，n 2 =5, when n=20, n 1 ＝40，n 2 ＝10；
After each execution of step 2031 to increase the number of nodes of the buffer pool 13, the ratio of the number of nodes of the buffer pool 13 after the current increase to the number of nodes of the buffer pool 13 before the current increase is equal to the ratio of the first preset number of times to the reference injection number of times;
specifically, for example, the number of nodes of the buffer pool 13 is increased to m= (1+r) ×m ', m' is the number of nodes of the buffer pool 13 before the increase, m is the number of nodes of the buffer pool 13 after the increase, and the ratio of the number of nodes of the buffer pool 13 after the increase to the number of nodes of the buffer pool 13 before the increase isn 1 ＝(1+r)×nThe ratio of the first preset number of times to the reference implantation number is +.>I.e. < ->For example, when the number of nodes in the buffer pool 13 is 100, n=10, n 1 ＝20，n 2 =5, and the number of times n=44 > N of traversal injection of the buffer pool 13 within the obtained first preset time period 1 That is, step 2031 is executed to increase the number of nodes in buffer pool 13 to m=2×100=200, at this time, the reference injection number is changed to n= (1+r) ×n ', n' which is the reference injection number before the number of nodes in buffer pool 13 is changed, n is the reference injection number after the number of nodes in buffer pool 13 is changed, that is, n=2×10=20, and the first preset number n is changed with the change of the reference injection number n 1 And a second preset number n of times 2 All change, n 1 ＝40，n 2 =10, then, n=44 > N obtained next time 1 That is, step 2031 is executed to increase the number of nodes in buffer pool 13 to m=2×200=400, and at this time, the reference injection number is changed to n=2×20=40, n 1 ＝80，n 2 =20. It should be noted that if the number of nodes in the buffer pool 13 has reached the maximum number, even if N > N is satisfied 1 Step 2031 is also not performed to prevent excessive occupancy of resources by buffer pool 13.
After each execution of step 2032 to reduce the number of nodes of the buffer pool 13, the ratio of the number of nodes of the buffer pool 13 after the current reduction to the number of nodes of the buffer pool 13 before the current reduction is equal to the ratio of the second preset number of times to the reference injection number of times.
Specifically, for example, the number of nodes of the buffer pool 13 is reduced to m= (1-R) ×m ", m" is the number of nodes of the buffer pool 13 before reduction, m is the number of nodes of the buffer pool 13 after reduction, and the ratio of the number of nodes of the buffer pool 13 after reduction to the number of nodes of the buffer pool 13 before reduction isn 2 = (1-R) ×n, the ratio of the second preset number of times to the reference number of injections is +.>I.e. < ->For example, when the number of nodes in the buffer pool 13 is 400, n=40, n 1 ＝80，n 2 =20, and the number of times n=8 < N of traversing injection of the buffer pool 13 in the obtained first preset time period 2 That is, step 2032 is performed to reduce the number of nodes in buffer pool 13, so that the number of nodes in buffer pool 13 is reduced to m=0.5x400=200, and at this time, the reference injection number is changed to n= (1-R) ×n ', n' which is the reference injection number before the number of nodes in buffer pool 13 is changed, and n which is the reference injection number after the number of nodes in buffer pool 13 is changed, that is, n=0.5x40=20, n 1 ＝40，n 2 =10, then, n=8 < N obtained next time 2 That is, step 2032 is executed to reduce the number of nodes in buffer pool 13 to m=0.5×200=100, and at this time, the reference injection number is changed to n=0.5×20=10, n with the change in the number of nodes in buffer pool 13 1 ＝20，n 2 =5. It should be noted that if the number of nodes of the buffer pool 13 has reached the minimum number, even if N < N is satisfied 2 Step 2032 is also not performed to prevent too few nodes in buffer pool 13.
The reference injection frequency N may be a preset empirical value or a test value, which is used to indicate the injection frequency of the buffer pool 13 under normal conditions, that is, an injection frequency standard value, if the number of times N of performing traversal injection on the buffer pool 13 in the first preset time period actually obtained is greater than N, it indicates that the injection frequency of the current buffer pool 13 is faster, so that when the actual injection frequency value is greater than 1+r times of the injection frequency standard value, the number of nodes of the buffer pool 13 is increased to 1+r times of the original number of nodes of the buffer pool 13, so that the matching degree of the number of nodes of the buffer pool 13 and the injection frequency is closer, but the node is not excessively changed; similarly, if N < N, it indicates that the injection frequency of the current buffer pool 13 is slow, and therefore, each time the actual value of the injection frequency is 1-R times smaller than the standard value of the injection frequency, the number of nodes of the buffer pool 13 is reduced to 1-R times as much as the original number, so that the number of nodes of the buffer pool 13 and the matching degree of the injection frequency are more similar, but do not change too much.
In addition, in other possible embodiments, the value after the number of nodes of each buffer pool 13 is changed may be directly related to the number of times N of performing traversal injection on the buffer pool within the first preset duration, for example, the number of times of performing traversal injection on the buffer pool 13 with the number of nodes being 100 in the first preset duration is 10 in the standard case, the number of nodes of the current buffer pool 13 is 100, and at this time, the number of times of performing traversal injection on the buffer pool 13 within the first preset duration is 30, which is greater than the first preset number of times, and since the actual value of the injection frequency is 3 times of the standard value of the injection frequency, the number of nodes of the buffer pool 13 may be increased to 3 times of the standard frequency, that is, from 100 to 300; similarly, the number of nodes of the current buffer pool 13 is 100, and at this time, the number of times of performing traversal injection on the buffer pool 13 in the first preset time period is 50, which is greater than the first preset time period, and since the actual value of the injection frequency is 5 times of the standard value of the injection frequency, the number of nodes of the buffer pool 13 can be increased to 5 times of the standard frequency, that is, from 100 to 500.
In addition, in the process of reducing the number of nodes in the buffer pool 13 in the step 2032, if the change value of the number of nodes in the buffer pool 13 is less than or equal to the number of empty nodes in the buffer pool 13, all task nodes in the buffer pool 13 are reserved, at least some empty nodes in the buffer pool 13 are removed, and if the change value of the number of nodes in the buffer pool 13 is greater than the number of empty nodes in the buffer pool 13, all empty nodes and some task nodes in the buffer pool 13 are removed, and the task corresponding to the removed task node in the buffer pool 13 is injected into the task pool 14, so that the task allocation relationship between the task corresponding to the removed task node in the buffer pool 13 and the executor 120 is cancelled. For example, before the number of nodes in the buffer pool 13 is reduced from 100 to 50, how there are 40 task nodes and 60 empty nodes in 100 nodes, and then 50 out of 60 empty nodes can be directly removed; if 60 task nodes and 40 empty nodes exist in 100 nodes, then 40 empty nodes are removed, 10 tasks are selected from the 60 task nodes, tasks in the 10 task nodes are injected into the task pool 14, and meanwhile, the task allocation relation between the tasks corresponding to the 10 previous task nodes and the executor 120 is cancelled, so that the executor 120 does not store information related to the tasks any more, and the tasks are no longer used as tasks to be executed which the executor 120 needs to process. The tasks injected into the task pool 14 may be allocated again according to logic of the task pool 14 itself, so as to avoid task loss, and ensure that the tasks removed from the buffer pool 13 are executed in a subsequent process.
Optionally, as shown in fig. 4 and fig. 6, fig. 6 is a flowchart illustrating another refinement step of step 201 in fig. 4, where the step 201 of periodically obtaining the number of times of performing traversal injection on the buffer pool 13 within the first preset duration includes:
step 2011, periodically acquiring the number of times N of traversing injection to the buffer pool, the number m of nodes in the buffer pool 13 and the number p of task nodes in a first preset time period, and then entering step 2012;
step 2012, determining whether there is no change in N, m and p within a second preset time period, if so, entering step 2013, otherwise, entering step 2021;
step 2013, reset buffer pool 13, and then proceed to step 101.
Step 2013, resetting the buffer pool 13 includes:
the task allocation relation between the tasks corresponding to all task nodes in the buffer pool 13 and the executor 120 is cancelled, the tasks corresponding to all task nodes in the buffer pool 13 are injected into the task pool 14, the task nodes become empty nodes, and the buffer pool 13 is emptied;
the task in the task pool 14 is injected into the emptied buffer pool 13, so that all empty nodes in the buffer pool 13 become task nodes, and the task injected into the buffer pool 13 is allocated to the executor 120.
Specifically, after periodically acquiring the parameters N, m and p of the buffer pool 13, firstly, a reset judgment is performed, for example, if each parameter in the buffer pool 13 is unchanged for a long time, which indicates that an abnormality occurs in task allocation, execution or processing of a node in the buffer pool 13, and in order to ensure task processing efficiency, the buffer pool 13 may be reset to restore to an initial state. In the resetting process, the tasks in the buffer pool 13 are firstly returned to the task pool 14, the task returning process comprises the steps of canceling the task allocation relation between the tasks in the buffer pool 13 and the executor 120, and injecting the tasks in the buffer pool 13 into the task pool 14 to avoid the loss of the tasks in the resetting process, so that the tasks in the buffer pool 13 can be executed in the subsequent process, and then the tasks in the task pool 14 are re-injected into the buffer pool 13 to restore the task allocation process of the buffer pool 13.
In addition, as shown in fig. 4 and fig. 7, fig. 7 is a schematic flow chart of a further refinement step of step 201 in fig. 4, where the step 201 of periodically obtaining the number of times of performing traversal injection on the buffer pool 13 within the first preset duration includes:
step 2011, periodically acquiring the number of times N of traversal injection to the buffer pool, the reference injection number of times N, the number of nodes m in the buffer pool 13 and the number of task nodes p in a first preset time period, and then entering step 2012;
Step 2012, determining whether N, m and p have no change within a second preset duration, if so, entering step 2013, and if not, entering step 2014;
step 2013, resetting the buffer pool 13, and then entering step 101, wherein the process of resetting the buffer pool 13 is the same as the above process, and will not be described again here;
step 2014, obtaining a first preset frequency and a second preset frequency according to the reference injection frequency n, wherein the first preset frequency is larger than the reference injection frequency and is positively correlated with the reference injection frequency, and the second preset frequency is smaller than the reference injection frequency and is positively correlated with the reference injection frequency.
The specific process and principle of step 2014 are the same as those of step 2012 in fig. 5, and will not be repeated here.
Optionally, the buffer pool has a Key-Value binary tree structure, where Key is used to store an index of a node in the buffer pool, value is used to store an address, if the address points to task information, the node is a task node, and if the address points to null, the node is a null node. By using the Key-Value binary tree structure as the data storage structure of the buffer pool, a binary tree conforming to the searching performance can be constructed so as to quickly locate the task position and clear the completed task, and meanwhile, the efficient injection of the task is performed without influencing the traversal of the binary tree.
It should be noted that, the task allocation method is only an allocation method corresponding to one buffer pool 13 in the task scheduling system shown in fig. 1, and the task allocation can be performed by the same method for each buffer pool 13 in the system.
It is to be understood that some or all of the steps or operations in the above-described embodiments are merely examples, and that embodiments of the present application may also perform other operations or variations of the various operations. Furthermore, the various steps may be performed in a different order presented in the above embodiments, and it is possible that not all of the operations in the above embodiments are performed.
The embodiment of the application also provides a task allocation device, as shown in fig. 8, fig. 8 is a structural block diagram of the task allocation device in the embodiment of the application, where the device includes: the first acquisition module 1 is used for acquiring the number of task nodes in the buffer pool; the injection module 2 is configured to determine whether the number of task nodes in the buffer pool is smaller than an injection threshold, if yes, perform a traversal injection on the buffer pool, and allocate the tasks injected into the buffer pool to the executor, where performing the traversal injection on the buffer pool includes: traversing each node in the buffer pool, and injecting the task in the task pool into an empty node to change the empty node into a task node; and the execution result processing module 3 is used for changing the corresponding node of the executed task in the buffer pool into an empty node when the execution result of the task of the executor is acquired.
The task allocation device may be applied to the task allocation method in the above embodiment, and the specific working process and principle thereof may be the same as those described in the above embodiment, and will not be described herein.
According to the task allocation device in the embodiment of the application, the buffer pool is arranged between the task pool and the executors, the tasks in the task pool are injected into the buffer pool according to the corresponding task quantity in the buffer pool as judgment basis, the tasks in the buffer pool are allocated to the corresponding executors, decoupling between the executors and the task pool is realized in a buffer mode, the task allocation process of the executors can not directly influence the task pool, and therefore, under the condition of uneven resource distribution, the overall task scheduling system still keeps higher task allocation and execution efficiency.
Optionally, as shown in fig. 9, fig. 9 is a block diagram of another task allocation device according to an embodiment of the present application, where the device further includes: the second obtaining module 4 is used for regularly obtaining the times of traversing injection to the buffer pool in the first preset time period; the node number adjusting module 5 is configured to determine whether the number of times of performing traversal injection on the buffer pool within the first preset duration meets a preset condition, and if yes, adjust the node number of the buffer pool.
Optionally, the injection threshold is positively correlated with the number of nodes of the buffer pool.
Optionally, the node number adjusting module is specifically configured to: determining whether the number of times of traversal injection on the buffer pool in the first preset time period is larger than the first preset number of times, if so, increasing the number of nodes of the buffer pool; determining whether the number of times of traversal injection on the buffer pool in the first preset time period is smaller than the second preset time period, and if yes, reducing the number of nodes of the buffer pool.
Optionally, the first preset number of times is positively correlated with the number of nodes of the buffer pool, and the second preset number of times is positively correlated with the number of nodes of the buffer pool.
Optionally, the magnitude of the change in the number of nodes of each buffer pool is positively correlated with the value before the change in the number of nodes of that buffer pool.
Optionally, the second obtaining module is specifically configured to:
periodically acquiring the times of traversing injection and the reference injection times of the buffer pool in a first preset time period, wherein the reference injection times are positively correlated with the node number of the buffer pool;
obtaining a first preset frequency and a second preset frequency according to the reference injection frequency, wherein the first preset frequency is larger than the reference injection frequency and is positively related to the reference injection frequency, and the second preset frequency is smaller than the reference injection frequency and is positively related to the reference injection frequency;
After the node number of the buffer pool is increased each time, the ratio of the node number of the buffer pool after the current increase to the node number of the buffer pool before the current increase is equal to the ratio of the first preset times to the reference injection times;
after the number of nodes of the buffer pool is reduced each time, the ratio of the number of nodes of the buffer pool after the current reduction to the number of nodes of the buffer pool before the current reduction is equal to the ratio of the second preset times to the reference injection times.
Optionally, the second obtaining module is specifically configured to:
periodically acquiring the times N of traversing injection of the buffer pool, the number m of nodes in the buffer pool and the number p of task nodes in a first preset time period;
determining whether N, m and p have no change in a second preset time period, and if so, resetting the buffer pool;
resetting the buffer pool includes:
the task allocation relation between the tasks corresponding to all task nodes in the buffer pool and the executor is cancelled, the tasks corresponding to all task nodes in the buffer pool are injected into the task pool, the task nodes become empty nodes, and the buffer pool is emptied;
and injecting the tasks in the task pool into the emptied buffer pool, enabling all empty nodes in the buffer pool to be changed into task nodes, and distributing the tasks injected into the buffer pool to the executor.
It should be understood that the above division of the modules of the apparatus shown in fig. 8-9 is merely a division of a logic function, and may be fully or partially integrated into a physical entity or may be physically separated. And these modules may all be implemented in software in the form of calls by the processing element; or can be realized in hardware; it is also possible that part of the modules are implemented in the form of software called by the processing element and part of the modules are implemented in the form of hardware. The processing element described herein may be an integrated circuit having signal processing capabilities. In implementation, each step of the above method or each module above may be implemented by an integrated logic circuit of hardware in a processor element or an instruction in a software form.
For example, the modules above may be one or more integrated circuits configured to implement the methods above, such as: one or more specific integrated circuits (Application Specific Integrated Circuit, ASIC), or one or more microprocessors (digital singnal processor, DSP), or one or more field programmable gate arrays (Field Programmable Gate Array, FPGA), or the like. For another example, when a module above is implemented in the form of a processing element scheduler, the processing element may be a general purpose processor, such as a central processing unit (Central Processing Unit, CPU) or other processor that may invoke the program.
The embodiment of the application also provides a task allocation device, as shown in fig. 10, fig. 10 is a structural block diagram of the task allocation device in the embodiment of the application, where the task allocation device includes: a processor 610 and a memory 620, the memory 620 for storing at least one instruction which, when loaded and executed by the processor 610, implements the methods of the embodiments described above. The number of processors 610 may be one or more, with one processor 610 being illustrated in fig. 8, and the processors 610 and memory 620 being connected by a bus or otherwise, with a bus connection being illustrated in fig. 10.
The memory 620 is used as a non-transitory computer readable storage medium for storing non-transitory software programs, non-transitory computer executable programs, and modules, such as program instructions/modules corresponding to the transmission method in the embodiment of the present application. The processor 610 executes various functional applications and data processing, i.e., implements the methods of any of the method embodiments described above, by running non-transitory software programs, instructions, and modules stored in the memory 620.
Memory 620 may include a storage program area that may store an operating system, at least one application program required for functionality, and a storage data area; and necessary data, etc. In addition, memory 620 may include high-speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid state storage device.
The task allocation device may be, for example, a server, and be applied to the task scheduling system shown in fig. 1, and is used as the task allocation device 11, where the executor set 12 may be, for example, different servers, and multiple executors 120 in the same executor set 12 may be, for example, different processes configured in the servers, and the buffer pool 13 and the task pool 14 may be formed by the task allocation device 11 or storage data areas in other servers.
The embodiment of the present application also provides a computer-readable storage medium having stored therein a computer program which, when run on a computer, causes the computer to perform the method described in the above embodiment.
Furthermore, embodiments of the present application provide a computer program product comprising a computer program which, when run on a computer, causes the computer to perform the method according to the above embodiments.
In the above embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions in accordance with the present application are produced in whole or in part. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by a wired (e.g., coaxial cable, fiber optic, digital subscriber line), or wireless (e.g., infrared, wireless, microwave, etc.). The computer readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that contains an integration of one or more available media. The usable medium may be a magnetic medium (e.g., floppy Disk, hard Disk, tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid State Disk), etc.
The foregoing description of the preferred embodiments of the invention is not intended to be limiting, but rather to enable any modification, equivalent replacement, improvement or the like to be made within the spirit and principles of the invention.
Claims (12)
1. A method of task allocation, comprising:
acquiring the number of task nodes in a buffer pool;
determining whether the number of task nodes in the buffer pool is smaller than an injection threshold, if so, performing one-time traversal injection on the buffer pool, and distributing the tasks injected into the buffer pool to an executor, wherein the performing one-time traversal injection on the buffer pool comprises: traversing each node in the buffer pool, and injecting tasks in the task pool into empty nodes to enable the empty nodes to be changed into task nodes;
when a task execution result of the executor is obtained, changing a corresponding node of the executed task in the buffer pool into an empty node;
periodically obtaining the times of traversing injection of the buffer pool in a first preset time period;
determining whether the times of performing traversal injection on the buffer pool within the first preset duration meets preset conditions, and if yes, adjusting the node number of the buffer pool;
The process of periodically obtaining the times of performing traversal injection on the buffer pool within the first preset time period comprises the following steps:
periodically acquiring the times of traversing injection and the reference injection times of the buffer pool in a first preset time period, wherein the reference injection times are positively correlated with the node number of the buffer pool;
obtaining a first preset time and a second preset time according to the reference injection time, wherein the first preset time is larger than the reference injection time and is positively correlated with the reference injection time, and the second preset time is smaller than the reference injection time and is positively correlated with the reference injection time;
after the node number of the buffer pool is increased each time, the ratio of the node number of the buffer pool after the current increase to the node number of the buffer pool before the current increase is equal to the ratio of the first preset times to the reference injection times;
after the number of nodes of the buffer pool is reduced each time, the ratio of the number of nodes of the buffer pool after the current reduction to the number of nodes of the buffer pool before the current reduction is equal to the ratio of the second preset times to the reference injection times.
2. The method of claim 1, wherein the step of determining the position of the substrate comprises,
the injection threshold is positively correlated with the number of nodes of the buffer pool.
3. The method of claim 1, wherein the step of determining the position of the substrate comprises,
determining whether the number of times of performing traversal injection on the buffer pool within the first preset time period meets a preset condition, and if yes, adjusting the number of nodes of the buffer pool includes:
determining whether the number of times of traversal injection on the buffer pool in the first preset time period is larger than a first preset number of times, if so, increasing the number of nodes of the buffer pool;
determining whether the number of times of traversal injection on the buffer pool in the first preset time period is smaller than a second preset time period, and if yes, reducing the number of nodes of the buffer pool.
4. The method of claim 3, wherein the step of,
the first preset times are positively correlated with the number of nodes of the buffer pool, and the second preset times are positively correlated with the number of nodes of the buffer pool.
5. The method of claim 3, wherein the step of,
the magnitude of the change in the number of nodes of the buffer pool each time is positively correlated with the value before the change in the number of nodes of the buffer pool each time.
6. A task assigning apparatus, comprising:
the first acquisition module is used for acquiring the number of task nodes in the buffer pool;
The injection module is configured to determine whether the number of task nodes in the buffer pool is smaller than an injection threshold, if yes, perform a traversal injection on the buffer pool, and allocate the tasks injected into the buffer pool to an executor, where performing the traversal injection on the buffer pool includes: traversing each node in the buffer pool, and injecting tasks in the task pool into empty nodes to enable the empty nodes to be changed into task nodes;
the execution result processing module is used for enabling corresponding nodes of the executed task in the buffer pool to become empty nodes when the task execution result of the executor is acquired;
the second acquisition module is used for periodically acquiring the times of traversal injection of the buffer pool in a first preset duration;
the node quantity adjusting module is used for determining whether the times of performing traversal injection on the buffer pool in the first preset duration meet preset conditions or not, and if yes, adjusting the node quantity of the buffer pool;
the second obtaining module is specifically configured to:
periodically acquiring the times of traversing injection and the reference injection times of the buffer pool in a first preset time period, wherein the reference injection times are positively correlated with the node number of the buffer pool;
Obtaining a first preset time and a second preset time according to the reference injection time, wherein the first preset time is larger than the reference injection time and is positively correlated with the reference injection time, and the second preset time is smaller than the reference injection time and is positively correlated with the reference injection time;
after the node number of the buffer pool is increased each time, the ratio of the node number of the buffer pool after the current increase to the node number of the buffer pool before the current increase is equal to the ratio of the first preset times to the reference injection times;
after the number of nodes of the buffer pool is reduced each time, the ratio of the number of nodes of the buffer pool after the current reduction to the number of nodes of the buffer pool before the current reduction is equal to the ratio of the second preset times to the reference injection times.
7. The apparatus of claim 6, wherein the device comprises a plurality of sensors,
the injection threshold is positively correlated with the number of nodes of the buffer pool.
8. The apparatus of claim 6, wherein the device comprises a plurality of sensors,
the node quantity adjusting module is specifically configured to:
determining whether the number of times of traversal injection on the buffer pool in the first preset time period is larger than a first preset number of times, if so, increasing the number of nodes of the buffer pool;
Determining whether the number of times of traversal injection on the buffer pool in the first preset time period is smaller than a second preset time period, and if yes, reducing the number of nodes of the buffer pool.
9. The apparatus of claim 8, wherein the device comprises a plurality of sensors,
the first preset times are positively correlated with the number of nodes of the buffer pool, and the second preset times are positively correlated with the number of nodes of the buffer pool.
10. The apparatus of claim 8, wherein the device comprises a plurality of sensors,
the magnitude of the change in the number of nodes of the buffer pool each time is positively correlated with the value before the change in the number of nodes of the buffer pool each time.
11. A task assigning apparatus, characterized by comprising:
a processor and a memory for storing at least one instruction which, when loaded and executed by the processor, implements the method of any one of claims 1 to 5.
12. A computer readable storage medium, characterized in that the computer readable storage medium has stored therein a computer program which, when run on a computer, causes the computer to perform the method according to any of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910895064.5A CN111324428B (en) | 2019-09-20 | 2019-09-20 | Task allocation method, device, equipment and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910895064.5A CN111324428B (en) | 2019-09-20 | 2019-09-20 | Task allocation method, device, equipment and computer readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111324428A CN111324428A (en) | 2020-06-23 |
| CN111324428B true CN111324428B (en) | 2023-08-22 |
Family
ID=71166863
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910895064.5A Active CN111324428B (en) | 2019-09-20 | 2019-09-20 | Task allocation method, device, equipment and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111324428B (en) |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5781801A (en) * | 1995-12-20 | 1998-07-14 | Emc Corporation | Method and apparatus for receive buffer management in multi-sender communication systems |
| US5784698A (en) * | 1995-12-05 | 1998-07-21 | International Business Machines Corporation | Dynamic memory allocation that enalbes efficient use of buffer pool memory segments |
| WO2002019168A1 (en) * | 2000-08-25 | 2002-03-07 | Hayes Scott R | Heuristic automated method for ideal bufferpool tuning in a computer database |
| CN103605567A (en) * | 2013-10-29 | 2014-02-26 | 河海大学 | Cloud computing task scheduling method facing real-time demand change |
| CN104580396A (en) * | 2014-12-19 | 2015-04-29 | 华为技术有限公司 | Task scheduling method, node and system |
| WO2017113278A1 (en) * | 2015-12-31 | 2017-07-06 | 华为技术有限公司 | Data processing method, apparatus and system |
| CN107153573A (en) * | 2016-03-02 | 2017-09-12 | 阿里巴巴集团控股有限公司 | Distributed task scheduling treating method and apparatus |
| CN108510150A (en) * | 2018-02-01 | 2018-09-07 | 东华大学 | A kind of spinning CPS and its real-time task processing method based on edge calculations |
| WO2018219480A1 (en) * | 2017-05-29 | 2018-12-06 | Barcelona Supercomputing Center - Centro Nacional De Supercomputación | Managing task dependency |
-
2019
- 2019-09-20 CN CN201910895064.5A patent/CN111324428B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5784698A (en) * | 1995-12-05 | 1998-07-21 | International Business Machines Corporation | Dynamic memory allocation that enalbes efficient use of buffer pool memory segments |
| US5781801A (en) * | 1995-12-20 | 1998-07-14 | Emc Corporation | Method and apparatus for receive buffer management in multi-sender communication systems |
| WO2002019168A1 (en) * | 2000-08-25 | 2002-03-07 | Hayes Scott R | Heuristic automated method for ideal bufferpool tuning in a computer database |
| CN103605567A (en) * | 2013-10-29 | 2014-02-26 | 河海大学 | Cloud computing task scheduling method facing real-time demand change |
| CN104580396A (en) * | 2014-12-19 | 2015-04-29 | 华为技术有限公司 | Task scheduling method, node and system |
| WO2017113278A1 (en) * | 2015-12-31 | 2017-07-06 | 华为技术有限公司 | Data processing method, apparatus and system |
| CN107153573A (en) * | 2016-03-02 | 2017-09-12 | 阿里巴巴集团控股有限公司 | Distributed task scheduling treating method and apparatus |
| WO2018219480A1 (en) * | 2017-05-29 | 2018-12-06 | Barcelona Supercomputing Center - Centro Nacional De Supercomputación | Managing task dependency |
| CN108510150A (en) * | 2018-02-01 | 2018-09-07 | 东华大学 | A kind of spinning CPS and its real-time task processing method based on edge calculations |
Non-Patent Citations (1)
| Title |
|---|
| 云计算中任务分配研究;蒋维成;李兰英;刘华春;柳军;;信息技术(09);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111324428A (en) | 2020-06-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108897628B (en) | Method and device for realizing distributed lock and electronic equipment | |
| CN106712981B (en) | Node change notification method and device | |
| CN111309644B (en) | Memory allocation method and device and computer readable storage medium | |
| US11675622B2 (en) | Leader election with lifetime term | |
| US9152457B2 (en) | Processing request management | |
| CN114168490A (en) | Method for determining memory recovery threshold and related equipment | |
| CN111158892B (en) | Task queue generating method, device and equipment | |
| CN111309548B (en) | Timeout monitoring method and device and computer readable storage medium | |
| CN109889406B (en) | Method, apparatus, device and storage medium for managing network connection | |
| CN111163140A (en) | Method, apparatus and computer readable storage medium for resource acquisition and allocation | |
| CN111324428B (en) | Task allocation method, device, equipment and computer readable storage medium | |
| CN111400241B (en) | Data reconstruction method and device | |
| CN116737345A (en) | Distributed task processing system, distributed task processing method, distributed task processing device, storage medium and storage device | |
| US9858185B1 (en) | Multi-tier data storage using inclusive/exclusive burst buffer caching based on reference counts | |
| CN111143063A (en) | Task resource reservation method and device | |
| CN112905322B (en) | Resource locking method, computing device and computer storage medium | |
| CN112395063B (en) | Dynamic multithreading scheduling method and system | |
| CN112035498B (en) | Data block scheduling method and device, scheduling layer node and storage layer node | |
| CN112506896B (en) | Data deleting method and device and electronic equipment | |
| KR20220069113A (en) | Data structures, storage means and devices | |
| CN111324438A (en) | Request scheduling method and device, storage medium and electronic equipment | |
| JP6551049B2 (en) | Bandwidth control circuit, arithmetic processing unit, and bandwidth control method of the device | |
| CN115599845B (en) | Service execution method, device, storage medium and electronic equipment | |
| CN109710179B (en) | Storage service quality control method and device | |
| CN114465958B (en) | Input and output control method, device and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |