CN107291547B - Task scheduling processing method, device and system - Google Patents
Task scheduling processing method, device and system Download PDFInfo
- Publication number
- CN107291547B CN107291547B CN201610197298.9A CN201610197298A CN107291547B CN 107291547 B CN107291547 B CN 107291547B CN 201610197298 A CN201610197298 A CN 201610197298A CN 107291547 B CN107291547 B CN 107291547B
- Authority
- CN
- China
- Prior art keywords
- task
- processing
- channel
- job
- resource
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
- G06F9/5038—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals considering the execution order of a plurality of tasks, e.g. taking priority or time dependency constraints into consideration
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Hardware Redundancy (AREA)
Abstract
The application provides a task scheduling processing method, device and system. The method may include mapping task processing channel information in a resource pool to channel resource information of a configuration center; when a scheduling request of at least one job task is received, inquiring whether idle channel resources for processing the job task exist in the configuration center; and when the query result of the configuration center indicates that the idle channel resources exist, selecting a task processing channel corresponding to the scheduling request from the resource pool according to the resource allocation priority set for the job task and allocating the task processing channel to the corresponding job task. By utilizing the method and the device, the problem of channel blockage caused by single operation deadlock can be solved by adopting a processing mode of configuring multiple channels, multiple operation tasks can be submitted and processed in parallel, required resources are rapidly and efficiently distributed to the operation tasks, and the task scheduling processing efficiency and the resource utilization rate are greatly improved.
Description
Technical Field
The present application relates to the field of computer data processing, and in particular, to a method, an apparatus, and a system for task scheduling processing.
Background
With the development of information technology and the increase of data service processing requirements, distributed systems are receiving more and more attention due to high cohesiveness and transparent lines.
In a distributed task processing scenario, there is typically one Driver (master scheduler or Driver) per task scheduling multiple Excutors (executors or executors). Fig. 1 is a schematic diagram of a task scheduling processing scheme in an existing distributed scenario, and as shown in fig. 1, a physical cluster may be deployed in the prior art, where a server in the physical cluster generally adopts a master-slave architecture mode, that is, a master (master server) manages and schedules multiple slaves (slave servers). The running program can distribute the jar packet to all machines in the cluster and apply for machine CPUs, memory resources and the like to the maser. A submit command may then be initiated, the main program submitted, and the entire scheduling task initiated. And applying for job resources to the maser according to the job request sent by the user. And starting the current job task after the application is successful, and specifically, distributing the current job task to a plurality of slave of a master under the master for processing. And releasing the operation resources after the current operation task is executed. The next job task is then processed again according to the user's job request.
In the prior art, the operation mode of such processing tasks in the distributed scenario is performed in a distributed manner at a job execution level, such as a slave or an executor, but a single Driver schedules tasks to the job execution level according to a single thread. This results in a single job occupying all resources and wasting other spare resources if one job requires less resources. In addition, in the first-in first-out mechanism, if the execution time of one job is long, the following job tasks cannot be allocated with resources, and a channel is blocked, which causes the job starvation. Therefore, in the prior art, the single-channel task scheduling processing mechanism in the distributed scenario has the problems of resource waste, incapability of executing subsequent operations when the operations are abnormal, and low processing efficiency.
Disclosure of Invention
The application aims to provide a task scheduling processing method, a task scheduling processing device and a task scheduling processing system, which can solve the problem of channel blockage caused by single operation deadlock by adopting a processing mode of configuring multiple channels, can provide parallel submission and parallel processing of multiple operation tasks, quickly and efficiently allocate required resources to the operation tasks, and greatly improve the task scheduling processing efficiency and the resource utilization rate.
The task scheduling processing method, device and system provided by the application are realized as follows:
a method of task scheduling processing, the method comprising:
mapping task processing channel information in the resource pool into channel resource information of a configuration center;
when a scheduling request of at least one job task is received, inquiring whether idle channel resources for processing the job task exist in the configuration center;
and when the query result of the configuration center indicates that the idle channel resources exist, selecting a task processing channel corresponding to the scheduling request from the resource pool according to the resource allocation priority set for the job task and allocating the task processing channel to the corresponding job task.
A task scheduling processing apparatus, the apparatus comprising:
the resource pool is used for storing the resources of the deployed task processing channel;
the configuration center is used for acquiring channel resource information mapped by the task processing channel information in the resource pool;
the resource query module is used for querying whether idle channel resources for processing the job task exist or not from the configuration center when a scheduling request of at least one job task is received;
and the scheduling processing module is used for selecting a task processing channel corresponding to the scheduling request from the resource pool to allocate to the corresponding job task according to the resource allocation priority set for the job task when the query result of the configuration center indicates that the idle channel resource exists.
A distributed task scheduling processing system, the system comprising:
the system comprises a resource pool comprising at least one physical cluster, wherein the physical cluster comprises a deployed task processing channel consisting of a main scheduling server and a corresponding execution server;
the configuration center is used for mapping the task processing channel in the resource pool into channel resource information, acquiring the survival condition of the task processing channel in the resource pool through heartbeat, and updating the task processing channel information in the resource pool in real time.
The scheduling cluster is used for inquiring whether idle channel resources for processing the job task exist in the configuration center according to the received scheduling request of the job task; and the resource processing module is further configured to select a task processing channel corresponding to the scheduling request from the resource pool according to the resource allocation priority set for the job task to allocate to the corresponding job task when the query result of the configuration center indicates that the idle channel resource exists.
According to the task scheduling processing method, the task scheduling processing device and the task scheduling processing system, a main scheduling server for processing job tasks and corresponding execution servers can be abstracted into channels in a distributed scene, and a plurality of channels can register channel information of the channels to a unified configuration center and map the channel information into channel resource information in the configuration center. When the job task needs to use resources, channel resource information can be acquired from the configuration center, and whether available idle channel resources of the job task exist at present is judged according to the resource information in the resource pool. If so, scheduling resources may be allocated for task job execution. In this way, in the process of task scheduling processing, when a scheduling request of a plurality of job tasks is received, each scheduling task can inquire whether available idle channel resources exist in a configuration center, and then allocate corresponding resources to the plurality of job tasks according to the set job task priority according to the inquiry result of the configuration center, so that the multi-channel parallel processing resource allocation for a plurality of concurrently submitted tasks is realized. In the queuing waiting process of the job tasks without preempting the resources, the configuration center feeds back idle channel resources after updating, and therefore the resources can be rapidly allocated to the job tasks in the queue. The task scheduling processing method can realize multi-channel resource allocation, can allocate task resources to jobs as required, solves the problem that the subsequent tasks are not executable due to channel blockage, provides the resource utilization rate, and improves the task scheduling processing efficiency and the resource utilization rate.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below, it is obvious that the drawings in the following description are only some embodiments described in the present application, and for those skilled in the art, other drawings can be obtained according to the drawings without any creative effort.
FIG. 1 is a flowchart illustrating a method according to an embodiment of a task scheduling processing method of the present application;
FIG. 2 is a schematic diagram of a scheduling framework deployed in the task scheduling processing method according to the present application;
FIG. 3 is a flowchart illustrating a task scheduling process according to another embodiment of the present disclosure;
FIG. 4 is a flowchart illustrating a task scheduling process according to another embodiment of the present disclosure;
FIG. 5 is a flowchart illustrating a task scheduling process according to another embodiment of the present disclosure;
FIG. 6 is a block diagram illustrating an exemplary task scheduling apparatus according to the present invention;
FIG. 7 is a block diagram illustrating an exemplary scheduling module of the task scheduling apparatus according to an embodiment;
FIG. 8 is a block diagram illustrating an exemplary task scheduling apparatus according to the present invention;
fig. 9 is a schematic block diagram of another embodiment of a task scheduling processing apparatus according to the present application;
fig. 10 is a schematic block diagram of another embodiment of a task scheduling processing apparatus according to the present application.
Detailed Description
In order to make those skilled in the art better understand the technical solutions in the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
Fig. 1 is a flowchart of a task scheduling processing method according to an embodiment of the present application. Although the present application provides method operational steps or apparatus configurations as illustrated in the following examples or figures, more or fewer operational steps or module configurations may be included in the method or apparatus based on conventional or non-inventive efforts. In the case of steps or structures where there is no logically necessary cause-and-effect relationship, the execution order of the steps or the block structure of the apparatus is not limited to the execution order or the block structure provided in the embodiments of the present application. The described methods or modular structures, as applied to a device or end product in practice, may be executed sequentially or in parallel (e.g., in the context of parallel processors or multi-threaded processing) in accordance with the embodiments or with the modular structures illustrated in the figures.
Specifically, as shown in fig. 1, the task scheduling processing method may include:
s1: and mapping the task processing channel information in the resource pool into the channel resource information of the configuration center.
The resource pool described in this embodiment may generally include a channel resource set for implementing job task deployment, and specifically may include a plurality of one or more physical clusters. Or in some embodiments the resource pool may also comprise a virtual resource pool, which may include one or more logical clusters deployed. Of course, the resource pool may also be deployed to include a physical cluster and a logical cluster according to the requirements of the actual distributed task processing scenario. In the application scenario of this embodiment, the resource pool may be deployed to include a plurality of physical clusters that process different task types, and the servers in one physical cluster are disposed in the same machine room.
Fig. 2 is a schematic diagram of a deployed scheduling processing framework in the task scheduling processing method according to the present application. In general, one or more task processing channels may be included in each physical cluster. The task processing channel in this embodiment may be composed of a master scheduling server (Driver) and a plurality of execution servers (instantiators) managed by the master scheduling server (Driver). In this way, a task processing channel in the present application may be considered as a channel resource that can be used for processing a job task, and the size of the resource of the task processing channel, such as the number of execution servers included in the master scheduling server, may reflect the job capability of the task processing channel to some extent. In this example, the task processing channels in one physical cluster may be allocated to the same resource size, that is, when the physical cluster is deployed, each task processing channel is set to have one main scheduling server corresponding to the same number of execution servers. Of course, in some other embodiments of the present application, the task processing channels in the resource pool are configured to be allocated as resources with different sizes according to preset processing requirements.
In a distributed task processing scenario, the individual task processing channel information in the deployed resource pool may be mapped to channel resource information of the configuration center. In the distributed task processing scenario of this embodiment, a configuration center may be set for task scheduling deployment, where the configuration center may be used to record and update channel resource information of a task processing channel in the resource pool, and may also provide a query access interface for a resource library in a scheduling cluster, so that a resource applicant may obtain resource information in the resource pool. In this embodiment, the task processing channel of the resource pool may register its own task processing channel information (resource condition) with the configuration center. After the registration is successful, the configuration center may map the task processing channels of each task processing channel of the resource pool into channel resource information of the configuration center according to the corresponding data format, and the configuration center may store and update the channel resource information in real time.
Generally, when a certain task processing channel is started to be used in a physical cluster, the resource condition of the resource can be registered with the configuration center. In an embodiment of the task scheduling processing method of the present application, the configuration center may further sense whether the task processing channel is still alive and effective through a heartbeat. When the task processing channel fails (such as a lower line or a fault), the channel resource information of the configuration center is correspondingly updated. In the actual task scheduling processing operation process, the occupation usage of the task processing channels in the resource pool is often dynamically changed in real time, and therefore, in an embodiment of the present application, the configuration center may be configured to sense the survival of each task channel in the resource pool in real time and update the occupation of the task processing channels in real time. Therefore, the channel resource information of the configuration center can accurately and reliably reflect the resource condition of the task processing channel in the current resource pool. Therefore, in another embodiment of a task scheduling processing method described in the present application, the configuration center may be configured to include:
acquiring the survival condition of the resource pool task processing channel through heartbeat; and the number of the first and second groups,
and updating the task processing channel information in the resource pool in real time.
The task processing channel information described in this embodiment may include information of resource conditions of the task processing channels, for example, if there is an idle task processing channel P1, the P1 includes a main scheduling server D1 and 10 corresponding execution servers E1 to E10. The channel resource information of the configuration center can reflect which task processing channels in which physical clusters are occupied in the current resource pool, and which task processing channels are idle and can be applied for occupied resource condition information.
In the embodiment of distributed task processing, the task processing channel information in the deployed resource pool can be mapped to the channel resource information of the configuration center. The configuration center may store and update channel resource information.
S2: and when a scheduling request of at least one job task is received, inquiring whether idle channel resources for processing the job task exist in the configuration center.
In the application scenario for implementing distributed task processing, when one or more job tasks need to be processed, the scheduling processing module may acquire channel resource information from the resource library. The resource library described in this embodiment may query and apply for resources from any configuration center, and if a resource application comes, the resource application may be submitted to the corresponding configuration center as a task. As shown in fig. 2, in the present application, a scheduling cluster of multiple resource pools may exist at the same time, that is, in the present application, scheduling requests of multiple job tasks are allowed to be processed simultaneously and in parallel. Specifically, when receiving a scheduling request of one or more job tasks, the resource pool in the scheduling cluster may query the configuration center whether there is a free channel resource for processing the job task currently. The configuration center stores channel resource information of the task processing channel information in the resource pool, and can respond to the query of the scheduling cluster.
When a scheduling request of one or more job tasks is received, whether idle channel resources for processing the job tasks exist or not can be inquired from the configuration center.
S3: and when the query result of the configuration center indicates that the idle channel resources exist, selecting a task processing channel corresponding to the scheduling request from the resource pool according to the resource allocation priority set for the job task and allocating the task processing channel to the corresponding job task.
The configuration center can record the channel resource information of the occupation execution condition of the current task processing channel in the resource pool, and can judge whether the idle channel resource for processing the job task exists according to the resource information applied in the scheduling request of the job task. If the scheduling requests of the current two job tasks T1 and T2 each require a task processing channel in the physical cluster a, the resource pools S1 and S2, which may be responsible for T1 and T2, query the configuration center whether there is a free channel resource for processing the job tasks T1 and T2. And the configuration center inquires whether a free channel resource capable of processing the job tasks T1 and T2 exists according to the recorded channel resource information of all the task processing channels in the resource pool. If the query result of the configuration center is that there are currently 6 idle channel resources of T1 and T2 that can process the job task, the idle task processing channel in the resource pool can be selected to be allocated to the corresponding task.
According to the task scheduling processing method, a plurality of task processing channels can be provided to distribute channel resources required by job tasks in parallel. In some application scenarios of the present application, when a scheduling request of a plurality of job tasks is received, a plurality of resource pools of a scheduling cluster may all query a configuration center whether there is an idle channel resource for processing the job task corresponding to the resource pool. At this time, if the information of the configuration center indicates that there is an idle channel resource in the current resource pool, but the number of resources is less than the resource request amount of the multiple job tasks, this embodiment may set a resource allocation priority for the concurrently submitted distributed tasks in advance according to a certain manner, so as to solve the problem of channel resource allocation when the multiple job tasks concurrently submit application resources. Specifically, the priority of resource allocation can be set when the job task contends for resources in a pre-selected manner according to the task scheduling processing requirement of the distributed task processing.
An embodiment of the present application provides a resource allocation processing method when a distributed task is concurrently submitted. Specifically, in another embodiment of the task scheduling processing method according to the present application, the setting of the resource allocation priority for the job task may include:
s301: when the configuration center is inquired that the idle channel resources exist and the number of the idle channel resources is smaller than the amount of the application resources of the current multiple job tasks, any one of the time sequence of applying the resources by the job tasks in the multiple job tasks and the weight priority of the job tasks is used as the resource allocation priority of the job tasks.
In a specific implementation process, a task scheduling processing method for concurrently submitting distributed tasks and applying for resource jobs can be set according to design requirements. For example, the resource allocation may be performed in the order of the job task application resource. When the number of resources in the resource pool is less than the resource application amount of a plurality of tasks submitted concurrently, the job task which submits the resource application first can preferentially preempt idle task processing channel resources, and the job task which is submitted later fails in resource preemption. Of course, the resource job task is not preempted, and the scheduling can be returned again to continue to apply for the resource. The job task applied to the corresponding task processing channel resource can execute the designated task processing running logic by using the task processing channel.
According to the task scheduling processing method, the main scheduling server for processing the job task and the corresponding execution server can be abstracted into channels in a distributed scene, and the channels can register own channel information to a unified configuration center and map the channel information into channel resource information in the configuration center. When the job task of the job request end needs to use resources, the channel resource information can be obtained from the configuration center, and whether idle channel resources available for the job task exist at present is judged according to the resource information in the resource pool. If so, scheduling resources may be allocated for task job execution. In this way, in the process of task scheduling processing, when a scheduling request of a plurality of job tasks is received, each scheduling task can inquire whether available idle channel resources exist in a configuration center, and then allocate corresponding resources to the plurality of job tasks according to the set job task priority according to the inquiry result of the configuration center, so that the multi-channel parallel processing resource allocation for a plurality of concurrently submitted tasks is realized. In the queuing waiting process of the job tasks without preempting the resources, the configuration center feeds back idle channel resources after updating, and therefore the resources can be rapidly allocated to the job tasks in the queue. The task scheduling processing method can realize multi-channel resource allocation, can allocate task resources to jobs as required, solves the problem that the subsequent tasks are not executable due to channel blockage, provides a resource utilization rate, and improves task scheduling processing efficiency.
As described above, in an embodiment of the present application, the configuration center may store channel resource information reflecting task processing channels in the resource pool. In a specific embodiment, whether the task processing channel is alive or not can be known through heartbeat-aware human processing, for example, the configuration center judges whether the task processing channel is failed or not through survival of a heartbeat-aware primary scheduling server. When a main scheduling server of a certain task processing channel in the resource pool fails, such as an abnormal fault or a resource offline, the configuration center can update the channel resource information in real time. Meanwhile, if there is a job task that has been run or allocated but has not executed the failed task processing channel resource, in an embodiment provided by the present application, the following manner may be adopted for processing, so as to avoid a situation that a single job task is abnormal, causing a task loss or a subsequent task job cannot be processed. Specifically, in another embodiment of a task scheduling processing method provided by the present application, the method may further include:
s401: when the task processing channel in the resource pool is judged to be invalid, updating the channel resource information corresponding to the configuration center;
and the number of the first and second groups,
s402: and processing the job task called to the failed task processing channel according to the scheduling request in the following way:
s4021: interrupting the job task executing the failed task processing channel;
s4022: abandoning the job task of the queued task processing channel which is not executed;
s4023: and rescheduling the job task which is called to the failed task processing channel, and distributing a corresponding task processing channel for the rescheduling job task according to the channel resource information updated by the configuration center.
Fig. 3 is a flowchart illustrating a method according to another embodiment of a task scheduling processing method according to the present application. In a specific execution process, whether a master dispatch server (Driver) is alive or not can be sensed through heartbeat. When the master scheduling server is sensed to be invalid, the corresponding channel resource information of the task processing channel information registered in the configuration center is also invalid. When the channel resource is offline, the interruption of the allowed job task can be directly closed, such as the running logic can be set to be identified as failure. And the number of the workers who are queuing to call the failed or offline task processing channel can be abandoned. Furthermore, the job task called to the failed task processing channel can be rescheduled, for example, the interrupted or abandoned job task can be rescheduled back to the scheduling cluster after delaying the preset time, and the channel resource is reapplied for task processing. Therefore, the processing method provided by the embodiment can not cause the problem of job task execution loss when the resources are off-line in the application scene of distributed task processing, and meanwhile, if one task channel is abnormal in the execution process, the job tasks which are subsequently queued and use the abnormal task processing channel can be executed by selecting other channels. Of course, the job task that is executing the exception handling channel may also be interrupted and another channel reselected to execute the job task.
In another application scenario of the task scheduling processing method, processing time of different job tasks is often different. Fig. 4 is a flowchart illustrating a method according to another embodiment of a task scheduling processing method according to the present application. In the application scenario of distributed task processing, the original reason that some workers have long processing time may be due to own job processing requirements. However, if the processing time of a job task is too long and exceeds the normal execution period, and it can be considered that the job task is abnormal in the processing process at this time, the job task with the too long execution time exceeding the set time threshold should be processed correspondingly. Therefore, in another embodiment of the task scheduling processing method according to the present application, a thread may be started when a job task is executed, so as to monitor an execution time of a task processing channel, and specifically, the method may further include:
s501: and monitoring the occupied execution time of the task processing channel of the job task, and stopping the job task with the occupied time being more than the set maximum execution time threshold when the occupied execution time is more than the set maximum execution time threshold.
The job task that is stopped may be set to exception handling and logged. Furthermore, a thread can be set for scanning the job tasks recorded as exception handling in the job tasks, and then determining whether to continue scheduling the exception handling job tasks according to the preset user configuration. Fig. 5 is a flowchart illustrating a method according to another embodiment of a task scheduling processing method according to the present application. As shown in fig. 5, in another embodiment of a task scheduling processing method provided by the present application, the method may further include:
s502: and scanning and acquiring the job task marked as abnormal processing according to a preset period, and determining whether to continuously call the job task marked as abnormal processing according to the set scheduling processing configuration information.
In a specific implementation process, if the scheduling processing configuration information of the user is set to be negative, the job task occupying the execution time greater than the maximum execution time threshold can be killed, the job task is discarded, and corresponding log records are performed. If the configuration information of the scheduling processing is set to be yes, the job tasks marked as abnormal processing can be rescheduled after a period of time delay according to the setting of the configuration information, channel resources can be reapplied, resources can be allocated, and the like. In the embodiment of the present application, the job task executing the logic running process is killed, and is converted to another channel for continuous processing, so as to free up time and channel resources for processing the subsequent job task. The implementation scheme of the embodiment can greatly improve the utilization rate of channel resources in the resource pool and improve the scheduling processing efficiency of the job task.
In the above-described embodiments, the task processing channels in the resource pool may be set to have the same resource allocation size, and the job processing capability of each task processing channel in one physical cluster in the resource pool is the same. In another embodiment of the task scheduling processing method, task processing channels with different job processing energies and different resource allocation sizes can be set in the same type of physical cluster. The task processing channel in the resource pool may include, in the task processing channel information registering itself with the configuration center, job capability indicating the task processing channel, or may also include other information such as a channel type. In this way, the client of the job request can register the channel type of interest with the configuration center, and acquire all task processing channel information of the channel type. When the job task really starts scheduling processing, the idle channel resource with the strongest current processing capability can be allocated for the job task to process. And updating the channel resource information to the registration center after the processing is finished. Therefore, the present application further provides another embodiment of a task scheduling processing method, where the channel resource information of the configuration center includes information of job processing capability of the task processing channel;
correspondingly, the selecting a task processing channel corresponding to the scheduling request from the resource pool and allocating the task processing channel to the corresponding job task may include: and selecting a task processing channel with the job processing capacity matched with the resource requirement of the scheduling request from the resource pool to distribute to the corresponding job task.
Therefore, resources can be distributed according to the resource requirements of the job task time as required, a task processing channel with high job capacity is distributed for the job task time when the resource requirements are high, and a corresponding task processing channel with low job capacity is distributed when the resource requirements are low, so that the waste of resources caused by the application of the job task occupying less resources is avoided.
Based on the task scheduling processing method, the application also provides a task scheduling processing device. Fig. 6 is a schematic block diagram of an embodiment of a task scheduling processing apparatus provided in the present application, and as shown in fig. 6, the apparatus may include:
a resource pool 101, which can be used for storing the resources of the deployed task processing channels;
the configuration center 102 may be configured to obtain channel resource information mapped by task processing channel information in the resource pool;
the resource query module 103 may be configured to, when receiving a scheduling request of at least one job task, query the configuration center whether there is an idle channel resource for processing the job task;
the scheduling processing module 104 may be configured to, when the query result of the configuration center indicates that the idle channel resource exists, select a task processing channel corresponding to the scheduling request from the resource pool according to the resource allocation priority set for the job task, and allocate the task processing channel to the corresponding job task.
The task scheduling processing device provided by the embodiment of the application can abstract a main scheduling server for processing job tasks and a corresponding execution server into channels in a distributed scene, and a plurality of channels can register channel information of the channels to a unified configuration center and map the channel information into channel resource information in the configuration center. When the job task of the job request end needs to use resources, the channel resource information can be obtained from the configuration center, and whether idle channel resources available for the job task exist at present is judged according to the resource information in the resource pool. If so, scheduling resources may be allocated for task job execution. In this way, in the process of task scheduling processing, when a scheduling request of a plurality of job tasks is received, each scheduling task can inquire whether available idle channel resources exist in a configuration center, and then allocate corresponding resources to the plurality of job tasks according to the set job task priority according to the inquiry result of the configuration center, so that the multi-channel parallel processing resource allocation for a plurality of concurrently submitted tasks is realized. In the queuing waiting process of the job tasks without preempting the resources, the configuration center feeds back idle channel resources after updating, and therefore the resources can be rapidly allocated to the job tasks in the queue. The task scheduling processing device can realize multi-channel resource allocation, can allocate task resources to jobs as required, solves the problem that follow-up tasks are not executable due to channel blockage, provides a resource utilization rate, and improves task scheduling processing efficiency.
As mentioned above, in one embodiment of the apparatus, the human processing channels in the resource pool 101 are enabled and register their own resource status with the configuration center. After the registration is successful, the configuration center 102 may sense the survival condition of the task processing channel through heartbeat. When the task processing channel resources in the resource pool are invalid or off-line, and the information of the applied occupation/release and the like can be synchronously updated to the configuration center. Thus, in an embodiment of the apparatus described herein, the configuration center 102 may be configured to include:
acquiring the survival condition of the resource pool task processing channel through heartbeat; and the number of the first and second groups,
and updating the task processing channel information in the resource pool in real time.
When the task processing channel fails (such as a lower line or a fault), the channel resource information of the configuration center is correspondingly updated. In the actual task scheduling processing operation process, the occupation and use conditions of the task processing channels in the resource pool are often dynamically changed in real time, so the configuration center can be set to sense the survival conditions of each task channel in the resource pool in real time and update the occupation conditions of the task processing channels in real time. Therefore, the channel resource information of the configuration center can accurately and reliably reflect the resource condition of the task processing channel in the current resource pool.
Fig. 7 is a schematic block diagram of an embodiment of a scheduling processing module 104 in the task scheduling processing apparatus provided in the present application. Specifically, as shown in fig. 7, the scheduling processing module 104 may include:
the resource judging module 1041 may be configured to judge, according to an available resource query result of the configuration center, whether the number of idle channel resources in the resource pool meets a requirement of the resource amount applied by the current multiple job tasks;
the priority setting module 1042 may be configured to, when the determination result of the resource determining module 1041 is negative, adopt any one of a time sequence of applying for resources by the job tasks among the plurality of job tasks and a priority of the weights of the job tasks as a resource allocation priority of the job tasks;
the channel resource allocation module 1043 may be configured to, when the resource query result of the configuration center indicates that the idle channel resource exists, select a task processing channel corresponding to the scheduling request from the resource pool according to the resource allocation priority set by the priority setting module 1042 to allocate to the corresponding job task.
In the specific implementation process, the resource allocation can be performed according to the sequence of the application resource of the job task. When the number of resources in the resource pool is less than the resource application amount of a plurality of tasks submitted concurrently, the job task which submits the resource application first can preferentially preempt idle task processing channel resources, and the job task which is submitted later fails in resource preemption. Of course, the resource job task is not preempted, and the scheduling can be returned again to continue to apply for the resource.
Fig. 8 is a schematic block structure diagram of another embodiment of a task scheduling processing apparatus provided in the present application, and as shown in fig. 8, the apparatus may further include:
the resource failure processing module 105 may be configured to update channel resource information corresponding to the configuration center when it is determined that a task processing channel in the resource pool is failed; and processing the job task called to the failed task processing channel according to the scheduling request in the following way:
interrupting the job task executing the failed task processing channel;
abandoning the job task of the queued task processing channel which is not executed;
and rescheduling the job task which is called to the failed task processing channel, and distributing a corresponding task processing channel for the rescheduling job task according to the channel resource information updated by the configuration center.
In a specific execution process, whether a master dispatch server (Driver) is alive or not can be sensed through heartbeat. When the master scheduling server is sensed to be invalid, the corresponding channel resource information of the task processing channel information registered in the configuration center is also invalid. When the channel resource is offline, the interruption of the allowed job task can be directly closed, such as the running logic can be set to be identified as failure. And the number of the workers who are queuing to call the failed or offline task processing channel can be abandoned. Furthermore, the job task called to the failed task processing channel can be rescheduled, for example, the interrupted or abandoned job task can be rescheduled back to the scheduling cluster after delaying the preset time, and the channel resource is reapplied for task processing. Therefore, the processing method provided by the embodiment can not cause the problem of job task execution loss when the resources are off-line in the application scene of distributed task processing, and meanwhile, if one task channel is abnormal in the execution process, the job tasks which are subsequently queued and use the abnormal task processing channel can be executed by selecting other channels.
Fig. 9 is a schematic block structure diagram of another embodiment of a task scheduling processing apparatus provided in this application, and as shown in fig. 9, the apparatus may further include:
the timeout processing module 106 may be configured to monitor an occupied execution time of a task processing channel of the job task, and stop the job task whose occupied time is greater than a set maximum execution time threshold when the occupied execution time is greater than the set maximum execution time threshold.
In an application scenario of distributed task processing, if the processing time of a job task is too long and exceeds a normal execution period, at this time, it can be considered that the job task is abnormal in the processing process, and the job task with the too long execution time exceeding a set time threshold should be processed correspondingly.
Fig. 10 is a schematic block structure diagram of another embodiment of a task scheduling processing apparatus provided in this application, and as shown in fig. 10, the apparatus may further include:
the task restarting module 107 may be configured to scan and obtain the job task marked as abnormal processing according to a preset period, and determine whether to continue to call the job task marked as abnormal processing according to the set scheduling processing configuration information.
Furthermore, a thread can be set for scanning the job tasks recorded as exception handling in the job tasks, and then determining whether to continue scheduling the exception handling job tasks according to the preset user configuration. If the scheduling processing configuration information of the user is set to be negative, the job task occupying the execution time larger than the maximum execution time threshold value can be killed, the job task is discarded, and corresponding log records are carried out. If the configuration information of the scheduling processing is set to be yes, the job tasks marked as abnormal processing can be rescheduled after a period of time delay according to the setting of the configuration information, channel resources can be reapplied, resources can be allocated, and the like. In the embodiment of the present application, the job task executing the logic running process is killed, and is converted to another channel for continuous processing, so as to free up time and channel resources for processing the subsequent job task. The implementation scheme of the embodiment can greatly improve the utilization rate of channel resources in the resource pool and improve the scheduling processing efficiency of the job task.
In another embodiment of the task scheduling processing apparatus provided by the present application, the channel resource information in the configuration center 102 may include information of job processing capability of a task processing channel;
correspondingly, the selecting, by the scheduling processing module 104, a task processing channel corresponding to the scheduling request from the resource pool to allocate to the corresponding job task includes: and selecting a task processing channel with the job processing capacity matched with the resource requirement of the scheduling request from the resource pool to distribute to the corresponding job task.
In this embodiment, the client of the job request may register an interested channel type with the configuration center, and obtain all task processing channel information of the channel type. When the job task really starts scheduling processing, the idle channel resource with the strongest current processing capability can be allocated for the job task to process. And updating the channel resource information to the registration center after the processing is finished. Therefore, resources can be distributed according to the resource requirements of the job task time as required, a task processing channel with high job capacity is distributed for the job task time when the resource requirements are high, and a corresponding task processing channel with low job capacity is distributed when the resource requirements are low, so that the waste of resources caused by the application of the job task occupying less resources is avoided.
The task scheduling processing method and the task scheduling processing device can be used in a service system for processing job tasks in a distributed service scene. The system can comprise one or more physical clusters, a resource information center and a scheduling processing module. Specifically, the application further provides a system for scheduling resources to process a plurality of job tasks in a distributed system application scenario. Specifically, in an embodiment of a distributed task scheduling processing system provided in the present application, the system may include:
the system comprises a resource pool comprising at least one physical cluster, wherein the physical cluster comprises a deployed task processing channel consisting of a main scheduling server and a corresponding execution server;
the configuration center may be configured to map a task processing channel in the resource pool to channel resource information, obtain a survival condition of the task processing channel in the resource pool through heartbeat, and update the task processing channel information in the resource pool in real time.
The scheduling cluster can be used for inquiring whether idle channel resources for processing the job task exist in the configuration center according to the received scheduling request of the job task; and the resource processing module is further configured to select a task processing channel corresponding to the scheduling request from the resource pool according to the resource allocation priority set for the job task to allocate to the corresponding job task when the query result of the configuration center indicates that the idle channel resource exists.
As described above, in another embodiment, the distributed task scheduling processing system may further include:
the resource failure processing unit may be configured to update channel resource information corresponding to the configuration center when it is determined that a task processing channel in the resource pool is failed; and processing the job task called to the failed task processing channel according to the scheduling request in the following way:
interrupting the job task executing the failed task processing channel;
abandoning the job task of the queued task processing channel which is not executed;
and rescheduling the job task which is called to the failed task processing channel, and distributing a corresponding task processing channel for the rescheduling job task according to the channel resource information updated by the configuration center.
In another embodiment, the distributed task scheduling processing system may further include:
the overtime processing module can be used for monitoring the occupied execution time of a task processing channel of the job task and stopping the job task of which the occupied time is greater than the set maximum execution time threshold when the occupied execution time is greater than the set maximum execution time threshold; the method can also be used for scanning and acquiring the job task marked as abnormal processing according to a preset period, and determining whether to continuously call the job task marked as abnormal processing according to the set scheduling processing configuration information.
Specifically, the description and the selection setting manner of the resource allocation priority, the physical cluster, the channel resource failure (offline), the job task execution timeout, and the like in the distributed task scheduling processing system provided in the foregoing embodiment may refer to the foregoing description, and are not described herein again.
The application provides a task scheduling processing method, a task scheduling processing device and a task scheduling processing system, which can solve the problem of channel blockage caused by single job deadlock by adopting a processing mode of configuring multiple channels, can provide parallel submission and parallel processing of multiple job tasks, quickly and efficiently allocate required resources to the job tasks, and greatly improve the task scheduling processing efficiency and the resource utilization rate.
Although the present application refers to the description of resource pool setting, channel resource mapping, Driver survival through heartbeat sensing, resource allocation, resource execution time monitoring by thread, job task restart, and other data setting, information interaction, information monitoring, and the like, the present application is not limited to the case where the data processing standard, the communication interaction identification protocol, or the embodiment is completely met. Some embodiments with slight modifications based on the description of design language, communication identification, protocol or embodiment can also achieve the same, equivalent or similar effect or the expected effect after modification. Of course, even if the data processing and determining modes of the industry standards and protocols are not adopted, the same application can still be realized as long as the resource configuration, information interaction and information determination feedback modes of the embodiments described above are met, and further description is omitted here.
Although the present application provides method steps as described in an embodiment or flowchart, more or fewer steps may be included based on conventional or non-inventive means. The order of steps recited in the embodiments is merely one manner of performing the steps in a multitude of orders and does not represent the only order of execution. When an actual apparatus or system product executes, it may execute sequentially or in parallel (e.g., in the context of parallel processors or multi-threaded processing) according to the embodiments or methods shown in the figures.
The units, devices or modules illustrated in the above embodiments may be implemented by a computer chip or an entity, or by a product with certain functions. For convenience of description, the above devices are described as being divided into various modules by functions, and are described separately. Of course, in implementing the present application, the functions of each module may be implemented in the same or multiple software and/or hardware, or the modules implementing the same functions may be implemented by a combination of multiple sub-modules or sub-units, for example, the memory may be divided into a template repository and a tag repository, and the template, the set structure tag, the set variable tag, and the like may be stored separately.
Those skilled in the art will also appreciate that, in addition to implementing the controller as pure computer readable program code, the same functionality can be implemented by logically programming method steps such that the controller is in the form of logic gates, switches, application specific integrated circuits, programmable logic controllers, embedded microcontrollers and the like. Such a controller may therefore be considered as a hardware component, and the means included therein for performing the various functions may also be considered as a structure within the hardware component. Or even means for performing the functions may be regarded as being both a software module for performing the method and a structure within a hardware component.
The application may be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, classes, etc. that perform particular tasks or implement particular abstract data types. The application may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media including memory storage devices.
From the above description of the embodiments, it is clear to those skilled in the art that the present application can be implemented by software plus necessary general hardware platform. Based on such understanding, the technical solutions of the present application may be embodied in the form of a software product, which may be stored in a storage medium, such as a ROM/RAM, a magnetic disk, an optical disk, or the like, and includes several instructions for enabling a computer device (which may be a personal computer, a mobile terminal, a server, or a network device) to execute the method according to the embodiments or some parts of the embodiments of the present application.
The embodiments in the present specification are described in a progressive manner, and the same or similar parts among the embodiments are referred to each other, and each embodiment focuses on the differences from the other embodiments. The application is operational with numerous general purpose or special purpose computing system environments or configurations. For example: personal computers, server computers, hand-held or portable devices, tablet-type devices, multiprocessor systems, microprocessor-based systems, set top boxes, programmable electronic devices, network PCs, minicomputers, mainframe computers, distributed computing environments that include any of the above systems or devices, and the like.
While the present application has been described with examples, those of ordinary skill in the art will appreciate that there are numerous variations and permutations of the present application without departing from the spirit of the application, and it is intended that the appended claims encompass such variations and permutations without departing from the spirit of the application.
Claims (17)
1. A task scheduling processing method is characterized by comprising the following steps:
mapping task processing channel information in the resource pool into channel resource information of a configuration center;
when a scheduling request of at least one job task is received, inquiring whether idle channel resources for processing the job task exist in the configuration center;
when the query result of the configuration center indicates that the idle channel resources exist, selecting a task processing channel corresponding to the scheduling request from the resource pool according to the resource allocation priority set for the job task and allocating the task processing channel to the corresponding job task;
the task processing channel consists of a main scheduling server and a plurality of execution servers correspondingly managed by the main scheduling server; the task processing channels in the resource pool are set to be resources with different sizes, the size of the resources of the task processing channels is determined at least based on the number of execution servers contained in the main scheduling server, or each task processing channel is set to be one main scheduling server corresponding to the same number of execution servers.
2. The method according to claim 1, wherein the setting of the resource allocation priority for the job task includes:
when the configuration center is inquired that the idle channel resources exist and the number of the idle channel resources is smaller than the amount of the application resources of the current multiple job tasks, any one of the time sequence of applying the resources by the job tasks in the multiple job tasks and the weight priority of the job tasks is used as the resource allocation priority of the job tasks.
3. A task scheduling processing method according to claim 1, wherein the configuration center is configured to include:
acquiring the survival condition of the resource pool task processing channel through heartbeat; and the number of the first and second groups,
and updating the task processing channel information in the resource pool in real time.
4. A method of task scheduling processing according to claim 3, wherein the method further comprises:
when the task processing channel in the resource pool is judged to be invalid, updating the channel resource information corresponding to the configuration center; and the number of the first and second groups,
and processing the job task called to the failed task processing channel according to the scheduling request in the following way:
interrupting the job task executing the failed task processing channel;
abandoning the job task of the queued task processing channel which is not executed;
and rescheduling the job task which is called to the failed task processing channel, and distributing a corresponding task processing channel for the rescheduling job task according to the channel resource information updated by the configuration center.
5. A method for task scheduling processing according to any of claims 1 to 4, wherein the method further comprises:
and monitoring the occupied execution time of a task processing channel of the job task, and stopping the job task of which the occupied time is greater than the set maximum execution time threshold when the occupied execution time is greater than the set maximum execution time threshold.
6. The method of claim 5, wherein the method further comprises:
and scanning and acquiring the job task marked as abnormal processing according to a preset period, and determining whether to continuously call the job task marked as abnormal processing according to the set scheduling processing configuration information.
7. The task scheduling processing method according to any one of claims 1 to 4, wherein the channel resource information of the configuration center includes information of job processing capability of a task processing channel;
correspondingly, the selecting a task processing channel corresponding to the scheduling request from the resource pool and allocating the task processing channel to the corresponding job task includes: and selecting a task processing channel with the job processing capacity matched with the resource requirement of the scheduling request from the resource pool to distribute to the corresponding job task.
8. A task scheduling processing apparatus, characterized in that the apparatus comprises:
the resource pool is used for storing resources of a deployed task processing channel, and the task processing channel consists of a main scheduling server and a plurality of execution servers correspondingly managed by the main scheduling server; the task processing channels in the resource pool are set to be resources with different sizes, the size of the resources of the task processing channels is determined at least based on the number of execution servers contained in the main scheduling server, or each task processing channel is set to be one main scheduling server corresponding to the same number of execution servers;
the configuration center is used for acquiring channel resource information mapped by the task processing channel information in the resource pool;
the resource query module is used for querying whether idle channel resources for processing the job task exist or not from the configuration center when a scheduling request of at least one job task is received;
and the scheduling processing module is used for selecting a task processing channel corresponding to the scheduling request from the resource pool to allocate to the corresponding job task according to the resource allocation priority set for the job task when the query result of the configuration center indicates that the idle channel resource exists.
9. A task scheduling processing apparatus according to claim 8, wherein said configuration is arranged to include:
acquiring the survival condition of the resource pool task processing channel through heartbeat; and the number of the first and second groups,
and updating the task processing channel information in the resource pool in real time.
10. The task scheduling processing apparatus according to claim 8, wherein the scheduling processing module includes:
the resource judging module is used for judging whether the number of idle channel resources in the resource pool meets the requirement of the resource amount applied by the current multiple job tasks according to the available resource query result of the configuration center;
the priority setting module is used for taking any one of the time sequence of applying for resources by the job tasks in the plurality of job tasks and the weight priority of the job tasks as the resource allocation priority of the job tasks when the judgment result of the resource judging module is negative;
and the channel resource allocation module is used for selecting a task processing channel corresponding to the scheduling request from the resource pool to allocate to the corresponding job task according to the resource allocation priority set by the priority setting module when the resource query result of the configuration center indicates that the idle channel resource exists.
11. A task schedule processing apparatus according to claim 9, characterized in that the apparatus further comprises:
the resource failure processing module is used for updating the channel resource information corresponding to the configuration center when the task processing channel in the resource pool is judged to be failed; and processing the job task called to the failed task processing channel according to the scheduling request in the following way:
interrupting the job task executing the failed task processing channel;
abandoning the job task of the queued task processing channel which is not executed;
and rescheduling the job task which is called to the failed task processing channel, and distributing a corresponding task processing channel for the rescheduling job task according to the channel resource information updated by the configuration center.
12. A task scheduling processing apparatus according to any one of claims 8 to 11, characterized in that the apparatus further comprises:
and the overtime processing module is used for monitoring the occupied execution time of the task processing channel of the job task and stopping the job task of which the occupied time is greater than the set maximum execution time threshold when the occupied execution time is greater than the set maximum execution time threshold.
13. A task schedule processing apparatus according to claim 12, characterized in that the apparatus further comprises:
and the task restarting module is used for scanning and acquiring the job task marked as abnormal processing according to a preset period and determining whether to continuously call the job task marked as abnormal processing according to the set scheduling processing configuration information.
14. A task scheduling processing apparatus according to any one of claims 8 to 11, wherein the channel resource information in the configuration center includes information of job processing capability of a task processing channel;
correspondingly, the step of selecting the task processing channel corresponding to the scheduling request from the resource pool by the scheduling processing module to allocate to the corresponding job task comprises: and selecting a task processing channel with the job processing capacity matched with the resource requirement of the scheduling request from the resource pool to distribute to the corresponding job task.
15. A distributed task scheduling processing system, the system comprising:
the system comprises a resource pool comprising at least one physical cluster, wherein the physical cluster comprises a deployed task processing channel consisting of a main scheduling server and a corresponding execution server, and the task processing channel consists of the main scheduling server and a plurality of execution servers correspondingly managed by the main scheduling server; the task processing channels in the resource pool are set to be resources with different sizes, the size of the resources of the task processing channels is determined at least based on the number of execution servers contained in the main scheduling server, or each task processing channel is set to be one main scheduling server corresponding to the same number of execution servers;
the configuration center is used for mapping the task processing channel in the resource pool into channel resource information, acquiring the survival condition of the task processing channel in the resource pool through heartbeat, and updating the task processing channel information in the resource pool in real time;
the scheduling cluster is used for inquiring whether idle channel resources for processing the job task exist in the configuration center according to the received scheduling request of the job task; and the resource processing module is further configured to select a task processing channel corresponding to the scheduling request from the resource pool according to the resource allocation priority set for the job task to allocate to the corresponding job task when the query result of the configuration center indicates that the idle channel resource exists.
16. A distributed task scheduling processing system according to claim 15 wherein said system further comprises:
the resource failure processing unit is used for updating the channel resource information corresponding to the configuration center when the task processing channel in the resource pool is judged to be failed; and processing the job task called to the failed task processing channel according to the scheduling request in the following way:
interrupting the job task executing the failed task processing channel;
abandoning the job task of the queued task processing channel which is not executed;
and rescheduling the job task which is called to the failed task processing channel, and distributing a corresponding task processing channel for the rescheduling job task according to the channel resource information updated by the configuration center.
17. A distributed task scheduling processing system according to claim 15 wherein said system further comprises:
the overtime processing module is used for monitoring the occupied execution time of a task processing channel of the job task and stopping the job task of which the occupied time is greater than the set maximum execution time threshold when the occupied execution time is greater than the set maximum execution time threshold; and the scheduling processing device is also used for scanning and acquiring the job task marked as abnormal processing according to a preset period, and determining whether to continuously call the job task marked as abnormal processing according to the set scheduling processing configuration information.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610197298.9A CN107291547B (en) | 2016-03-31 | 2016-03-31 | Task scheduling processing method, device and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610197298.9A CN107291547B (en) | 2016-03-31 | 2016-03-31 | Task scheduling processing method, device and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107291547A CN107291547A (en) | 2017-10-24 |
| CN107291547B true CN107291547B (en) | 2021-02-05 |
Family
ID=60087724
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610197298.9A Active CN107291547B (en) | 2016-03-31 | 2016-03-31 | Task scheduling processing method, device and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107291547B (en) |
Families Citing this family (25)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107908487B (en) * | 2017-11-08 | 2020-11-24 | 中国平安人寿保险股份有限公司 | Task control management method, device and equipment and computer readable storage medium |
| CN107944822A (en) * | 2017-11-22 | 2018-04-20 | 中国银行股份有限公司 | A kind of system and method for submitting operation |
| CN109992403B (en) * | 2017-12-30 | 2021-06-01 | 中国移动通信集团福建有限公司 | Optimization method and device for multi-tenant resource scheduling, terminal equipment and storage medium |
| CN108228355A (en) * | 2018-01-02 | 2018-06-29 | 武汉斗鱼网络科技有限公司 | Task processing method and device, method for scheduling task and device |
| CN110633133A (en) * | 2018-06-21 | 2019-12-31 | 中兴通讯股份有限公司 | Task processing method and device and computer readable storage medium |
| CN109060829A (en) * | 2018-08-31 | 2018-12-21 | 深圳步智造科技有限公司 | Adhesive tape for wiper blade defect inspection method, device, equipment and readable storage medium storing program for executing |
| CN109284190A (en) * | 2018-09-20 | 2019-01-29 | 中链科技有限公司 | A kind of task processing method and device |
| CN111026521B (en) * | 2018-10-10 | 2023-07-14 | 上海寒武纪信息科技有限公司 | Task scheduler, task processing system and task processing method |
| CN109445921A (en) * | 2018-10-24 | 2019-03-08 | 东北石油大学 | A kind of distributed data task processing method and device |
| CN109710391A (en) * | 2018-12-20 | 2019-05-03 | 西安四叶草信息技术有限公司 | Method for scheduling task and equipment |
| CN109821466B (en) * | 2019-01-09 | 2021-07-06 | 武汉智能装备工业技术研究院有限公司 | Multi-channel liquid batching full-automatic scheduling system and method |
| CN110097268B (en) * | 2019-04-19 | 2022-08-19 | 北京金山安全软件有限公司 | Task allocation method and device, electronic equipment and storage medium |
| CN110376451B (en) * | 2019-06-14 | 2021-11-05 | 南京冉亚电子技术有限公司 | Automatic distribution method and system for vector network analyzer channels |
| CN110376450B (en) * | 2019-06-14 | 2021-11-05 | 南京冉亚电子技术有限公司 | Multi-user multi-port expansion method and system based on vector network analyzer |
| CN110267110B (en) * | 2019-06-20 | 2021-09-10 | 视联动力信息技术股份有限公司 | Concurrent on-demand processing method and system based on video network |
| CN110287018B (en) * | 2019-07-04 | 2021-08-13 | 中国工商银行股份有限公司 | Batch task arranging method and device |
| CN110427259A (en) * | 2019-08-05 | 2019-11-08 | 北京奇艺世纪科技有限公司 | A kind of task processing method and device |
| CN111258555A (en) * | 2020-01-15 | 2020-06-09 | 上海知白智能科技有限公司 | Software implementation device |
| CN111614377B (en) * | 2020-04-24 | 2021-09-14 | 江苏方天电力技术有限公司 | Acquisition multi-task scheduling method and system based on HPLC concurrent channel |
| CN111666324B (en) * | 2020-05-18 | 2023-06-27 | 新浪技术（中国）有限公司 | ETL scheduling method and device between relational databases |
| CN111709723B (en) * | 2020-07-20 | 2023-11-28 | 港胜技术服务(深圳)有限公司 | RPA business process intelligent processing method, device, computer equipment and storage medium |
| CN112035232B (en) * | 2020-09-01 | 2023-08-18 | 中国银行股份有限公司 | Job operation priority determining method and related equipment |
| CN112148448B (en) * | 2020-09-22 | 2024-01-19 | 抖音视界有限公司 | Resource allocation method, apparatus, device and computer readable medium |
| CN112685151B (en) * | 2020-12-24 | 2022-02-01 | 四川长虹电器股份有限公司 | Resource scheduling system based on optimized weight model |
| CN115994023B (en) * | 2023-03-23 | 2023-06-06 | 鹏城实验室 | Visual intelligent system resource information scheduling method, device, terminal and medium |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101719079A (en) * | 2009-12-17 | 2010-06-02 | 中国电力科学研究院 | Method and device for processing tasks |
| CN102521044A (en) * | 2011-12-30 | 2012-06-27 | 北京拓明科技有限公司 | Distributed task scheduling method and system based on messaging middleware |
| CN102567086A (en) * | 2010-12-30 | 2012-07-11 | 中国移动通信集团公司 | Task scheduling method, equipment and system |
| CN103324539A (en) * | 2013-06-24 | 2013-09-25 | 浪潮电子信息产业股份有限公司 | Job scheduling management system and method |
| CN103530182A (en) * | 2013-10-22 | 2014-01-22 | 海南大学 | Working scheduling method and device |
| CN103699445A (en) * | 2013-12-19 | 2014-04-02 | 北京奇艺世纪科技有限公司 | Task scheduling method, device and system |
| CN104427623A (en) * | 2013-08-30 | 2015-03-18 | 中国移动通信集团设计院有限公司 | Service scheduling method and service scheduling device |
| CN104657214A (en) * | 2015-03-13 | 2015-05-27 | 华存数据信息技术有限公司 | Multi-queue multi-priority big data task management system and method for achieving big data task management by utilizing system |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7356820B2 (en) * | 2001-07-02 | 2008-04-08 | International Business Machines Corporation | Method of launching low-priority tasks |
| US9946563B2 (en) * | 2012-02-21 | 2018-04-17 | Disney Enterprises, Inc. | Batch scheduler management of virtual machines |
-
2016
- 2016-03-31 CN CN201610197298.9A patent/CN107291547B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101719079A (en) * | 2009-12-17 | 2010-06-02 | 中国电力科学研究院 | Method and device for processing tasks |
| CN102567086A (en) * | 2010-12-30 | 2012-07-11 | 中国移动通信集团公司 | Task scheduling method, equipment and system |
| CN102521044A (en) * | 2011-12-30 | 2012-06-27 | 北京拓明科技有限公司 | Distributed task scheduling method and system based on messaging middleware |
| CN103324539A (en) * | 2013-06-24 | 2013-09-25 | 浪潮电子信息产业股份有限公司 | Job scheduling management system and method |
| CN104427623A (en) * | 2013-08-30 | 2015-03-18 | 中国移动通信集团设计院有限公司 | Service scheduling method and service scheduling device |
| CN103530182A (en) * | 2013-10-22 | 2014-01-22 | 海南大学 | Working scheduling method and device |
| CN103699445A (en) * | 2013-12-19 | 2014-04-02 | 北京奇艺世纪科技有限公司 | Task scheduling method, device and system |
| CN104657214A (en) * | 2015-03-13 | 2015-05-27 | 华存数据信息技术有限公司 | Multi-queue multi-priority big data task management system and method for achieving big data task management by utilizing system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107291547A (en) | 2017-10-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107291547B (en) | Task scheduling processing method, device and system | |
| CN112162865B (en) | Scheduling method and device of server and server | |
| EP3567829B1 (en) | Resource management method and apparatus | |
| US10754706B1 (en) | Task scheduling for multiprocessor systems | |
| CA2849565C (en) | Method, apparatus, and system for scheduling processor core in multiprocessor core system | |
| US20130290974A1 (en) | Workflow control of reservations and regular jobs using a flexible job scheduler | |
| CN109564528B (en) | System and method for computing resource allocation in distributed computing | |
| US20140181818A1 (en) | Optimization of packet processing by delaying a processor from entering an idle state | |
| EP3675434B1 (en) | Distributed system resource allocation method, device and system | |
| CN110888743A (en) | GPU resource using method, device and storage medium | |
| US9535749B2 (en) | Methods for managing work load bursts and devices thereof | |
| CN111897637B (en) | Job scheduling method, device, host and storage medium | |
| US20100042723A1 (en) | Method and system for managing load in a network | |
| US10606650B2 (en) | Methods and nodes for scheduling data processing | |
| US10592107B2 (en) | Virtual machine storage management queue | |
| US9367349B2 (en) | Multi-core system and scheduling method | |
| EP4361808A1 (en) | Resource scheduling method and device and computing node | |
| CN106412088B (en) | A kind of method of data synchronization and terminal | |
| CN114327894A (en) | Resource allocation method, device, electronic equipment and storage medium | |
| US10523746B2 (en) | Coexistence of a synchronous architecture and an asynchronous architecture in a server | |
| CN109964206B (en) | Device and method for processing tasks | |
| KR20150089665A (en) | Appratus for workflow job scheduling | |
| CN114138626A (en) | Horizontally scalable distributed system for automated firmware testing and method thereof | |
| CN113254143A (en) | Virtual network function network element arranging and scheduling method, device and system | |
| CN115811549B (en) | Cloud edge resource management scheduling method and system supporting hybrid heterogeneous operation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20200924 Address after: Cayman Enterprise Centre, 27 Hospital Road, George Town, Grand Cayman Islands Applicant after: Innovative advanced technology Co.,Ltd. Address before: Cayman Enterprise Centre, 27 Hospital Road, George Town, Grand Cayman Islands Applicant before: Advanced innovation technology Co.,Ltd. Effective date of registration: 20200924 Address after: Cayman Enterprise Centre, 27 Hospital Road, George Town, Grand Cayman Islands Applicant after: Advanced innovation technology Co.,Ltd. Address before: Greater Cayman, British Cayman Islands Applicant before: Alibaba Group Holding Ltd. |
|
| TA01 | Transfer of patent application right | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |