CN115794322A - Task scheduling method for large mobile equipment mobile data center - Google Patents
Task scheduling method for large mobile equipment mobile data center Download PDFInfo
- Publication number
- CN115794322A CN115794322A CN202111062549.XA CN202111062549A CN115794322A CN 115794322 A CN115794322 A CN 115794322A CN 202111062549 A CN202111062549 A CN 202111062549A CN 115794322 A CN115794322 A CN 115794322A
- Authority
- CN
- China
- Prior art keywords
- task
- server
- data
- time
- tasks
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Multi Processors (AREA)
Abstract
The invention discloses a task scheduling method for a large mobile equipment mobile data center. The method comprises the following steps: (1) task receiving: collecting all task information of a mobile data center, data storage and current to-be-scheduled; (2) and (3) triggering type task scheduling: distributing the tasks to the servers meeting the time delay constraint for execution according to the factors such as task submission time, data transmission time, task scheduling time, task time delay constraint and the like; (3) interactive task scheduling: according to the task submission sequence, each task is distributed to a plurality of servers for storing data to be processed to be executed in a distributed mode, and the task processing time is reduced; (4) task migration: if the trigger type task and the interactive type task are simultaneously distributed on the server, the trigger type task is preferentially executed, and whether the interactive type task is migrated to other servers to be executed or not is judged according to the waiting time and the migration time. The invention can obviously improve the task execution efficiency of the mobile data center of the large-scale mobile equipment and reduce the task processing time delay.
Description
Technical Field
The invention relates to a task scheduling method for constructing a mobile data center on large-scale mobile equipment, belonging to the field of mobile equipment data processing.
Background
With the development of technologies such as the internet of things, big data, artificial intelligence and the like, the intelligentization level of large-scale mobile equipment such as airplanes and ships becomes higher and higher, and as if the large-scale mobile equipment is a mobile intelligent fort, a large number of high-precision sensors and a high-performance data processing technology are supported behind the high intelligentization level of the large-scale mobile equipment. Task scheduling is a key technology of high-performance data processing, and tasks are allocated to data processing resources according to a specific sequence and executed according to the task characteristics of the data processing and the distribution condition of the data processing resources. Because the traditional mobile equipment has less data acquisition and processing, a centralized data processing mode is generally adopted, namely all data tasks are uniformly processed by one server, and the server sequentially processes each task according to a certain sequence. With the increase of the number of sensors, the frequent and large-scale data acquisition, the increase of the number and complexity of data processing tasks, and the constraint of real-time performance and reliability of task processing, a single server cannot meet the requirement of data processing, so that currently, a large-scale mobile device is usually provided with a plurality of servers at different positions of the device, and the servers are organized into a mobile data center, and the efficiency and reliability of task processing are improved by using a distributed technology. However, the mobile data center has a great difference from the conventional data center in terms of physical architecture and application requirements, including that the number of servers is small, the servers are directly connected to each other through a high-speed data bus, the types of tasks are few, most of the tasks are monitoring, early warning or statistical analysis of perception data, the data reliability requirement is high, the recovery cannot be performed once the data are lost, the real-time performance of task processing is high, and a serious result is caused if the response is not timely.
Task scheduling of a traditional mobile device is essentially that each task competes for the use right of a unique server, the priority of the tasks is usually calculated according to certain constraints of the tasks, such as response time, resource requirements and the like, then the tasks are sequenced according to the priority, and the task with the high priority is executed first and is not suitable for task scheduling of multiple servers. Currently, research aiming at a mobile data center on a large-scale mobile device is in a starting stage, for example, a distributed comprehensive modular avionics system of an airplane manages each control device on the airplane by using a distributed idea, but the current research mainly focuses on the aspects of network structure design, hardware resource deployment, safety verification and the like, and has less skill on a task scheduling method. The task scheduling problem of the distributed system has been proved to be an NP difficult problem, and an optimal scheduling algorithm with polynomial complexity cannot be found, so that the existing optimization model or heuristic strategy is mostly adopted to find a better scheduling scheme. Task scheduling methods of conventional data centers have been widely researched and generate many scheduling algorithms with high precision, but these methods cannot be applied to task scheduling of mobile data centers of mobile devices because of the following reasons: firstly, the method focusing on the optimal solution of the scheduling scheme mostly adopts engineering algorithms such as genetic algorithm and the like or deep learning algorithms such as neural network and the like, but the method needs the support of large-scale high-performance computing resources, and a mobile data center is usually only provided with a plurality of servers, which may cause the scheduling time of tasks to exceed the task execution time; secondly, the scheduling method focusing on resource utilization rate is generally oriented to various tasks such as computation intensive tasks, data intensive tasks, network intensive tasks and the like, most of the tasks of the mobile data center are data intensive tasks, and a large number of tasks are concentrated on a certain server to influence task response time; thirdly, the scheduling method focusing on the performance of the algorithm generally sets a greedy allocation strategy for each task for specific applications, greedy is generally carried out on the basis of local information, the distributed tasks do not perform preemption and migration, the mobile data center can conveniently acquire global information and task migration, and the tasks with high real-time performance must support preemption and execution.
In summary, the task scheduling method for the mobile data center of the mobile device needs to comprehensively consider the network structure characteristics of the mobile data center and the data processing task characteristics of the mobile device, such as the number and the computing capacity of servers, the connection mode of the servers, the task type, the task response time, the task timeout influence, and other factors. The efficient scheduling algorithm can rapidly distribute each task to a proper server to be executed, can ensure that each task is completed within a limited time, and provides technical support for intelligent control of the mobile equipment.
Disclosure of Invention
The invention provides a task scheduling method facing a large mobile equipment mobile data center, aiming at scheduling the most common trigger type and interactive type tasks of mobile equipment according to the structural characteristics of the mobile data center and the task characteristics of the mobile equipment, comprehensively considering the factors of data storage distribution, the emergency degree of the tasks, the transmission capability of a high-speed bus, the computing power of a server, the migration cost of the tasks and the like to sequence, distribute, preempt and migrate the tasks, and improving the performance and the precision of task scheduling of the mobile data center.
The technical scheme adopted by the invention is as follows:
a task scheduling method for a large mobile equipment mobile data center specifically comprises the following steps:
(1) Task receiving: collecting relevant information of a mobile data center, data storage distribution conditions, relevant information of all trigger type and interactive type tasks which need to be scheduled currently by mobile equipment and the like;
(2) And (3) triggering type task scheduling: the scheduling of the trigger type task is carried out on the premise of guaranteeing the task response time, and the task is distributed according to the factors such as task submission time, data transmission time, task scheduling time, task delay constraint and the like, so that the trigger type task can be completed within the delay constraint;
(3) Interactive task scheduling: scheduling is sequentially carried out according to the task submission time, and according to the distribution condition of the data set to be processed of each task, task processing is distributed to a plurality of servers for storing the data to be processed by using a distributed idea and processed at the same time, so that the processing time of interactive tasks is reduced;
(4) Task migration: in the running process of the mobile equipment, if a trigger type task with stricter time delay constraint is received in the execution process of the interactive type task, the trigger type task preempts the processing resource of the server, and the subsequent execution mode of the original interactive type task is judged by comprehensively considering the task waiting time and the task migration time.
The invention has the beneficial effects that:
(1) The structural characteristics of the mobile data center and the task characteristics of the mobile equipment are comprehensively considered, the scheduling method is in accordance with the application requirements, and the high efficiency and the accuracy of task scheduling of the mobile data center are guaranteed.
(2) Two tasks which are most frequently used by the mobile equipment are extracted as scheduling objects, the characteristics of the tasks are fully considered in the sequencing, distributing, seizing and transferring stages of the scheduling process, and the practicability of the task scheduling method is ensured.
(3) The steps in the task scheduling process are simple to operate, the scheduling algorithm is low in calculation complexity, and the method is suitable for the characteristics of small quantity of servers and insufficient calculation resources in the mobile data center, and ensures the high efficiency of task scheduling.
Drawings
Fig. 1 is a network architecture of the task scheduling method for a mobile data center of a large mobile device according to the present invention.
Fig. 2 is a specific flow of the task scheduling method for a mobile data center of a large mobile device according to the present invention.
Fig. 3 is a resource allocation prediction process according to an embodiment of the present invention.
Detailed Description
The technical solution of the present invention is further illustrated by the following examples.
The task scheduling method provided by the invention is mainly used for a mobile data center constructed on large-scale mobile equipment such as an airplane, a ship and the like, and is used for scheduling execution of various data processing tasks on the mobile equipment on each server of the mobile data center, so that each task can be rapidly scheduled to be executed on a proper server and can be processed and completed within a limited time. The intelligent control of the mobile device is usually based on a large number of data processing tasks, and mainly includes two types, namely a trigger type task triggered by meeting a preset condition and an interactive type task required by normal analysis control, generally speaking, the trigger type task is often various abnormal or early warning tasks, the requirement on the response time of task processing is high, and the requirement on the response time of the interactive type task is not strict. The scheduling method needs to consider not only the structural characteristics of the mobile data center, but also the characteristics of different tasks.
Referring to fig. 1, each server is responsible for storing and processing data acquired by a sensor in a physical area, the servers are directly connected with each other in pairs through a high-speed data bus, and the sensor in each physical area is uniquely connected to the server responsible for the area through a dedicated bus.
At present, task scheduling methods for a large mobile equipment mobile data center mainly comprise two types, namely, a centralized scheduling method is used for sequentially executing all tasks on a unique server after sequencing, and the problem of distribution of the tasks to the server is not involved; and secondly, a scheduling method of a traditional data center is used, but the characteristics of small quantity of servers, insufficient computing resources and mobile equipment tasks of the mobile data center are not considered. The scheduling flow of the proposed task scheduling method for the large mobile device mobile data center is shown in fig. 2. The invention starts from the task characteristics of the mobile equipment and the structural characteristics of the mobile data center, comprehensively considers the data storage distribution, the server scale and processing capacity constraint, the data requirement and response time constraint of the data processing task, and improves the high efficiency of the task scheduling process and the accuracy of the scheduling result. The specific task scheduling process comprises the following four steps:
1. task receiving: collecting relevant information of a mobile data center, data storage distribution conditions and information of all tasks which need to be scheduled currently by mobile equipment, wherein the specific information comprises:
(1) m servers P = { P) of mobile data center 1 ,P 2 ,…,P m Receiving various tasks needing to be processed currently, including a trigger type task set
And interactive task collections
Wherein NT and NI are the number of the trigger type task and the interactive type task respectively, and m represents the number of the mobile data center server;
(2) collecting trigger type task information: information relating to each trigger-type task
Wherein i is not less than 1 and not more than NT, at i Hexix- i Respectively representing tasks
The submission time and the computational complexity factor, w, of the unit data quantity i Representing tasks
Amount of data to be processed, w i Is stored in the server P j Amount of data on is w i,j I.e. w i ＝(w i,1 ,w i,2 ,L,w i,m )，1≤j≤m；
(3) Collecting interactive task information: information related to each interactive task
Wherein at i Representing tasks
Time of arrival, χ i Representing tasks
Complex coefficient of calculation of a single data quantity, X i ＝(x j,k ) s×m Representing tasks
Distribution of data sets to be processed on servers, tasks
Processed large-scale data set D i Divided into s slices { D i,1 ,D i,2 ,…,D i,s Store on different servers if j data slice D i,j Stored in the server P k Upper rule x j,k =1, otherwise x j,k ＝0，1≤i≤NI，1≤k≤m，1≤j≤s。
2. And (3) triggering type task scheduling: the trigger task scheduling is based on the premise of guaranteeing task response time, and the specific process is as follows:
(1) calculating data transmission time: calculate each task
At P = { P 1 ,P 2 ,…,P m Each server P of j On-executing tasksPending data transfer to P j Data transmission time of
Data transmission time set of task executing on all servers
Wherein i is more than or equal to 1 and less than or equal to NT, j is more than or equal to 1 and less than or equal to m, d k,j Presentation Server P k And P j If k = j indicates the data storage of the server itself, i.e. d j,j =0, τ denotes the time taken for a unit distance to be transmitted per unit amount of data;
(2) and (3) task sequencing: calculate each task
Minimum transfer time performed on all servers
All triggered tasks
Sequencing according to the descending order of the minimum transmission time, and sequentially scheduling each task according to the descending order of the data transmission time;
(3) computing resource requirements: for each task
Task processing remaining time
Total calculation of tasks
Minimum processing resource requirement for a server
Wherein at i Representing tasks
Trigger time of (Tthreshold) T Representing the maximum allowable delay of the trigger type task, and ct is the current time;
(4) and (3) task allocation: for each task
The server corresponding to the minimum data transmission time is
Judgment server P k ActiveResource (P) which is a data processing resource available on the Internet k ) Whether to satisfy the task
If minimum processing resource requirements are met
Then the task will be
Distribution to servers
Executing; if it is
Repeating the steps (1) to (4) to perform the task
Set of servers P' = P- { P k }＝{P 1 ,P 2 ,…,P k-1 ,P k+1 ,…,P m Task allocation is carried out;
(5) task caching: if all the current servers cannot meet the task
Will be the task if the processing resource requirements of
And adding the task into a task buffer queue, and scheduling the task together with the trigger type task submitted by the user at the next moment.
3. Interactive task scheduling: the interactive tasks are sequentially scheduled according to the task submission time, and each analysis type task is scheduled
According to the distribution X of the data sets to be processed i ＝(x j,k ) s×m To be tasked with
The scheduling process is distributed to the proper server for execution, and the scheduling process comprises the following steps:
(1) selecting a candidate server: each server P j Task of upper storage
The set of data slices to be processed is DC i,j Number of data fragments to be processed
All DC i,j Servers not equal to phi are all tasks
Candidate servers that can be dispatched, the set of all candidate servers being PC = { P = { (P) j |DC i,j J is not equal to phi and is not less than 1 and not more than m, wherein s is a task
The number of the data to be processed is divided into pieces, j is more than or equal to 1 and less than or equal to m, and i is more than or equal to 1 and less than or equal to NI;
(2) and (3) task allocation: selecting the server P with the least data fragments to be processed from the candidate server set PC k E.g. PC for executing task
Is responsible for its storageTo-be-processed data slicing set DC i,k Wherein k is more than or equal to 1 and less than or equal to m;
(3) and (3) candidate server updating: updating each candidate server P j To-be-processed data fragment DC stored in PC i,j ＝DC i,j -DC i,k If DC i,j = Φ, then remove it from the candidate server set, where P k For the task currently allocated to execute
J is more than or equal to 1 and less than or equal to m;
(4) and (4) terminating the distribution: circularly executing the steps (2) and (3) until the task
The data fragments to be processed are each processed by an assigned server, i.e.
And the data shards handled by each server are different, i.e.
Wherein j is more than or equal to 1, k is more than or equal to m, and j is not equal to k;
(5) calculating task processing time: task
Time of treatment of
Wherein the task processing time of each server
Wherein
Represents P j The task actually processed
Is the number of data slices, upsilon isUnit data amount per data slice, resource j Is P j The currently available data processing resources χ i Representing tasks
And j is more than or equal to 1 and less than or equal to m.
4. Task migration: when the server P j In executing interactive tasks
Receive the triggered task
When, interactive tasks need to be decided
Whether the task needs to be migrated to another server for execution or is continuously executed after waiting for the trigger-type task to be completed on the current server, the specific process is as follows:
(1) and task waiting judgment: if interactive task
At the server P j The expected processing time of (A) is
Interactive task
Is treated for a time of
Triggered task
At the server P j Is executed at an execution time of
When in use
Without the need to interwork with the task
Migration to other servers P j Performing the step (b), wherein i is more than or equal to 1 and less than or equal to NI, j is more than or equal to 1 and less than or equal to m, and h is more than or equal to 1 and less than or equal to NT;
(2) and (3) task migration judgment: if the server set which does not execute the interaction task in the mobile data center is PM, the current server P is used j Last incomplete interactive task
And unprocessed data
Migration to any server P in the PM with sufficient data processing resources k Go on to execute, unprocessed data
Slave server P j Migration to P k Data transmission time of
When in use
When the task is migrated, when
Time-waiting triggered task
Continuing to execute the original interactive task after the execution is finished
Wherein d is j,k Presentation Server P j To P k The distance of transmission of (a) is,
represents the migrated data volume, tau is the time required for transmitting a unit data volume on a high-speed bus of unit length between servers,
for triggered type tasks
At the server P j The execution time of.
Examples
Referring to fig. 3, the resource allocation prediction process in this embodiment specifically includes the following steps: if the mobile data center of a certain mobile device is composed of 3 servers P 1 ,P 2 ,P 3 The task scheduling method comprises the following steps of:
(1) if the interactive task J1 is submitted at the time t1, the processed data set comprises four data fragments { D1, D2, D3, D4}, wherein the server P1 stores the { D1, D3}, the P2 stores the { D1, D2}, and the P3 stores the { D2, D4};
(2) the scheduling process of the interactive task J1 is as follows: firstly, calculating current candidate servers PC = { P1, P2 and P3}, wherein each server stores two to-be-processed data fragments { D1, D3}, { D1, D2}, and { D2 and D4}, and randomly selecting the server P1 to process the data fragments D1 and D3; the to-be-processed data fragments of the servers P1, P2 and P3 are updated to phi, { D2} and { D2, D4}, the candidate server set is updated to { P2, P3}, and the server P2 with the least to-be-processed data fragments is selected to process the data fragments D2; the to-be-processed data fragments of the servers P1, P2 and P3 are updated to phi, phi and { D4}, the candidate server set is updated to { P3}, the server P3 is selected to process the data fragment D4, and at the moment, the task J1 is scheduled to be completed;
(3) if the processing time of each data fragment is 2s, the services P1, P2 and P3 respectively need 4s, 2s and 2s to complete respective data processing, and the completion time of the task J1 is 4s;
(4) if the interactive task J1 submits the trigger task J2 at the time t2 after 1s, the data to be processed are w1 on P1, w2 on P2 and w3 on P3 respectively, and if the task J2 is executed on the servers P1, P2 and P3, 0.5s, 0.2s and 0.4s are needed for data transmission respectively;
(5) if the triggered task J2 is immediately scheduled when submitted, the maximum allowable time delay is 1s, and if the task J2 executes the maximum tasks on the servers P1, P2 and P3, the processing time is 0.5s, 0.8s and 0.6s respectively; firstly, a task J2 is tried to be distributed to a server P2 for execution, and if the available resources of the server P2 can meet the requirement of the task J2, a trigger type task J2 is distributed to the server P2 for execution;
(6) at the moment t2, the triggered task J2 can be distributed to the server P2 to be executed, after 0.2s, all data required to be processed by the task J2 are transmitted to the server P2, and at the moment, the interactive task J1 is executed on the server P2; at this time, the server P2 suspends the execution of the interactive task J1 and then executes the triggered task J2;
(7) the task J1 is suspended after being executed for 1.2s on the server P2, the time that the triggered task J2 needs to occupy the server P2 is 0.8s, if the server P2 continues to execute the task J1 after the task J2 is executed, the data processing time of the server P2 to the task J1 is 2.8s and is less than the processing time 4s of the task J1, therefore, the processing to the task J1 on the server P2 is not migrated, and the processing of the data fragment { D2} of the task J1 is continued on the server P2 after the triggered task J2 is executed;
(8) finally, the interactive task J1 is executed after being submitted for 4s, and the execution process is completed by the servers P1, P2 and P3 together; the triggered task J2 completes data transmission from P1 and P3 to P2 within 0.2s after submission, and then completes execution on the server P2 within 0.8 s; since the data processing and the data transmission use different server resources, the services P1 and P3 complete the data processing of the task J1 and the data transmission of the task J2 at the same time within 0.2s after the task J2 is submitted.
The present embodiments are described above with reference to the accompanying drawings, in which like reference numerals are used to designate like elements, and in which like reference numerals are used to designate like elements.
Unless otherwise defined, terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art; further, it will be understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and will not be interpreted in an idealized or overly formal sense.
Claims (5)
1. A task scheduling method for a large mobile equipment mobile data center is characterized by comprising the following steps:
(1) Task receiving: collecting relevant information of a mobile data center, data storage distribution conditions and relevant information of all trigger type and interactive type tasks which need to be scheduled currently by mobile equipment;
(2) And (3) triggering type task scheduling: the scheduling of the trigger type task is carried out on the premise of guaranteeing the task response time, and the task is distributed according to the task submission time, the data transmission time, the task scheduling time and the task time delay constraint, so that the trigger type task can be completed within the time delay constraint;
(3) Interactive task scheduling: scheduling is sequentially carried out according to the task submission time, and according to the distribution condition of the data set to be processed of each task, task processing is distributed to a plurality of servers for storing the data to be processed by using a distributed idea and processed at the same time, so that the processing time of interactive tasks is reduced;
(4) Task migration: in the running process of the mobile equipment, if a trigger type task with stricter time delay constraint is received in the execution process of the interactive type task, the trigger type task preempts the processing resource of the server, and the subsequent execution mode of the original interactive type task is judged by comprehensively considering the task waiting time and the task migration time.
2. The large mobile equipment mobile data center-oriented task scheduling method according to claim 1, wherein the specific steps of the step (2) are as follows:
(1) calculating data transmission time: calculating the transmission time of the data to be processed required by each trigger type task to be executed on each server according to the structure of the mobile data center and the source of the data set to be processed of each task;
(2) and (3) task sequencing: counting the minimum transmission time required by each task to be executed on each server, sequencing all the trigger tasks according to the descending order of the minimum transmission time, and preferentially scheduling the task with the maximum data transmission;
(3) computing resource requirements: calculating the remaining time of each task, the total calculated amount of the tasks and the lowest processing resource requirement on the server;
(4) and (3) task allocation: pre-distributing each task to a server with the minimum data transmission time, judging whether available resources on the server can meet the minimum processing resource requirement of the task, if so, distributing the task to the server for execution, otherwise, repeatedly executing the steps (1) to (4) of the step (2), and distributing the task to other servers which can finish processing in the shortest time for execution;
(5) task caching: and if all the current servers cannot meet the processing resource requirements of the tasks, adding the tasks into a task cache queue, and scheduling the tasks together with the triggered tasks submitted by the users at the next moment.
3. The method for scheduling tasks for the large mobile equipment mobile data center according to claim 1, wherein the specific steps of the step (3) are as follows:
(1) selecting a candidate server: fragmenting to-be-processed data of each interactive task, wherein all servers storing to-be-processed data fragments are candidate servers, and all data fragments to be processed of the current task stored in each candidate server form a to-be-processed data fragment set of the server;
(2) and (3) task allocation: allocating a server with the least number of to-be-processed data fragments in the candidate servers to execute partial data processing operation of the current task, wherein the data processing operation executed by the server mainly aims at a stored to-be-processed data fragment set of the current task;
(3) and (3) candidate server updating: removing the currently allocated and executed data fragments from the to-be-processed data fragment set of each candidate server, and removing the to-be-processed data fragment set from the candidate servers to be an empty server;
(4) and (4) terminating the distribution: circularly executing the steps (2) and (3) of the step (3) until all the data fragments to be processed of the current interactive task are distributed to the candidate server for processing;
(5) calculating task processing time: and calculating the processing time of each selected candidate server for processing the data fragment set to be processed, wherein the processing time of the current task is the maximum value of the processing times of all the selected candidate servers.
4. The method for scheduling tasks for the large mobile equipment mobile data center according to claim 1, wherein the specific steps of the step (4) are as follows:
(1) and task waiting judgment: if the sum of the processing time of the trigger type task of the current preempting server and the expected processing time of the preempted interactive type task on the server does not exceed the processing time of the preempted interactive type task, the preempted interactive type task is continuously executed on the original server after the trigger type task of the preempting server is executed;
(2) and (3) task migration judgment: and if the data transmission time of the residual unprocessed data fragments migrated to other servers by the current preempted interactive task is less than the execution time of the trigger type task of the preempted server, migrating the unprocessed part of the preempted interactive task to other servers for execution.
5. The method for scheduling the tasks for the large mobile equipment mobile data center according to claim 1, wherein the specific task scheduling process comprises the following four steps:
(1) And task receiving: collecting relevant information of a mobile data center, data storage distribution conditions and information of all tasks which need to be scheduled currently by mobile equipment, wherein the specific information comprises:
(1) m servers P = { P) of mobile data center 1 ,P 2 ,…,P m Receiving various tasks needing to be processed currently, including a trigger type task set
And interactive task collections
Wherein NT and NI are the number of the trigger type task and the interactive type task respectively, and m represents the number of the mobile data center server;
(2) collecting trigger type task information: information relating to each trigger-type task
Wherein i is more than or equal to 1 and less than or equal to NT, at i Hexix- i Respectively representing tasks
The submission time and the computational complexity factor, w, of the unit data quantity i Representing tasks
Amount of data to be processed, w i Is stored in the server P j The amount of data on is w i,j I.e. w i ＝(w i,1 ,w i,2 ,L,w i,m )，1≤j≤m；
(3) Collecting interactive task information: information related to each interactive task
Wherein at i Representing tasks
Time of arrival, χ i Representing tasks
Complex coefficient of calculation of a single data quantity, X i ＝(x j,k ) s×m Representing tasks
Distribution of data sets to be processed on servers, tasks
Processed large-scale data set D i Divided into s slices { D i,1 ,D i,2 ,…,D i,s Store on different servers if j data slice D i,j Stored in the server P k Upper rule x j,k =1, otherwise x j,k ＝0，1≤i≤NI，1≤k≤m，1≤j≤s；
(2) And triggering type task scheduling: the trigger task scheduling is based on the premise of guaranteeing task response time, and the specific process is as follows:
(1) calculating data transmission time: calculate each task
At P = { P 1 ,P 2 ,…,P m Each server P of j Transmitting data to P to be processed by task in up-execution j Data transmission time of
Data transmission time set of task executing on all servers
Wherein i is more than or equal to 1 and less than or equal to NT, j is more than or equal to 1 and less than or equal to m, d k,j Presentation Server P k And P j If k = j indicates the serverOwn data storage, i.e. d j,j =0, τ denotes the time taken for a unit distance to be transmitted per unit amount of data;
(2) task sequencing: calculate each task
Minimum transfer time performed on all servers
All triggered tasks
Sequencing according to the descending order of the minimum transmission time, and sequentially scheduling each task according to the descending order of the data transmission time;
(3) computing resource requirements: for each task
Task processing remaining time
Total calculation of tasks
Minimum processing resource requirement for a server
Wherein at i Representing tasks
Trigger time of (Tthreshold) T Representing the maximum allowable delay of the trigger type task, and ct is the current time;
(4) and (3) task allocation: for each task
Minimum sizeThe data transmission time of the server is
Judgment server P k ActiveResource (P) which is a data processing resource available on the Internet k ) Whether to satisfy the task
If minimum processing resource requirements are met
Then the task will be executed
Distribution to servers
Executing; if it is
Repeating the steps (1) to (4) to perform the task
Set of servers P' = P- { P k }＝{P 1 ,P 2 ,…,P k-1 ,P k+1 ,…,P m Task allocation is carried out;
(5) task caching: if all the current servers cannot meet the task
Will be the task if the processing resource requirements of
Adding the task buffer queue and scheduling the task buffer queue and the trigger type task submitted by the user at the next moment;
(3) And interactive task scheduling: interactive tasks by task submission timeScheduling in turn for each analytic task
According to the distribution X of the data sets to be processed i ＝(x j,k ) s×m To be tasked with
The scheduling process is distributed to the proper server for execution, and the scheduling process comprises the following steps:
(1) selecting a candidate server: each server P j Task of upper storage
The set of data slices to be processed being DC i,j Number of data fragments to be processed
All DC i,j Servers not equal to Φ are all tasks
Candidate servers that can be dispatched, the set of all candidate servers being PC = { P = { (P) j |DC i,j J is not equal to phi and is not less than 1 and not more than m, wherein s is a task
The number of the data to be processed is divided into pieces, j is more than or equal to 1 and less than or equal to m, and i is more than or equal to 1 and less than or equal to NI;
(2) and (3) task allocation: selecting the server P with the least data fragments to be processed from the candidate server set PC k Using epsilon PC to execute task
Set of pending data slices DC responsible for its storage i,k Wherein k is more than or equal to 1 and less than or equal to m;
(3) and (4) updating the candidate server: updating each candidate server P j To-be-processed data fragmentation stored in PC (personal computer)DC i,j ＝DC i,j -DC i,k If DC i,j = Φ, it is removed from the candidate server set, where P k For the task currently allocated to execute
J is more than or equal to 1 and less than or equal to m;
(4) and (4) terminating the distribution: circularly executing the steps (2) and (3) until the task
The data fragments to be processed are each processed by an assigned server, i.e.
And the data fragmentation handled by each server is different, i.e.
Wherein j is more than or equal to 1, k is more than or equal to m, and j is not equal to k;
(5) calculating task processing time: task
Time of treatment of
Wherein the task processing time of each server
Wherein
Represents P j The task actually processed
Is the unit data amount per data slice, resource j Is P j Is currently availableData processing resources, χ i Representing tasks
The complex coefficient of calculation of the single data quantity is more than or equal to 1 and less than or equal to m;
(4) And task migration: when the server P j In executing interactive tasks
Receive the triggered task
When, interactive tasks need to be decided
Whether the task needs to be migrated to another server for execution or is continuously executed after waiting for the trigger-type task to be completed on the current server, the specific process is as follows:
(1) and task waiting judgment: if interactive task
At the server P j Is the expected processing time of
Interactive task
Is treated for a time of
Triggered task
At the server P j Is executed at an execution time of
When in use
Without the need to interwork with the task
Migration to other servers P j Performing the step (b), wherein i is more than or equal to 1 and less than or equal to NI, j is more than or equal to 1 and less than or equal to m, and h is more than or equal to 1 and less than or equal to NT;
(2) and (3) task migration judgment: if the server set which does not execute the interaction task in the mobile data center is PM, the current server P is used j Last incomplete interactive task
And unprocessed data
Migration to any server P in the PM with sufficient data processing resources k Go on to execute, unprocessed data
Slave server P j Migration to P k Data transmission time of
When in use
When the task is migrated, when
Waiting triggered task
Continuing to execute the original interactive task after the execution is finished
Wherein d is j,k Presentation Server P j To P k The distance of transmission of (a) is,
represents the migrated data volume, tau is the time required for transmitting a unit data volume on a high-speed bus with unit length between servers,
for triggered type tasks
At the server P j The execution time of.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111062549.XA CN115794322A (en) | 2021-09-10 | 2021-09-10 | Task scheduling method for large mobile equipment mobile data center |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111062549.XA CN115794322A (en) | 2021-09-10 | 2021-09-10 | Task scheduling method for large mobile equipment mobile data center |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115794322A true CN115794322A (en) | 2023-03-14 |
Family
ID=85417294
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111062549.XA Pending CN115794322A (en) | 2021-09-10 | 2021-09-10 | Task scheduling method for large mobile equipment mobile data center |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115794322A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117130790A (en) * | 2023-10-23 | 2023-11-28 | 云南蓝队云计算有限公司 | Dynamic scheduling method for cloud computing resource pool |
-
2021
- 2021-09-10 CN CN202111062549.XA patent/CN115794322A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117130790A (en) * | 2023-10-23 | 2023-11-28 | 云南蓝队云计算有限公司 | Dynamic scheduling method for cloud computing resource pool |
| CN117130790B (en) * | 2023-10-23 | 2023-12-29 | 云南蓝队云计算有限公司 | Dynamic scheduling method for cloud computing resource pool |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20180331897A1 (en) | Method and device for training model in distributed system | |
| CN110442451B (en) | Deep learning-oriented multi-type GPU cluster resource management scheduling method and system | |
| CN108762896A (en) | One kind being based on Hadoop cluster tasks dispatching method and computer equipment | |
| US9846589B2 (en) | Virtual machine placement optimization with generalized organizational scenarios | |
| US10783436B2 (en) | Deep learning application distribution | |
| CN109117265A (en) | The method, apparatus, equipment and storage medium of schedule job in the cluster | |
| CN107168782A (en) | A kind of concurrent computational system based on Spark and GPU | |
| US20140059054A1 (en) | Parallel generation of topics from documents | |
| CN104765640B (en) | A kind of intelligent Service dispatching method | |
| CN110347489B (en) | Multi-center data collaborative computing stream processing method based on Spark | |
| WO2024021489A1 (en) | Task scheduling method and apparatus, and kubernetes scheduler | |
| CN105740059B (en) | A kind of population dispatching method towards Divisible task | |
| CN114610474A (en) | Multi-strategy job scheduling method and system in heterogeneous supercomputing environment | |
| CN116263701A (en) | Computing power network task scheduling method and device, computer equipment and storage medium | |
| CN108572872B (en) | Resource management method based on FPGA reconfigurable technology | |
| WO2020227582A2 (en) | Method and apparatus for scheduling matrix operations in digital processing systems | |
| CN115794322A (en) | Task scheduling method for large mobile equipment mobile data center | |
| CN111061565A (en) | Two-stage pipeline task scheduling method and system in Spark environment | |
| CN102184124B (en) | Task scheduling method and system | |
| US20210390405A1 (en) | Microservice-based training systems in heterogeneous graphic processor unit (gpu) cluster and operating method thereof | |
| CN109976873A (en) | The scheduling scheme acquisition methods and dispatching method of containerization distributed computing framework | |
| Silberstein | Building an online domain-specific computing service over non-dedicated grid and cloud resources: The superlink-online experience | |
| CN116700933A (en) | Heterogeneous computing power federation-oriented multi-cluster job scheduling system and method | |
| CN109446641B (en) | Multi-stage reliability modeling analysis method of cloud computing service system | |
| Khalil et al. | Survey of Apache Spark optimized job scheduling in Big Data |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |