CN108536539B - Task scheduling method in industrial distributed data acquisition system - Google Patents
Task scheduling method in industrial distributed data acquisition system Download PDFInfo
- Publication number
- CN108536539B CN108536539B CN201810384049.XA CN201810384049A CN108536539B CN 108536539 B CN108536539 B CN 108536539B CN 201810384049 A CN201810384049 A CN 201810384049A CN 108536539 B CN108536539 B CN 108536539B
- Authority
- CN
- China
- Prior art keywords
- node
- acquisition
- task
- collection
- tasks
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 33
- 238000013508 migration Methods 0.000 claims abstract description 112
- 230000005012 migration Effects 0.000 claims abstract description 103
- 238000004891 communication Methods 0.000 claims abstract description 13
- 238000005457 optimization Methods 0.000 claims description 18
- 230000008569 process Effects 0.000 claims description 10
- 238000013480 data collection Methods 0.000 claims description 3
- 238000004519 manufacturing process Methods 0.000 description 5
- 238000013507 mapping Methods 0.000 description 3
- 230000002159 abnormal effect Effects 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000009776 industrial production Methods 0.000 description 2
- 238000004088 simulation Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000012938 design process Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5083—Techniques for rebalancing the load in a distributed system
- G06F9/5088—Techniques for rebalancing the load in a distributed system involving task migration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1474—Saving, restoring, recovering or retrying in transactions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/48—Program initiating; Program switching, e.g. by interrupt
- G06F9/4806—Task transfer initiation or dispatching
- G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
- G06F9/4881—Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Quality & Reliability (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention provides a task scheduling method in an industrial distributed data acquisition system, and relates to the technical field of data acquisition. The method comprises the steps of establishing a corresponding relation between acquisition resources and acquisition tasks of each acquisition node, a corresponding relation between acquisition time of each acquisition node and acquisition tasks and utilization rates of the node resources, a corresponding relation between resources consumed by task migration of each acquisition node and the number of migration tasks, and a corresponding relation between communication overhead of task migration among the acquisition nodes and the migration tasks, determining the minimum number of the acquisition nodes in initial work, starting the corresponding acquisition nodes, taking other acquisition node resources as shared resources for standby, and distributing the initial acquisition tasks and redundant tasks of important tasks in part of the initial tasks to the acquisition nodes. The invention comprehensively meets the requirements of industrial distributed data acquisition on acquisition instantaneity, reliability, effective utilization of resources and the like in an industrial big data environment, and improves the resource utilization rate, acquisition efficiency and acquisition reliability of an industrial data acquisition system.
Description
Technical Field
The invention relates to the technical field of data acquisition, in particular to a task scheduling method in an industrial distributed data acquisition system.
Background
With the arrival of industrial big data environment, data sources are increasingly diversified and data scale is increased in the industrial process, and in the face of industrial large-scale high-frequency data acquisition and some new application requirements, more and more enterprises begin to consider adopting a distributed system to carry out industrial data acquisition in order to ensure the time sequence, real-time performance and reliability of data acquisition. In the design process of the distributed data acquisition system, the task scheduling strategy is the most critical link, and the task scheduling strategy directly influences the performance of the distributed system. The good task scheduling scheme can reduce the acquisition time of the acquisition task and improve the acquisition efficiency of the system.
In a distributed environment, because each acquisition node can be flexibly added or withdrawn, the data acquired by each acquisition node does not correspond to a fixed production link any more, and has certain flexibility, due to the burstiness and instability of the network and the reasons of the acquisition nodes, abnormal conditions such as some acquisition node faults, overload, failure of acquisition of a certain acquisition data group or data item on the acquisition node, acquisition time exceeding a set acquisition period and the like can occur, how to carry out dynamic task scheduling and migration according to the resource use condition of each acquisition node can be realized, so that under the abnormal conditions, the data of each production link can still be normally acquired, and the load balance among the acquisition nodes can still be maintained, thereby ensuring the reliability and high efficiency of acquisition, ensuring the amount of migrated tasks as small as possible, ensuring the monotonicity of task migration, and saving the system overhead during task migration as much as possible, the design of the distributed data acquisition system is a problem to be solved urgently, in addition, because the data reliability requirement of an important industrial production link is strict, sometimes, the importance degrees of different data of the same production link are different, the acquisition task of the important data usually needs to be backed up, the backup task only needs to be acquired and does not need to be stored, and only the main task fails to be acquired is stored, under the distributed environment, how to reasonably schedule and migrate the main task and meet the load balance and monotonicity is realized, reasonable scheduling and migration are also realized for the backup task, so that the monotonicity of task migration is ensured as much as possible, and meanwhile, the main task and the corresponding backup task are ensured not to migrate to the same acquisition node, and the problem needs to be considered in the industrial data acquisition.
At present, in the aspect of task scheduling of an industrial distributed data acquisition system, there are mainly a large-scale distributed data acquisition system and method based on an industrial process with a patent number of CN105527948A, a large-scale distributed data acquisition system and method with a patent number of cn201610522950.x, a large-scale distributed intelligent data acquisition system and method based on an industrial cloud with a patent number of CN201610736266.1, and a task scheduling mechanism and system based on a consistent hash algorithm with a patent number of 201610622589.8. When a fault of a collection client is detected, task scheduling is realized by reallocating collection tasks to the rest of the collection clients, although the scheduling scheme meets load balance among nodes, the reallocation of the tasks causes that a large number of tasks need to be migrated on each node, monotonicity of task migration cannot be met, task migration cost is too large, resource utilization rate is reduced, the task scheduling schemes in the two patents are not comprehensive enough, and corresponding solutions are not provided for situations such as failure of task collection on the collection nodes; although the patent CN201610736266.1 gives a task scheduling scheme for collecting node faults, overload, collection task collection failure, and collection time not meeting the requirement of the collection period, how many tasks to be specifically migrated and which tasks to be migrated are not described in detail in scheduling, and the scheduling scheme of the patent does not consider the problems of load balancing of each node, task migration overhead, and the like in the scheduling process, thereby easily affecting the efficiency and real-time performance of data collection; patent 201610622589.8 discloses that mapping relationships between tasks and a plurality of execution units are established on a hash ring by mapping hash values of the tasks and corresponding execution units onto the same hash ring according to the number of task executions and the selected search direction, and when an execution unit is added or deleted or a new task is added, dynamic task scheduling is realized by modifying the mapping relationships. Although the technical scheme in the above patent can realize task scheduling of node failure, the scheduling scheme only aims at the situation that all tasks are important tasks and need to be backed up, or the situation that all tasks do not realize backup, but in the actual industrial production process, all tasks do not have backup, in order to improve the resource utilization rate, the collection tasks corresponding to part of important data in the tasks are often backed up, and the task scheduling mode in the existing patent is not suitable in such a situation.
Disclosure of Invention
The technical problem to be solved by the present invention is to provide a task scheduling method in an industrial distributed data acquisition system, aiming at the defects of the prior art, so as to achieve the purpose of improving the resource utilization rate, the acquisition efficiency and the acquisition reliability of the industrial data acquisition system.
In order to solve the technical problems, the technical scheme adopted by the invention is as follows:
a task scheduling method in an industrial distributed data acquisition system comprises the following steps:
step 1, establishing a corresponding relation between the collection resources and the collection tasks of each collection node, and the specific process is as follows:
step 1.1, distributing acquisition tasks to each acquisition node independently, changing the number of the acquisition tasks, and measuring the number T of the acquisition tasks consumed on the ith acquisition node under the condition of a large number of different acquisition tasksiAnd collecting resource utilization rate data uiAnd the acquisition time data time required by all acquisition tasks distributed on the acquisition nodei；
Step 1.2, collecting task data T on the ith collecting node obtained in step 1.1iAnd corresponding resource consumption data
Fitting to obtain a functional relation between the two
Step 2, establishing a corresponding relation between the acquisition time and the acquisition task of each acquisition node and the utilization rate of the node resources, and counting the number T of the acquisition tasks obtained in the step 1.1iAnd corresponding acquisition resource utilization data uiTime for collecting time dataiFitting to obtain the time of collecting time data on the ith collecting nodeiAnd the number T of collection tasksiAnd resource utilization data uiTime of function relationi＝g(Ti,ui)；
Step 3, establishing a corresponding relation between resources consumed by task migration on each acquisition node and the number of migration tasks;
step 4, establishing a corresponding relation between the communication overhead of task migration and the migration task among the collection nodes;
step 5, determining the number of the minimum acquisition nodes in the initial work and starting the corresponding acquisition nodes, wherein the rest acquisition node resources are used as shared resources for standby;
step 6, distributing the initial acquisition tasks and redundant tasks of important tasks in part of the initial tasks to each acquisition node determined in the step 5;
judging whether a fault of the acquisition node exists or not, if so, executing a step 7; judging whether the conditions that the acquisition nodes are overloaded or the acquisition time on the acquisition nodes does not meet the requirements of the acquisition period exist, if so, executing a step 8; judging whether the condition that the collection of important collection tasks on the collection nodes fails exists or not, if so, executing a step 9; if the situations do not exist, the task scheduling is finished;
and 7, when the acquisition node fault exists, scheduling tasks after the acquisition node fault, wherein the specific scheduling method comprises the following steps:
7.1, in the current acquisition period, storing the acquisition result of the redundant task corresponding to the important task in the initial tasks on the fault acquisition node into a database of the data acquisition system from the corresponding acquisition node;
7.2, in the next acquisition period, searching a node which is closest to the resource residual situation of the fault acquisition node from the redundant hot standby acquisition nodes, taking the node as a newly added acquisition node, and transferring all initial tasks and redundant tasks on the fault acquisition node to the newly added acquisition node for acquisition;
and 8, when the collection node is overloaded or the collection time on the collection node does not meet the requirement of the collection period, scheduling the tasks after the collection node is overloaded, wherein the specific scheduling method comprises the following steps:
step 8.1, establishing an optimization model of the number of the newly added acquisition nodes according to the corresponding relation from the step 1 to the step 4, and determining the minimum value m of the number of the newly added acquisition nodes, wherein the optimization model is as follows:
min m
s.t Tij×Tji＝0 i＝1,2…n；j＝1,2…n,j≠i (1-1)
wherein,
timei≤Time i＝1,2…n (1-7)
wherein,
wherein,
timej≤Time j＝n+1,n+2…n+m (1-12)
wherein,
wherein, the expression (1-1) indicates that if the task is migrated from the node i to the node j, the task is not migrated from the node j to the node i, so that the monotonicity of task migration is ensured, and TijIndicates the number of tasks to be migrated from the ith node to the jth node, TjiRepresents from the jth sectionThe number of tasks for migrating the point to the ith node; the formula (1-2) indicates that one node cannot migrate tasks and cannot migrate tasks, and n indicates the number of collection nodes which work before the node is newly added; the expression (1-3) shows that the resource utilization rate of the ith acquisition node in the initial work acquisition nodes in the task migration process and after the task migration is finished cannot exceed the resource utilization rate upper limit set by the user, ui0Representing the resource utilization rate, R, of the ith node in the initial work collection node before the task is not allocatediRepresents the total amount of available resources, u, configured on the ith collection node in the initial working collection nodeshRepresents the upper limit of the node resource utilization rate, u, set by the userwIndicating a user-set threshold bandwidth of node resource utilization, TiRepresenting the number of collection tasks on the ith node in the initial work collection nodes before task migration,
representing the resources consumed by the execution of the collection task on the ith node in the initial work collection nodes after the task migration,
representing the collection of resources consumed by task migration on the ith node in the initial work collection nodes after the task migration,
and
the values of the two-dimensional space-time-domain data are respectively determined by formulas (1-4) and (1-5), and the specific functional relationship is determined by the steps 1 and 3; the formula (1-6) represents the resource utilization rate of the ith node in the initial work acquisition nodes after the task migration is completed; the expression (1-7) represents the collection time of executing a whole collection task on the ith collection node in the initial work collection nodes after the task migrationiThe collection period Time, Time set by the user cannot be exceedediRepresenting the collection time of executing a whole collection task on the ith node in the initial work collection nodes after the task migration, timeiThe value of (A) is determined by the formula (1-8)Determining, the specific functional relation is determined by the step 2; the expression (1-9) shows that the resource utilization rate of the j acquisition node in the newly added node cannot exceed the resource utilization rate upper limit set by the user, ujIndicating the resource utilization rate u of the j-th node newly added into the nodes after the task migration is finishedj0Indicating the resource utilization rate, R, of the j-th node in the newly added nodes before the task is not allocatedjRepresenting the total amount of available resources configured on the jth collection node in the newly joined nodes,
representing that the j node newly added into the nodes after the task migration collects the resources consumed by the task execution,
representing that the j node newly added into the nodes after the task migration collects the resources consumed by the task migration,
and
the values of the two-dimensional space-time-domain data are respectively determined by formulas (1-10) and (1-11), and the specific functional relationship is determined by the steps 1 and 3; the expression (1-12) shows that the acquisition Time of executing a whole acquisition task on the jth acquisition node in the newly added nodes after the task migration cannot exceed the acquisition period Time, Time set by the userjRepresents the collection time of a whole collection task executed on the jth node newly added into the nodes after the task migration, and is timejThe value of (2) is determined by the formula (1-13), and the specific functional relationship is determined by the step (2);
step 8.2, solving the optimization model in the step 8.1, solving the number of newly added minimum acquisition nodes, selecting the acquisition nodes with corresponding number from the redundant hot standby acquisition nodes, and adding acquisition work;
step 8.3, establishing a task migration optimization model of the collection node, wherein the model is as follows:
wherein,
s.t Tij×Tji＝0 i＝1,2…n；j＝1,2…n,j≠i (1-18)
wherein,
timei≤Time i＝1,2…n (1-23)
wherein,
wherein,
timej≤Time j＝n+1,n+2…n+m (1-27)
wherein,
wherein,
wherein, the expression (1-14) represents that the sum of the resource and communication overhead of each node in the task migration process and the load difference between each collection node after the task migration is completed is minimized, and the sum is delta U, p (T)ij) The communication overhead of task migration between the node i and the node j is represented, the specific functional relationship is determined in the step 4, the delta U represents the sum of load differences among the collection nodes after the task migration is completed, and the value of the delta U is determined by the formula (1-17); the expression (1-29) shows that the number of the tasks migrated from the ith node to the jth node is equal to the sum of the number of the initial tasks and the number of the redundant tasks migrated from the ith node to the jth node, and taskk,i,jRepresenting the initial task of migrating from the ith node to the jth node with number k,
representing a redundant task with the number l from the ith node to the jth node; the formula (1-30) represents taskk,i,jWhen the initial task with the number k is located on the ith node, the corresponding redundant task is not located on the jth node, and the task needs to be migrated to the jth node, the taskk,i,j1, otherwise taskk,i,j＝0，TaskiRepresenting the collection Task sequence number set, Task, on the ith collection node before Task migrationjRepresenting the collection Task sequence number set, Task, on the jth collection node before Task migrationijRepresenting a task sequence number set of the ith collection node migrated to the jth collection node; the formula (1-31) represents
When the redundant task with the number l is positioned on the ith node, the corresponding redundant task is not positioned on the jth node and needs to be migrated to the jth node,
otherwise
Step 8.4, solving the optimization model in the step 8.3 to obtain a Task number set Task for migrating the ith collection node to the jth collection nodeij；
8.5, carrying out task migration among the collection nodes according to the result obtained in the step 8.4;
and 9, when the collection of important collection tasks on the collection nodes fails, storing the collection results of the corresponding redundant tasks into a database of the data collection system from the corresponding collection nodes.
Adopt the produced beneficial effect of above-mentioned technical scheme to lie in: according to the task scheduling method in the industrial distributed data acquisition system, multiple factors such as resource utilization rate of acquisition nodes, acquisition efficiency, load balance, acquisition reliability, heterogeneous acquisition nodes, resources for dynamic task migration between nodes, communication overhead and the like are comprehensively considered, a series of optimization models based on simulation are established, the dynamic task scheduling problem that the acquisition nodes in the industrial distributed data acquisition system are failed, overload is caused, the acquisition time is longer than the acquisition period, and part of acquisition tasks fail to include 1:1 redundant tasks and non-redundant tasks is realized, and the requirements of industrial distributed data acquisition on acquisition instantaneity, reliability, effective utilization of resources and the like in an industrial large data environment are met.
Drawings
Fig. 1 is a flowchart of a task scheduling method in an industrial distributed data acquisition system according to an embodiment of the present invention.
Detailed Description
The following detailed description of embodiments of the present invention is provided in connection with the accompanying drawings and examples. The following examples are intended to illustrate the invention but are not intended to limit the scope of the invention.
In this embodiment, 4 collection nodes are provided, which are numbered 1, 2, 3, and 4 respectively. The memory configuration conditions of the 4 acquisition nodes are respectively 4G, 8G, 4G and 16G, and the resource utilization rates of the 4 acquisition nodes are respectively 10%, 15%, 13% and 27% at the initial allocation moment.
In this embodiment, 35 initial collection tasks F1 to F35 are set, and correspond to collection of data on 35 fans F1 to F35, respectively, each collection task includes 404 collection data items, where 31 Single floor type data, 52 Double floor type data, 319 Boolean floor type data, and 2 unidimensional Integer type data. Among them, F11-F22 are important tasks and need to be redundant. The 35 tasks F1-F35 are numbered as 1-35 respectively, and the redundant tasks F11-F22 are numbered as 11 respectively*,12*,13*,14*,15*,16*,17*,18*,19*,20*,21*,22*。
A task scheduling method in an industrial distributed data acquisition system according to this embodiment is shown in fig. 1, and the specific method is described as follows.
Step 1, establishing a corresponding relation between the acquisition resources of each acquisition node and the acquisition tasks.
Step 1.1, distributing acquisition tasks to each acquisition node independently, changing the number of the acquisition tasks, and measuring the number T of the acquisition tasks consumed on the ith acquisition node when a large number of acquisition tasks with different numbers are acquirediAnd collecting resource utilization rate data uiAnd the acquisition time data time required by all acquisition tasks distributed on the acquisition nodei；
In this embodiment, the collected resource data refers to a memory resource.
In this embodiment, acquisition tasks are allocated to acquisition nodes No. 1, 2, 3, and 4, and the memory resources consumed when acquiring different numbers of acquisition tasks, the memory usage rate thereof, and the acquisition time are measured, and the measured data are shown in tables 1, 2, 3, and 4, respectively.
Table 11 number collection node collects task number, memory utilization rate and collection time data
| Number of collection tasks | 1 | 2 | 3 | 4 | 5 | 6 |
| Consume memory resources (G) | 0.19 | 0.37 | 0.55 | 0.76 | 0.97 | 1.15 |
| Memory usage (%) | 14.75 | 19.25 | 23.75 | 29 | 34.25 | 38.75 |
| Acquisition time (ms) | 305 | 330 | 357 | 390 | 427 | 469 |
| Number of collection tasks | 7 | 8 | 9 | 10 | 11 | 12 |
| Consume memory resources (G) | 1.33 | 1.53 | 1.72 | 1.91 | 2.11 | 2.29 |
| Memory usage (%) | 43.25 | 48.25 | 53 | 57.25 | 62.75 | 67.25 |
| Acquisition time (ms) | 508 | 559 | 607 | 661 | 728 | 788 |
Table 22 number collection node collects task number, memory utilization rate and collection time data
Acquisition node No. 33 in table acquires task number, memory utilization rate and acquisition time data
| Number of collection tasks | 1 | 2 | 3 | 4 | 5 | 6 |
| Consume memory resources (G) | 0.18 | 0.38 | 0.59 | 0.77 | 0.98 | 1.17 |
| Memory usage (%) | 17.5 | 22.5 | 27.75 | 32.25 | 37.5 | 42.25 |
| Acquisition time (ms) | 319 | 346 | 380 | 415 | 454 | 498 |
| Number of collection tasks | 7 | 8 | 9 | 10 | 11 | 12 |
| Consume memory resources (G) | 1.34 | 1.52 | 1.72 | 1.88 | 2.09 | 2.27 |
| Memory usage (%) | 46.5 | 51 | 56 | 60 | 65.25 | 69.75 |
| Acquisition time (ms) | 539 | 589 | 647 | 694 | 763 | 824 |
Table 44 number collection node collects task number, memory utilization rate and collection time data
Step 1.2, collecting task data T on the ith collecting node obtained in step 1.1iAnd corresponding resource consumption data
Fitting is carried outTo obtain
And TiFunctional relation of
In this embodiment, number 1 acquisition node obtained by fitting
And T1Has a functional relation of
No. 2 acquisition node
And T2Has a functional relation of
No. 3 acquisition node
And T3Has a functional relation of
No. 3 acquisition node
And T4Has a functional relation of
Step 2, establishing a corresponding relation between the acquisition time of each acquisition node and the resource utilization rate corresponding to the acquisition of different numbers of acquisition tasks, and acquiring T on each node obtained in step 1.1iAcquisition resource utilization rate u corresponding to each acquisition taskiAnd the collection timeiFitting to obtain the acquisition time on the ith acquisition nodeiAnd collect T on the nodeiIndividual miningCollecting resource utilization rate u corresponding to task setiTime of function relationi＝g(ui)。
In this embodiment, the data obtained in step 1.1 is fitted by using a least square method, and the number 1 acquisition node time obtained by fitting1And u1Has a functional relation of time1＝0.0869u1 2+2.0390u1+257.6073, No. 2 collecting node time2And u2Has a functional relation of time2＝0.1370u2 2-0.2669u2+161.6201, No. 3 acquisition node time3And u3Has a functional relation of time3＝0.0880u3 2+2.0146u3+256.5218, No. 4 acquisition node time4And u4Has a functional relation of time4＝0.3159u4 2-10.3928u4+190.3659。
And 3, establishing a corresponding relation between resources consumed by task migration on each acquisition node and the number of migration tasks.
In this embodiment, the resource overhead of the migration task between the nodes is very small compared with the resource overhead of the task execution, and can be ignored.
And 4, establishing a corresponding relation between the communication overhead of task migration and the number of the migration tasks among the collection nodes.
In this embodiment, since optical fiber communication is adopted among the nodes, the communication overhead of task migration is very small and can be ignored.
And 5, determining the number of the minimum acquisition nodes in the initial work, starting the corresponding acquisition nodes, and taking the rest acquisition node resources as shared resources for standby.
In this embodiment, the number of the collection nodes in the initial work is 2, which are respectively No. 3 and No. 4 collection nodes, and No. 1 and No. 2 collection nodes are used as shared resources for standby. The purpose of sharing resource standby is to save server resources for data acquisition, improve the utilization rate of the whole resources, and provide resource application to the system when the acquisition task changes or resources need to be added.
And 6, distributing the initial acquisition tasks and redundant tasks of important tasks in part of the initial tasks to the acquisition nodes determined in the step 5.
In this embodiment, the task set assigned to the collection node No. 3 is {11-16, 17-22 }, and the task set assigned to the collection node No. 4 is {1-10, 11-16, 17-35 }.
And 7, when the acquisition node fails, scheduling tasks after the acquisition node fails.
In this embodiment, assuming that the No. 3 collection node fails, the task scheduling method after the collection node fails is as follows.
7.1, in the current acquisition period, storing the acquisition result of the redundant task corresponding to the important task in the initial tasks on the fault acquisition node into a database of the data acquisition system from the corresponding acquisition node; the data acquisition system can be an SQL Server database or an HBase database and the like according to the type of the database of the data acquisition system;
in this embodiment, in the current fault acquisition period, redundant tasks corresponding to acquisition tasks No. 11 to 16 on acquisition node No. 3, that is, 11*-16*The number acquisition task is stored in a database from the number 4 acquisition node;
7.2, in the next acquisition period, searching a node which is closest to the resource residual situation of the fault acquisition node from the redundant hot standby acquisition nodes, taking the node as a newly added acquisition node, and transferring all initial tasks and redundant tasks on the fault acquisition node to the newly added acquisition node for acquisition;
in this embodiment, in the next acquisition cycle after the fault, the acquisition node No. 1 similar to the resource residual situation of the acquisition node No. 3 with the fault is selected from the acquisition nodes No. 1 and No. 2 as a newly added acquisition node, and all the initial tasks {11-16} and the redundant tasks {17 } on the acquisition node No. 3 are performed*-22*And moving to the No. 1 acquisition node for acquisition.
Step 8, when the collection node is overloaded or the collection time on the collection node does not meet the requirement of the collection period, scheduling tasks after the collection node is overloaded;
in this embodiment, assuming that the number 3 acquisition node is overloaded, the task scheduling method after the overload of the acquisition node is as follows.
Step 8.1, establishing an optimization model of the number of the newly added acquisition nodes according to the simulation results of the steps 1 to 4, and determining the minimum value m of the number of the newly added acquisition nodes, wherein the optimization model is as follows:
min m
s.t Tij×Tji＝0 i＝1,2…n；j＝1,2…n,j≠i (1-1)
wherein,
timei≤Time i＝1,2…n (1-7)
wherein,
wherein,
timej≤Time j＝n+1,n+2…n+m (1-12)
wherein,
wherein, the expression (1-1) indicates that if the task is migrated from the node i to the node j, the task is not migrated from the node j to the node i, so that the monotonicity of task migration is ensured, and TijIndicates the number of tasks to be migrated from the ith node to the jth node, TjiRepresenting the number of tasks migrated from the jth node to the ith node; the formula (1-2) indicates that one node cannot migrate tasks and cannot migrate tasks, and n indicates the number of collection nodes which work before the node is newly added; the expression (1-3) shows that the resource utilization rate of the ith acquisition node in the initial work acquisition nodes in the task migration process and after the task migration is finished cannot exceed the resource utilization rate upper limit set by the user, ui0Representing the resource utilization rate, R, of the ith node in the initial work collection node before the task is not allocatediRepresents the total amount of available resources, u, configured on the ith collection node in the initial working collection nodeshRepresents the upper limit of the node resource utilization rate, u, set by the userwIndicating a user-set threshold bandwidth of node resource utilization, TiRepresenting the number of collection tasks on the ith node in the initial work collection nodes before task migration,
representing the resources consumed by the execution of the collection task on the ith node in the initial work collection nodes after the task migration,
representing resources consumed by the migration of the collection task on the ith node in the initial work collection nodes,
and
the values of the two-dimensional space-time-domain data are respectively determined by formulas (1-4) and (1-5), and the specific functional relationship is determined by the steps 1 and 3; the formula (1-6) represents the resource utilization rate of the ith node in the initial work acquisition nodes after the task migration is completed; the expression (1-7) shows that the acquisition Time of executing all acquisition tasks once on the ith acquisition node in the initial work acquisition nodes after the task migration cannot exceed the acquisition period Time, Time set by the useriRepresenting the collection time of executing a whole collection task on the ith node in the initial work collection nodes after the task migration, timeiThe value of (2) is determined by the formula (1-8), and the specific functional relationship is determined by the step (2); the expression (1-9) shows that the resource utilization rate of the j acquisition node in the newly added node cannot exceed the resource utilization rate upper limit set by the user, ujIndicating the resource utilization rate u of the j-th node newly added into the nodes after the task migration is finishedj0Indicating the resource utilization rate, R, of the j-th node in the newly added nodes before the task is not allocatedjRepresenting the total amount of available resources configured on the jth collection node in the newly joined nodes,
representing that the j node newly added into the nodes after the task migration collects the resources consumed by the task execution,
indicating resources consumed by the collection task migration on the jth node in the newly added nodes,
and
the values of the two-dimensional space-time-domain data are respectively determined by formulas (1-10) and (1-11), and the specific functional relationship is determined by the steps 1 and 3; the expression (1-12) shows that the acquisition time of executing a whole acquisition task on the jth acquisition node in the newly added nodes after the task migration cannot exceed the user settingFixed acquisition period TimejRepresents the collection time of a whole collection task executed on the jth node newly added into the nodes after the task migration, and is timejThe value of (2) is determined by the formula (1-13), and the specific functional relationship is determined by the step (2);
in this example, R1＝4G，R2＝8G，R3＝4G，R4＝16G，uh＝80％，uw＝5％，u10＝10％，u20＝15％，u30＝13％，u40When the Time is 27%, the Time is 1000ms, and M is 47, the optimization model is specifically:
min m
s.t T43×T34＝0 (2-1)
wherein,
wherein,
time3≤1000 (2-10)
wherein, time3＝0.0880u3 2+2.0146u3+256.5218 (2-11)
time4≤1000 (2-12)
Wherein, time4＝0.3159u4 2-10.3928u4+190.3659 (2-13)
Wherein,
wherein,
time1≤1000 (2-18)
wherein, time1＝0.0869u1 2+2.0390u1+257.6073 (2-19)
time2≤1000 (2-20)
Wherein, time2＝0.1370u2 2-0.2669u2+161.6201 (2-21)
Step 8.2, solving the optimization model in the step 8.1, solving the number of newly added minimum acquisition nodes, selecting corresponding acquisition nodes from the redundant hot standby acquisition nodes, and adding acquisition work;
in this embodiment, the number of the minimum acquisition nodes obtained by solving is 1, and the newly added acquisition node is the acquisition node No. 1;
step 8.3, establishing a task migration optimization model of the collection node, wherein the model is as follows:
wherein,
s.t Tij×Tji＝0 i＝1,2…n；j＝1,2…n,j≠i (1-18)
wherein,
timei≤Time i＝1,2…n (1-23)
wherein,
wherein,
timej≤Time j＝n+1,n+2…n+m (1-27)
wherein,
wherein,
wherein, the expression (1-14) represents that the sum of the resource and communication overhead of each node in the task migration process and the load difference between each collection node after the task migration is completed is minimized,
representing the collection of resources consumed by task migration on the ith node in the initial work collection nodes after the task migration,
representing that the j node newly added into the nodes after the task migration collects the resources consumed by the task migration,
and
the values of (A) are respectively determined by the formulas (1-15) and (1-16), the specific functional relationship is determined by the step 3, and p (T)ij) The communication overhead of task migration between the node i and the node j is represented, the specific functional relationship is determined in the step 4, the delta U represents the sum of load differences among the collection nodes after the task migration is completed, and the value of the delta U is determined by the formula (1-17); the expression (1-18) shows that if the task is migrated from the node i to the node j, the task is not migrated from the node j to the node i, so that the monotonicity of task migration is ensured, and TijIndicates the number of tasks to be migrated from the ith node to the jth node, TjiRepresenting the number of tasks migrated from the jth node to the ith node; the formula (1-19) indicates that one node cannot migrate tasks and cannot migrate tasks, and n indicates the number of collection nodes which work before the node is newly added; the expression (1-20) shows that the resource utilization rate of the ith acquisition node in the initial work acquisition nodes in the task migration process and after the task migration is finished cannot exceed the resource utilization rate upper limit set by the user, ui0Representing the resource utilization rate, R, of the ith node in the initial work collection node before the task is not allocatediRepresents the total amount of available resources, u, configured on the ith collection node in the initial working collection nodeshRepresents the upper limit of the node resource utilization rate, u, set by the userwIndicating a user-set threshold bandwidth of node resource utilization, TiRepresenting the number of collection tasks on the ith node in the initial work collection nodes before task migration,
representing the resources consumed by the execution of the collection task on the ith node in the initial work collection nodes after the task migration,
the value of (a) is determined by an expression (1-21), and the specific functional relationship is determined by the step (1); the formula (1-22) represents the resource utilization rate of the ith node in the initial work acquisition node after the task is migrated; the expression (1-23) represents the collection time of executing a whole collection task on the ith collection node in the initial work collection nodes after the task migrationiAcquisition period set by user cannot be exceededTime，timeiThe value of (2) is determined by the formula (1-24), and the specific functional relationship is determined by the step (2); the expression (1-25) shows that the resource utilization rate of the j acquisition node in the newly added node cannot exceed the resource utilization rate upper limit set by the user, ujIndicating the resource utilization rate u of the j-th node newly added into the nodes after the task migration is finishedj0Indicating that the j node in the newly added node is not allocated with the resource utilization rate before the task, RjRepresenting the total amount of available resources configured on the jth collection node in the newly joined nodes,
representing that the j node newly added into the nodes after the task migration collects the resources consumed by the task execution,
the value of (2) is determined by the formula (1-26), and the specific functional relationship is determined by the step (1); the expression (1-27) represents the collection time of executing a whole collection task on the jth collection node in the newly added nodes after the task migrationjThe collection period Time, Time set by the user cannot be exceededjThe value of (2) is determined by the formula (1-28), and the specific functional relationship is determined by the step (2); the expression (1-29) shows that the number of the tasks migrated from the ith node to the jth node is equal to the sum of the number of the initial tasks and the number of the redundant tasks migrated from the ith node to the jth node, and taskk,i,jRepresenting the initial task of migrating from the ith node to the jth node with number k,
representing a redundant task with the number l from the ith node to the jth node; the formula (1-30) represents taskk,i,jWhen the initial task with the number k is located on the ith node, the corresponding redundant task is not located on the jth node, and the task needs to be migrated to the jth node, the taskk,i,j1, otherwise taskk,i,j＝0，TaskiRepresenting the collection Task sequence number set, Task, on the ith collection node before Task migrationjRepresenting collection task sequence number set on jth collection node before task migration，TaskijRepresenting a task sequence number set of the ith collection node migrated to the jth collection node; the formula (1-31) represents
When the redundant task with the number l is positioned on the ith node, the corresponding redundant task is not positioned on the jth node and needs to be migrated to the jth node,
otherwise
In this embodiment, the optimization model is as follows:
min Cost＝ΔU (2-22)
wherein,
s.t T43×T34＝0 (2-24)
(T31+T34)×T43＝0 (2-25)
(T41+T43)×T34＝0 (2-26)
wherein,
wherein,
time3≤1000 (2-33)
wherein, time3＝0.0880u3 2+2.0146u3+256.5218 (2-34)
time4≤1000 (2-35)
Wherein, time4＝0.3159u4 2-10.3928u4+190.3659 (2-36)
Wherein,
time1≤1000 (2-39)
wherein, time1＝0.0869u1 2+2.0390u1+257.6073 (2-40)
Wherein,
step 8.4, solving the optimization model in the step 8.3 to obtain the migration from the ith collection node to the jth collection nodeTask sequence number set Task of collection nodeij；
In this embodiment, one feasible migration scheme obtained by solving is as follows: task sequence number set Task for migrating No. 3 collection node to No. 1 collection node31Collection node # 4 migrates to collection node # 1's Task number set Task {11,12}41＝{1,2,3,4,5,6,7,8}；
And 8.5, carrying out task migration among the collection nodes according to the result obtained in the step 8.4.
In this embodiment, according to the result obtained in step 8.4, the acquisition tasks No. 11 and No. 12 on the acquisition node No. 3 are migrated to the acquisition node No. 1 that is newly added, and the acquisition tasks No. 1 to No. 8 on the acquisition node No. 4 are migrated to the acquisition node No. 1 that is newly added.
9, when the collection of important collection tasks on the collection nodes fails, storing the collection results of the corresponding redundant tasks in a database from the corresponding collection nodes;
in this embodiment, assuming that the acquisition of the acquisition task No. 11 on the acquisition node No. 3 fails, the corresponding redundant task 11 is executed*And storing the data from the No. 4 acquisition node into a database.
Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some or all of the technical features may be equivalently replaced; such modifications and substitutions do not depart from the spirit of the corresponding technical solutions and scope of the present invention as defined in the appended claims.
Claims (1)
1. A task scheduling method in an industrial distributed data acquisition system is characterized in that: the method comprises the following steps:
step 1, establishing a corresponding relation between the collection resources and the collection tasks of each collection node; the specific process is as follows:
step 1.1, distributing acquisition tasks to each acquisition node independently and changing acquisitionIntegrating the number of tasks, measuring the number T of acquisition tasks consumed on the ith acquisition node under the condition of a large number of different acquisition tasksiAnd collecting resource utilization rate data uiAnd the acquisition time data time required by all acquisition tasks distributed on the acquisition nodei；
Step 1.2, collecting task data T on the ith collecting node obtained in step 1.1iAnd corresponding resource consumption data
Fitting to obtain a functional relation between the two
Step 2, establishing a corresponding relation between the acquisition time and the acquisition task of each acquisition node and the utilization rate of the node resources;
in the step 2, the number T of the collection tasks obtained in the step 1.1 is countediAnd corresponding acquisition resource utilization data uiTime for collecting time dataiFitting to obtain the time of collecting time data on the ith collecting nodeiAnd the number T of collection tasksiAnd resource utilization data uiTime of function relationi＝g(Ti,ui)；
Step 3, establishing a corresponding relation between resources consumed by task migration on each acquisition node and the number of migration tasks;
step 4, establishing a corresponding relation between the communication overhead of task migration and the migration task among the collection nodes;
step 5, determining the number of the minimum acquisition nodes in the initial work and starting the corresponding acquisition nodes, wherein the rest acquisition node resources are used as shared resources for standby;
step 6, distributing the initial acquisition tasks and redundant tasks of important tasks in part of the initial tasks to each acquisition node determined in the step 5;
judging whether a fault of the acquisition node exists or not, if so, executing a step 7; judging whether the conditions that the acquisition nodes are overloaded or the acquisition time on the acquisition nodes does not meet the requirements of the acquisition period exist, if so, executing a step 8; judging whether the condition that the collection of important collection tasks on the collection nodes fails exists or not, if so, executing a step 9; if the situations do not exist, the task scheduling is finished;
and 7, when the acquisition node fault exists, scheduling tasks after the acquisition node fault, wherein the specific scheduling method comprises the following steps:
7.1, in the current acquisition period, storing the acquisition result of the redundant task corresponding to the important task in the initial tasks on the fault acquisition node into a database of the data acquisition system from the corresponding acquisition node;
7.2, in the next acquisition period, searching a node which is closest to the resource residual situation of the fault acquisition node from the redundant hot standby acquisition nodes, taking the node as a newly added acquisition node, and transferring all initial tasks and redundant tasks on the fault acquisition node to the newly added acquisition node for acquisition;
and 8, when the collection node is overloaded or the collection time on the collection node does not meet the requirement of the collection period, scheduling the tasks after the collection node is overloaded, wherein the specific scheduling method comprises the following steps:
step 8.1, establishing an optimization model of the number of the newly added acquisition nodes according to the corresponding relation from the step 1 to the step 4, and determining the minimum value m of the number of the newly added acquisition nodes; the optimization model is as follows:
min m
s.t Tij×Tji＝0 i＝1,2…n；j＝1,2…n,j≠i (1-1)
wherein,
timei≤Time i＝1,2…n (1-7)
wherein,
wherein,
timej≤Time j＝n+1,n+2…n+m (1-12)
wherein,
wherein, the expression (1-1) indicates that if the task is migrated from the node i to the node j, the task is not migrated from the node j to the node i, so that the monotonicity of task migration is ensured, and TijIndicates the number of tasks to be migrated from the ith node to the jth node, TjiRepresenting the number of tasks migrated from the jth node to the ith node; the formula (1-2) indicates that a node cannot migrate both tasks, n indicates work before a new node is addedThe number of the collection nodes; the expression (1-3) shows that the resource utilization rate of the ith acquisition node in the initial work acquisition nodes in the task migration process and after the task migration is finished cannot exceed the resource utilization rate upper limit set by the user, ui0Representing the resource utilization rate, R, of the ith node in the initial work collection node before the task is not allocatediRepresents the total amount of available resources, u, configured on the ith collection node in the initial working collection nodeshRepresents the upper limit of the node resource utilization rate, u, set by the userwIndicating a user-set threshold bandwidth of node resource utilization, TiRepresenting the number of collection tasks on the ith node in the initial work collection nodes before task migration,
representing the resources consumed by the execution of the collection task on the ith node in the initial work collection nodes after the task migration,
representing the collection of resources consumed by task migration on the ith node in the initial work collection nodes after the task migration,
and
the values of the two-dimensional space-time-domain data are respectively determined by formulas (1-4) and (1-5), and the specific functional relationship is determined by the steps 1 and 3; the formula (1-6) represents the resource utilization rate of the ith node in the initial work acquisition nodes after the task migration is completed; the expression (1-7) represents the collection time of executing a whole collection task on the ith collection node in the initial work collection nodes after the task migrationiThe collection period Time, Time set by the user cannot be exceedediRepresenting the collection time of executing a whole collection task on the ith node in the initial work collection nodes after the task migration, timeiThe value of (2) is determined by the formula (1-8), and the specific functional relationship is determined by the step (2); the expression (1-9) indicates the j-th collection in the newly added nodeThe resource utilization rate of the node can not exceed the upper limit of the resource utilization rate set by the user ujIndicating the resource utilization rate u of the j-th node newly added into the nodes after the task migration is finishedj0Indicating the resource utilization rate, R, of the j-th node in the newly added nodes before the task is not allocatedjRepresenting the total amount of available resources configured on the jth collection node in the newly joined nodes,
representing that the j node newly added into the nodes after the task migration collects the resources consumed by the task execution,
representing that the j node newly added into the nodes after the task migration collects the resources consumed by the task migration,
and
the values of the two-dimensional space-time-domain data are respectively determined by formulas (1-10) and (1-11), and the specific functional relationship is determined by the steps 1 and 3; the expression (1-12) shows that the acquisition Time of executing a whole acquisition task on the jth acquisition node in the newly added nodes after the task migration cannot exceed the acquisition period Time, Time set by the userjRepresents the collection time of a whole collection task executed on the jth node newly added into the nodes after the task migration, and is timejThe value of (2) is determined by the formula (1-13), and the specific functional relationship is determined by the step (2);
step 8.2, solving the optimization model in the step 8.1, solving the number of newly added minimum acquisition nodes, selecting the acquisition nodes with corresponding number from the redundant hot standby acquisition nodes, and adding acquisition work;
step 8.3, establishing a task migration optimization model of the collection node, which comprises the following steps:
wherein,
s.t Tij×Tji＝0 i＝1,2…n；j＝1,2…n,j≠i (1-18)
wherein,
timei≤Time i＝1,2…n (1-23)
wherein,
wherein,
timej≤Time j＝n+1,n+2…n+m (1-27)
wherein,
wherein,
wherein, the expression (1-14) represents that the sum of the resource and communication overhead of each node in the task migration process and the load difference between each collection node after the task migration is completed is minimized, and the sum is delta U, p (T)ij) The communication overhead of task migration between the node i and the node j is represented, the specific functional relationship is determined in the step 4, the delta U represents the sum of load differences among the collection nodes after the task migration is completed, and the value of the delta U is determined by the formula (1-17); the expression (1-29) shows that the number of the tasks migrated from the ith node to the jth node is equal to the sum of the number of the initial tasks and the number of the redundant tasks migrated from the ith node to the jth node, and taskk,i,jRepresenting the initial task of migrating from the ith node to the jth node with number k,
representing a redundant task with the number l from the ith node to the jth node; the formula (1-30) represents taskk,i,jWhen the initial task with the number k is located at the second placeWhen the corresponding redundant task on the i nodes is not positioned on the jth node and needs to be migrated to the jth node, taskk,i,j1, otherwise taskk,i,j＝0，TaskiRepresenting the collection Task sequence number set, Task, on the ith collection node before Task migrationjRepresenting the collection Task sequence number set, Task, on the jth collection node before Task migrationijRepresenting a task sequence number set of the ith collection node migrated to the jth collection node; the formula (1-31) represents
When the redundant task with the number l is positioned on the ith node, the corresponding redundant task is not positioned on the jth node and needs to be migrated to the jth node,
otherwise
Step 8.4, solving the optimization model in the step 8.3 to obtain a Task number set Task for migrating the ith collection node to the jth collection nodeij；
8.5, carrying out task migration among the collection nodes according to the result obtained in the step 8.4;
and 9, when the collection of important collection tasks on the collection nodes fails, storing the collection results of the corresponding redundant tasks into a database of the data collection system from the corresponding collection nodes.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810384049.XA CN108536539B (en) | 2018-04-26 | 2018-04-26 | Task scheduling method in industrial distributed data acquisition system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810384049.XA CN108536539B (en) | 2018-04-26 | 2018-04-26 | Task scheduling method in industrial distributed data acquisition system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108536539A CN108536539A (en) | 2018-09-14 |
| CN108536539B true CN108536539B (en) | 2021-12-17 |
Family
ID=63478226
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810384049.XA Active CN108536539B (en) | 2018-04-26 | 2018-04-26 | Task scheduling method in industrial distributed data acquisition system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108536539B (en) |
Families Citing this family (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109634915A (en) * | 2018-11-28 | 2019-04-16 | 深圳市网心科技有限公司 | File dispositions method, Cloud Server, system and storage medium |
| CN110377430B (en) * | 2019-07-24 | 2021-08-31 | 中南民族大学 | Data migration method, device, storage medium and device |
| CN110389555B (en) * | 2019-08-16 | 2022-05-31 | 山东省金曼克电气集团股份有限公司 | Monitoring system and monitoring method for marine transformer |
| CN111142788B (en) * | 2019-11-29 | 2021-10-15 | 浪潮电子信息产业股份有限公司 | Data migration method and device and computer readable storage medium |
| WO2021174466A1 (en) * | 2020-03-04 | 2021-09-10 | 深圳市大疆创新科技有限公司 | Self-adaptive load balancing method and system, and storage medium |
| CN114124958A (en) * | 2021-11-26 | 2022-03-01 | 智昌科技集团股份有限公司 | Operation method of distributed data acquisition system |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1756190A (en) * | 2004-09-30 | 2006-04-05 | 北京航空航天大学 | Distributed performance data acquisition method |
| CN104702598A (en) * | 2015-02-16 | 2015-06-10 | 南京邮电大学 | Distributed network protocol security detection method for smart power grid |
| CN105527948A (en) * | 2015-12-11 | 2016-04-27 | 东北大学 | Large scale distributed data acquisition system and method based on industrial process |
| CN106126346A (en) * | 2016-07-05 | 2016-11-16 | 东北大学 | A kind of large-scale distributed data collecting system and method |
| CN106357426A (en) * | 2016-08-26 | 2017-01-25 | 东北大学 | Large-scale distribution intelligent data collection system and method based on industrial cloud |
| CN107918561A (en) * | 2017-11-17 | 2018-04-17 | 东北大学 | A kind of method for allocating tasks in industrial allocation formula data collecting system |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2014142678A (en) * | 2013-01-22 | 2014-08-07 | Hitachi Ltd | Virtual server transfer plan generation method and system |
-
2018
- 2018-04-26 CN CN201810384049.XA patent/CN108536539B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1756190A (en) * | 2004-09-30 | 2006-04-05 | 北京航空航天大学 | Distributed performance data acquisition method |
| CN104702598A (en) * | 2015-02-16 | 2015-06-10 | 南京邮电大学 | Distributed network protocol security detection method for smart power grid |
| CN105527948A (en) * | 2015-12-11 | 2016-04-27 | 东北大学 | Large scale distributed data acquisition system and method based on industrial process |
| CN106126346A (en) * | 2016-07-05 | 2016-11-16 | 东北大学 | A kind of large-scale distributed data collecting system and method |
| CN106357426A (en) * | 2016-08-26 | 2017-01-25 | 东北大学 | Large-scale distribution intelligent data collection system and method based on industrial cloud |
| CN107918561A (en) * | 2017-11-17 | 2018-04-17 | 东北大学 | A kind of method for allocating tasks in industrial allocation formula data collecting system |
Non-Patent Citations (1)
| Title |
|---|
| 流程工业中设备逻辑控制程序测试平台的建立;吴永建 等;《第25届中国过程控制会议》;20140809;第1-8页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108536539A (en) | 2018-09-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108536539B (en) | Task scheduling method in industrial distributed data acquisition system | |
| CN106126346B (en) | A kind of large-scale distributed data collection system and method | |
| CN107918561B (en) | Task allocation method in industrial distributed data acquisition system | |
| US11474874B2 (en) | Systems and methods for auto-scaling a big data system | |
| Shi et al. | Empirical evaluation of vector bin packing algorithms for energy efficient data centers | |
| CN102685237B (en) | Method for requesting session maintaining and dispatching in cluster environment | |
| CN103227838B (en) | A kind of multi-load equilibrium treatment apparatus and method | |
| CN105159769A (en) | Distributed job scheduling method suitable for heterogeneous computational capability cluster | |
| CN105635199B (en) | A kind of self-organizing cluster server of holding load equilibrium | |
| Chang et al. | A load-balance based resource-scheduling algorithm under cloud computing environment | |
| CN110489204A (en) | A kind of big data platform architecture system based on container cluster | |
| Madsen et al. | Integrative dynamic reconfiguration in a parallel stream processing engine | |
| CN110958311A (en) | YARN-based shared cluster elastic expansion system and method | |
| CN112631764A (en) | Task scheduling method and device, computer equipment and computer readable medium | |
| Yagoubi et al. | A load balancing model for grid environment | |
| CN104484228A (en) | Distributed parallel task processing system based on Intelli-DSC (Intelligence-Data Service Center) | |
| Singh et al. | Evaluating overheads of integrated multilevel checkpointing algorithms in cloud computing environment | |
| Sharma et al. | An agent based dynamic resource scheduling model with FCFS-job grouping strategy in grid computing | |
| CN103268261A (en) | Hierarchical computing resource management method suitable for large-scale high-performance computer | |
| CN110838988B (en) | Network flow current limiting system | |
| Wang et al. | A BSP-based parallel iterative processing system with multiple partition strategies for big graphs | |
| Zhang et al. | An on-the-fly scheduling strategy for distributed stream processing platform | |
| CN103973811A (en) | High-availability cluster management method capable of conducting dynamic migration | |
| CN113872673A (en) | Satellite constellation simulation container cloud resource scheduling system and scheduling method thereof | |
| CN106899392B (en) | Method for carrying out fault tolerance on instantaneous fault in EtherCAT message transmission process |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |