CN111539863A - Intelligent city operation method and system based on multi-source task line - Google Patents
Intelligent city operation method and system based on multi-source task line Download PDFInfo
- Publication number
- CN111539863A CN111539863A CN202010225913.9A CN202010225913A CN111539863A CN 111539863 A CN111539863 A CN 111539863A CN 202010225913 A CN202010225913 A CN 202010225913A CN 111539863 A CN111539863 A CN 111539863A
- Authority
- CN
- China
- Prior art keywords
- task
- calculation
- cloud server
- cloud
- local
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Systems or methods specially adapted for specific business sectors, e.g. utilities or tourism
- G06Q50/10—Services
- G06Q50/26—Government or public services
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/48—Program initiating; Program switching, e.g. by interrupt
- G06F9/4806—Task transfer initiation or dispatching
- G06F9/4812—Task transfer initiation or dispatching by interrupt, e.g. masked
- G06F9/4831—Task transfer initiation or dispatching by interrupt, e.g. masked with variable priority
Abstract
The invention discloses a method and a system for running an intelligent city based on a multi-source task line, wherein the method mainly comprises the steps of analyzing and calculating any data in the running of the intelligent city, and splitting the data into a local calculation task line and one or more cloud calculation task lines; determining the data calculation amount and the calculation thread number of the cloud server according to the principle of optimal utility, and further determining the number of task threads capable of being born by the cloud server and the expected completion time of the task threads; and distributing the split cloud computing task lines to appropriate cloud servers in a mode of maximally paralleling the local computing task lines and the cloud computing task lines of one data analysis operation requirement. The system mainly comprises a cloud server, a local terminal, a task scheduling controller and a coordinator. By the system and the method, the calculation of the multitask line is realized, the calculation and transmission efficiency is improved, and the overall benefit is maximized.
Description
Technical Field
The invention relates to the technical field of multisource task lines of intelligent cities, in particular to a method and a system for operating an intelligent city based on a multisource task line.
Background
The intelligent city is an intelligent system which takes the wireless Internet of things as a medium and covers the whole city range, various sensing facilities, monitoring facilities, access control card swiping facilities, traffic facilities, lighting facilities, multimedia facilities and the like in the city can upload and download data by utilizing the wireless Internet of things, so that the preset intelligent function of the intelligent city is realized on the basis of data acquisition and analysis, for example, the sensing facilities analyze the conditions of temperature, humidity, illumination intensity and the like of a city area, the monitoring facilities and the access control card swiping facilities perform identity verification, traffic control, suspicious alarm and the like on the basis of human body characteristic quantity identification such as human faces and the like, and the traffic facilities realize traffic current limiting, traffic command and the like, so that the operation of the whole intelligent city is supported.
As described above, most of the data analysis operations in the smart city have large computation and data processing amounts, and as the intelligence develops in the depth direction, the magnitude of the computation and data amount of the data analysis operations increases significantly, and the processing capability, data storage capability, battery standby capability, and the like of local computation of a single smart facility become insufficient.
The intelligent facility at the front end of the cloud computing is only responsible for basic data acquisition, result feedback, action execution and the like, data analysis and operation are moved to a server at the cloud end, and the server is executed by utilizing the powerful computing capability. However, in the scale and data magnitude of the smart city, cloud computing adds intermediate links, which increases communication load, increases overhead of data operation, and decreases response speed of smart facilities.
Therefore, for the operation of the smart city, the data analysis operation needs to realize the optimized balance between the local calculation and the cloud calculation, and the operation tasks are disassembled and distributed locally and at the cloud, so that the overall benefit is optimized.
In addition, the coverage space and the network scale of the Internet of things of the intelligent city are large, so that the sources of data analysis and operation tasks are numerous, and the parallel characteristic of a multi-source task line is presented. How to realize effective dynamic management between local calculation and cloud calculation for multi-source and concurrent task lines related to data analysis and calculation, keep the overall smoothness of the task lines, avoid excessive stagnation and interruption and solve the problems.
Disclosure of Invention
Objects of the invention
In order to overcome at least one defect in the prior art, the method and the system for operating the smart city based on the multi-source task line distribute the calculation tasks to the local calculation task line and the cloud task line for calculation, so that the calculation and the processing of the multi-source task line are realized, the optimal balance between the local calculation and the cloud calculation is realized through data analysis and calculation, and the calculation tasks are disassembled and distributed locally and at the cloud, so that the overall benefit is optimized.
(II) technical scheme
As a first aspect of the invention, the invention discloses a method for operating an intelligent city based on a multi-source task line, which comprises the following steps:
for any data analysis and operation requirement in the operation of the intelligent city, splitting the data analysis and operation requirement into a local calculation task line and one or more cloud calculation task lines;
determining the data calculation amount and the calculation thread number of the cloud server according to the principle of optimal utility, and further determining the number of task threads capable of being born by the cloud server and the expected completion time of the task threads;
and distributing the split cloud computing task lines to appropriate cloud servers in a mode of maximally paralleling the local computing task lines and the cloud computing task lines of one data analysis operation requirement.
In a possible implementation manner, the specific manner of splitting any data analysis operation requirement into a local computing task line and one or more cloud computing task lines includes:
splitting the data analysis operation requirement into a locally calculated task line MkLAnd one or more cloud computing task lines MkC1,MkC2...MkCn...; wherein the local computing task line MkLCorresponding to an operation data amount of DkLOne or more cloud computing task lines MkC1,MkC2...MkCn.., the total corresponding operation data amount is DkC；
Wherein Dk＝DkL+(1-)DkC；
The splitting ratio of local calculation and cloud calculation is expressed, and is more than or equal to 0 and less than or equal to 1;
wherein the local computing task line MkLIs calculated with a time delay of tkLCloud computing task line MkC1,MkC2...MkCn.., the calculated time delays are tkC1,tkC2...tkCn...；
The way of calculating the split ratio is: when the constraint condition is satisfied, i.e. t is more than or equal to 0kL,tkC1,tkC2...tkCn...≤tmaxLet max (t)kL,tkC1,tkC2...tkCn) At a minimum; wherein t ismaxIs the maximum allowed time delay;
on the basis of the determination, determining a task line M of cloud computing splitting according to the number of available cloud serverskC1,MkC2...MkCn...
In a possible implementation manner, the specific manner of determining the data computation amount and the computation thread number of the cloud server, and further determining the number of task threads that can be assumed by the cloud server and the expected completion time of the task threads includes:
defining the cost of local computation as
Where n is the number of the local terminal,
represents the total overhead of the nth local terminal,
a weight of the temporal overhead is represented,
a time overhead is represented by the time of day,
wherein
Is the number of calculation threads that can be invoked by the local terminal in local unit time, dnThe calculation thread number started by the task line of the local terminal to successfully finish the calculation is shown,
representing the weight of the overhead of the computational resource,
is the overhead of the computational resources and is,
wherein
Is the computational resource overhead for each computational thread,
computational resource overhead in units of data volume;
the cost of the cloud server for carrying out cloud computing on the computing task uploaded by the nth local terminal is
Wherein the content of the first and second substances,
representing the time to upload a computing task from the local terminal to the cloud server,
representing the computation time of the cloud server,
wherein C is the calculation thread number, pi, which can be used by the cloud server in unit timenRepresenting a profit value obtained by the cloud server after completing a calculation task;
the comprehensive utility of the cloud server is
Wherein R represents the comprehensive utility of the cloud server, K is a task line corresponding to the cloud server, and K task lines are calculated by the cloud server; n is a radical ofkFor the number of successfully completed calculations, π, within the utility statistics window in the kth task linekThe profit value pr obtained by the cloud server after the kth task line completes calculation successfully every timebCost, pr, representing unit data computation of cloud serverdCost of the unit calculation thread number of the cloud server, bkRepresenting the amount of data computation incurred by each successful completion of the computation by the kth task line, dkRepresenting the number of calculation threads started by the kth task line for successfully completing the calculation each time;
further, local terminal overhead is set
Overhead greater than cloud server
Taking the maximum comprehensive utility R of the cloud server as a target for constraint conditions, and solving bk，dkAnd pik；
Reference bk，dkAnd pikAnd distributing the calculation tasks to the cloud server, namely acquiring the data calculation amount and the calculation thread number born by the cloud server for the kth task line according to the maximization of the comprehensive utility R of the cloud server, and determining the calculation born by the cloud server according to the data calculation amount and the calculation thread number.
In a possible implementation manner, the specific manner of splitting any data analysis operation requirement into a local computation task line and one or more cloud computation task lines, determining the data computation amount and the computation thread number of the cloud server, and further determining the number of task lines that the cloud server can undertake and the expected completion time of the task lines includes:
after the data calculation amount and the calculation thread number borne by the cloud server are determined for the cloud server, the cloud task lines split according to each data analysis calculation requirement are determined, the task lines are determined to be allocated to the queuing period faced by the cloud server according to the data calculation amount and the calculation thread number borne by the cloud server, and then the cloud task lines are allocated to the appropriate cloud server under the condition that the local calculation task lines and the cloud calculation task lines are maximally parallel.
As a second aspect of the invention, the invention also discloses a system for operating the intelligent city based on the multi-source task line, which comprises:
the system comprises a cloud server, a local terminal, a task scheduling controller and a coordinator.
The task scheduling controller is central computing equipment of the whole network, the coordinator is connected below the task scheduling controller, the coordinator distributes tasks to a cloud computing task line and a local computing task line, the cloud computing task line is provided with the cloud server, and the local computing task line is provided with the local terminal. The local terminal undertakes calculation of local tasks, and the cloud server undertakes calculation of cloud tasks. And the cloud computing task line and the local computing task line perform edge computing and transmit the computed data to the task scheduling controller.
The task scheduling controller is used for receiving data calculated by the local terminal and the cloud server;
the coordinator is used for splitting any data analysis operation requirement into a local computing task line and one or more cloud computing task lines; determining the data calculation amount and the calculation thread number of the cloud server according to the principle of optimal utility, and further determining the number of task threads capable of being born by the cloud server and the expected completion time of the task threads; and distributing the split cloud computing task lines to appropriate cloud servers in a mode of maximizing parallelism of the local computing task lines and the cloud computing task lines of one data analysis operation requirement.
The coordinator comprises a task line splitting module which is used forSplitting the data analysis operation requirement into a locally calculated task line MkLAnd one or more cloud computing task lines MkC1,MkC2...MkCn...; wherein the local computing task line MkLCorresponding to an operation data amount of DkLOne or more cloud computing task lines MkC1,MkC2...MkCn.., the total corresponding operation data amount is DkC；
Wherein Dk＝DkL+(1-)DkC；
The splitting ratio of local calculation and cloud calculation is expressed, and is more than or equal to 0 and less than or equal to 1;
wherein the local computing task line MkLIs calculated with a time delay of tkLCloud computing task line MkC1,MkC2...MkCn.., the calculated time delays are tkC1,tkC2...tkCn...；
The way of calculating the split ratio is: when the constraint condition is satisfied, i.e. t is more than or equal to 0kL,tkC1,tkC2...tkCn...≤tmaxLet max (t)kL,tkC1,tkC2...tkCn) At a minimum; wherein t ismaxIs the maximum allowed time delay;
on the basis of the determination, determining a task line M of cloud computing splitting according to the number of available cloud serverskC1,MkC2...MkCn...
The coordinator comprises a cloud server calculation amount evaluation module, and the cost of local calculation is defined as
Where n is the number of the local terminal,
represents the total overhead of the nth local terminal,
a weight of the temporal overhead is represented,
a time overhead is represented by the time of day,
wherein
Is the number of calculation threads that can be invoked by the local terminal in local unit time, dnCan be expressed in units of the calculated amount of a single thread per unit time,
representing the weight of the overhead of the computational resource,
is the overhead of the computational resources and is,
wherein
Is the computational resource overhead for each computational thread,
computational resource overhead in units of data size, bnIs the amount of data processed by the local terminal;
the cost of the cloud server for carrying out cloud computing on the computing task uploaded by the nth local terminal is
Wherein the content of the first and second substances,
representing the time to upload a computing task from the local terminal to the cloud server,
representing the computation time of the cloud server,
wherein C is the calculation thread number, pi, which can be used by the cloud server in unit timenRepresenting a profit value obtained by the cloud server after completing a calculation task;
the comprehensive utility of the cloud server is
Wherein R represents the comprehensive utility of the cloud server, K is a task line corresponding to the cloud server, and K task lines are calculated by the cloud server; n is a radical ofkFor the number of successfully completed calculations, π, within the utility statistics window in the kth task linekThe profit value pr obtained by the cloud server after the kth task line completes calculation successfully every timebCost, pr, representing unit data computation of cloud serverdCost of the unit calculation thread number of the cloud server, bkRepresenting the amount of data computation incurred by each successful completion of the computation by the kth task line, dkRepresents the number of calculation threads, g, opened for each successful completion of the calculation of the kth task linekRepresenting the cost of the cloud server after the kth task line successfully completes the calculation each time;
further, local terminal overhead is set
Overhead greater than cloud server
Taking the maximum comprehensive utility R of the cloud server as a target for constraint conditions, and solving bk，dkAnd pik；
Reference bk，dkAnd pikAnd distributing the computing task to the cloud server, namely obtaining the data computing amount and the computing thread number born by the cloud server for the kth task line according to the maximization of the comprehensive utility R of the cloud server, and determining the cloud service according to the data computing amount and the computing thread numberThe calculation undertaken by the device.
The coordinator also comprises a cloud task line distribution module, and the cloud task line distribution module determines the data calculation amount and the calculation thread number born by the cloud server for the cloud server; and determining the queuing period for allocating the task lines to the cloud server according to the data calculation amount and the calculation thread number born by the cloud server for each cloud task line split according to the data analysis calculation requirements, and allocating the cloud task lines to the appropriate cloud server under the condition that the local calculation task lines and the cloud calculation task lines are maximally parallel.
(III) advantageous effects
The invention discloses a method and a system for intelligent city operation based on a multi-source task line, which have the following beneficial effects:
1. improving the response speed of the local facility: the multi-source task line can be calculated and processed at the same time without being clamped on one line.
2. The data analysis operation realizes the optimized balance between the local calculation and the cloud calculation, and the operation tasks are disassembled and distributed at the local and the cloud, so that the overall benefit is optimized.
3. Effective dynamic management is realized between local computing and cloud computing, the whole smoothness of the task line is kept, and excessive stagnation interruption is avoided.
Drawings
The embodiments described below with reference to the drawings are exemplary and intended to be used for explaining and illustrating the present invention and should not be construed as limiting the scope of the present invention.
FIG. 1 is a schematic structural diagram of a system for intelligent city operation based on a multi-source task line, which is disclosed by the invention;
FIG. 2 is a flow chart of a method for operating an intelligent city based on a multi-source task line, which is disclosed by the invention.
Detailed Description
In order to make the implementation objects, technical solutions and advantages of the present invention clearer, the technical solutions in the embodiments of the present invention will be described in more detail below with reference to the accompanying drawings in the embodiments of the present invention.
It should be noted that: in the drawings, the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described are some embodiments of the present invention, not all embodiments, and features in embodiments and embodiments in the present application may be combined with each other without conflict. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
With the development of science and technology, the artificial intelligent internet of things is increasingly integrated into the daily life of people. The smart city is a specific application case. The operation of the intelligent city cannot be separated from the analysis and operation of data, wherein most of the data analysis and operation have larger operation amount and data processing amount, along with the development of intellectualization to the depth direction, the operation amount and the magnitude of the data amount of the data analysis and operation are remarkably increased, and the processing capability, the data storage capability, the battery standby and other aspects of local calculation of a single intelligent facility are gradually insufficient. For example: temperature and humidity sensors, air quality sensors and optical sensors can collect a large amount of receipts of cities, and the processing capability, data storage capability, battery standby and other aspects of local calculation which only depend on single intelligent facilities cannot meet the requirements of the stability and real-time performance of the whole internet of things.
The intelligent facility at the front end of the cloud computing is only responsible for basic data acquisition, result feedback, action execution and the like, data analysis and operation are moved to a server at the cloud end, and the server is executed by utilizing the powerful computing capability. However, there are thousands of sensors in a smart city, which collect various data in real time and transmit the data to the central computing module. Under the scale and data magnitude of an intelligent city, cloud computing can increase intermediate links, so that the communication load is increased, the overhead of data operation is increased, and meanwhile, the response speed of an intelligent facility is reduced.
Therefore, for the operation of the smart city, the data analysis operation needs to realize the optimized balance between the local calculation and the cloud calculation, and the operation tasks are disassembled and distributed locally and at the cloud, so that the overall benefit is optimized.
In addition, the coverage space and the network scale of the Internet of things of the intelligent city are large, so that the sources of data analysis and operation tasks are numerous, and the parallel characteristic of a multi-source task line is presented. How to realize effective dynamic management between local calculation and cloud calculation for multi-source and concurrent task lines related to data analysis and calculation, keep the overall smoothness of the task lines, avoid excessive stagnation and interruption and solve the problems.
In order to process data collected by various types of sensors and meet interaction of bottom-layer interaction equipment, the intelligent Internet of things continuously generates calculation tasks to be executed. The invention adopts the framework of a multi-source task line, and a task is calculated to generate a task request; the task scheduling controller distributes tasks to a cloud task line and a local task line according to the requirements of the task request on computing resources, real-time performance and network transmission stability, the various sensors and various bottom layer interaction devices transmit the acquired data to a cloud server and a local terminal, and the cloud server and the local terminal meet the task request and execute related task computation.
In order to meet the difference requirements of task requests of the intelligent internet of things on computing resources, real-time performance and network transmission stability, an intelligent city operation method based on a multi-source task line is designed, the method is combined with a multi-task line computing task scheduling method of a task scheduling controller technology, and a network structure diagram is shown in fig. 1. The central computing equipment of the whole network is a task scheduling controller, a coordinator is connected below the task scheduling controller, the coordinator distributes tasks to a cloud task line and a local task line, a cloud server is arranged on the cloud task line, and a local terminal is arranged on the local task line. The local terminal undertakes the calculation of local tasks, and the cloud server undertakes the calculation of cloud tasks. And the cloud task line and the local task line perform edge calculation and transmit the calculated data to the task scheduling controller. The coordinator allocates the tasks to the cloud task lines and the local task lines through the following rules. Data analysis and operation need to realize optimized balance between local calculation and cloud calculation, and operation tasks are disassembled and distributed locally and at the cloud, so that the overall benefit is optimized.
As shown in fig. 2, the method for operating an intelligent city based on a multi-source task line in this embodiment mainly includes the following steps:
300. for any data analysis and operation requirement in the operation of the intelligent city, splitting the data analysis and operation requirement into a task line for local computation and one or more task lines for cloud computation;
400. determining the data calculation amount and the calculation thread number of the cloud server according to the principle of optimal utility, and further determining the number of task threads capable of being born by the cloud server and the expected completion time of the task threads;
500. and distributing the split task lines to the appropriate cloud servers in a mode of maximally paralleling the local computing task line and the cloud computing task line of one data analysis operation requirement.
300. For any data analysis and operation requirement in the operation of the intelligent city, splitting the data analysis and operation requirement into a task line for local computation and one or more task lines for cloud computation;
wherein one data analysis operation requirement is expressed as Mk＝{Dk,Ck,tmaxIn which D iskIs the amount of operation data of the demand, CkIs the number of computational threads required by the demand, tmaxIs the maximum completion time for the demand.
Splitting the data analysis operation requirement into a locally calculated task line MkLAnd one or more cloud computing task lines MkC1,MkC2...MkCn...; wherein the local computing task line MkLCorresponding to an operation data amount of DkLOne or more cloud computing task lines MkC1,MkC2...MkCn.., the total corresponding operation data amount is DkC；
Wherein Dk＝DkL+(1-)DkC；
The splitting ratio of local calculation and cloud calculation is expressed, and is more than or equal to 0 and less than or equal to 1;
wherein the local computing task line MkLIs calculated with a time delay of tkLCloud computing task line MkC1,MkC2...MkCn.., the calculated time delays are tkC1,tkC2...tkCn...；
The way of calculating the split ratio is: when the constraint condition is satisfied, i.e. t is more than or equal to 0kL,tkC1,tkC2...tkCn...≤tmaxLet max (t)kL,tkC1,tkC2...tkCn) At a minimum; wherein t ismaxIs the maximum allowed time delay;
on the basis of the determination, determining a task line M of cloud computing splitting according to the number of available cloud serverskC1,MkC2...MkCn...
400. Determining the data calculation amount and the calculation thread number of the cloud server according to the principle of optimal utility, and further determining the number of task threads capable of being born by the cloud server and the expected completion time of the task threads;
the invention defines the cost of local computation as
Where n is the number of the local terminal,
represents the total overhead of the nth local terminal,
a weight of the temporal overhead is represented,
a time overhead is represented by the time of day,
wherein
Is the calculation thread number which can be started in unit time at the local terminal,
representing the weight of the overhead of the computational resource,
is the overhead of the computational resources and is,
wherein
Is the computational resource overhead for each computational thread,
computational resource overhead in units of data volume;
the cost of the cloud server for carrying out cloud computing on the nth local terminal is
Wherein the content of the first and second substances,
representing the time to upload a computing task from the local terminal to the cloud server,
representing the computation time of the cloud server,
wherein C is the calculation thread number, pi, which can be used by the cloud server in unit timenRepresenting a profit value obtained by the cloud server after completing PoW calculation;
the comprehensive utility of the cloud server is
Wherein R represents the comprehensive utility of the cloud serverK is a task line corresponding to the cloud server, and K task lines are calculated by the cloud server; n is a radical ofkThe number of local terminals, pi, successfully completing the calculation in the utility statistical window in the kth task linekThe profit value pr obtained by the cloud server after the kth task line completes calculation successfully every timebCost, pr, representing unit data computation of cloud serverdCost of the unit calculation thread number of the cloud server, bkRepresenting the amount of data computation incurred by each successful completion of the computation by the kth task line, dkIndicating the number of calculation threads opened for each successful completion of the calculation of the kth task line.
Further, local terminal overhead is set
Overhead greater than cloud server
Taking the maximum comprehensive utility R of the cloud server as a target for constraint conditions, and solving bk，dkAnd pik。
Reference bk，dkAnd pikAnd distributing the PoW calculation to the cloud server, namely acquiring the data calculation amount and the calculation thread number born by the cloud server for the kth task line according to the maximization of the comprehensive utility R of the cloud server, and determining the calculation born by the cloud server according to the data calculation amount and the calculation thread number.
500. And distributing the split task lines to the appropriate cloud servers in a mode of maximally paralleling the local computing task line and the cloud computing task line of one data analysis operation requirement.
In the steps, the data calculation amount and the calculation thread number born by the cloud server are determined for the cloud server; and determining a queuing period for allocating the task lines to the cloud server according to the data calculation amount and the calculation thread number born by the cloud server for each cloud computing task line split according to the data analysis calculation requirements, and allocating the cloud computing task lines to the appropriate cloud server under the condition that the local computing task lines and the cloud computing task lines are maximally parallel.
The above description is only for the specific embodiments of the present invention, but the scope of the present invention is not limited thereto, and any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope of the present invention are included in the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (8)
1. A method for running an intelligent city based on a multi-source task line is characterized by comprising the following steps:
for any data analysis and operation requirement in the operation of the intelligent city, splitting the data analysis and operation requirement into a local calculation task line and one or more cloud calculation task lines;
determining the data calculation amount and the calculation thread number of the cloud server according to the principle of optimal utility, and further determining the number of task threads capable of being born by the cloud server and the expected completion time of the task threads;
and distributing the split cloud computing task lines to appropriate cloud servers in a mode of maximally paralleling the local computing task lines and the cloud computing task lines of one data analysis operation requirement.
2. The method of claim 1, wherein any data analysis computation requirement is split into a local computation task line, and the specific way of one or more cloud computation task lines is:
splitting the data analysis operation requirement into a local computation task line MkLAnd one or more cloud computing task lines MkC1,MkC2...MkCn...; wherein the local computing task line MkLCorresponding to an operation data amount of DkLOne or more cloud computing task lines MkC1,MkC2...MkCn.., the total corresponding operation data amount is DkC；
Wherein Dk＝DkL+(1-)DkC；
The splitting ratio of local calculation and cloud calculation is expressed, and is more than or equal to 0 and less than or equal to 1;
wherein the local computing task line MkLIs calculated with a time delay of tkLCloud computing task line MkC1,MkC2...MkCn.., the calculated time delays are tkC1,tkC2...tkCn...；
The way of calculating the split ratio is: when the constraint condition is satisfied, i.e. t is more than or equal to 0kL,tkC1,tkC2...tkCn...≤tmaxLet max (t)kL,tkC1,tkC2...tkCn) At a minimum; wherein t ismaxIs the maximum allowed time delay;
on the basis of the determination, determining a task line M of cloud computing splitting according to the number of available cloud serverskC1,MkC2...MkCn...
3. The method of claim 1, wherein the determining the data computation amount and the computation thread count of the cloud server, and further determining the number of the task threads that can be assumed by the cloud server and the expected completion time of the task threads is performed by:
defining the cost of local computation as
Where n is the number of the local terminal,
represents the total overhead of the nth local terminal,
a weight of the temporal overhead is represented,
a time overhead is represented by the time of day,
wherein
Is the number of calculation threads that can be invoked by the local terminal in local unit time, dnCan be expressed in units of the calculated amount of a single thread per unit time,
representing the weight of the overhead of the computational resource,
is the overhead of the computational resources and is,
wherein
Is the computational resource overhead for each computational thread,
computational resource overhead in units of data size, bnIs the amount of data processed by the local terminal;
the cost of the cloud server for carrying out cloud computing on the computing task uploaded by the nth local terminal is
Wherein the content of the first and second substances,
representing the time to upload a computing task from the local terminal to the cloud server,
representing cloud serversThe time is calculated and the time is calculated,
wherein C is the calculation thread number, pi, which can be used by the cloud server in unit timenRepresenting a profit value obtained by the cloud server after completing a calculation task;
the comprehensive utility of the cloud server is
Wherein R represents the comprehensive utility of the cloud server, K is a task line corresponding to the cloud server, and K task lines are calculated by the cloud server; n is a radical ofkFor the number of successfully completed calculations, π, within the utility statistics window in the kth task linekThe profit value pr obtained by the cloud server after the kth task line completes calculation successfully every timebCost, pr, representing unit data computation of cloud serverdCost of the unit calculation thread number of the cloud server, bkRepresenting the amount of data computation incurred by each successful completion of the computation by the kth task line, dkRepresents the number of calculation threads, g, opened for each successful completion of the calculation of the kth task linekRepresenting the cost of the cloud server after the kth task line successfully completes the calculation each time;
further, local terminal overhead is set
Overhead greater than cloud server
Taking the maximum comprehensive utility R of the cloud server as a target for constraint conditions, and solving bk，dkAnd pik；
Reference bk，dkAnd pikAnd distributing the computing task to the cloud server, namely obtaining the k-th task line supported by the cloud server according to the maximization of the comprehensive utility R of the cloud serverAnd determining the calculation born by the cloud server according to the data calculation amount and the calculation thread number.
4. The method according to any one of claims 1 to 3,
after the data calculation amount and the calculation thread number borne by the cloud server are determined for the cloud server, the cloud task lines split according to each data analysis calculation requirement are determined, the task lines are determined to be allocated to the queuing period faced by the cloud server according to the data calculation amount and the calculation thread number borne by the cloud server, and then the cloud task lines are allocated to the appropriate cloud server under the condition that the local calculation task lines and the cloud calculation task lines are maximally parallel.
5. A system for intelligent city operation based on multi-source task lines is characterized by comprising:
a cloud server for computing the computing task of the cloud task line
The local terminal is used for calculating the calculation task of the local task line;
the task scheduling controller is used for receiving data calculated by the local terminal and the cloud server;
the coordinator is used for splitting any data analysis operation requirement into a local computing task line and one or more cloud computing task lines; determining the data calculation amount and the calculation thread number of the cloud server according to the principle of optimal utility, and further determining the number of task threads capable of being born by the cloud server and the expected completion time of the task threads; and distributing the split cloud computing task lines to appropriate cloud servers in a mode of maximizing parallelism of the local computing task lines and the cloud computing task lines of one data analysis operation requirement.
6. The system of claim 5,
the coordinator comprises a task line splitting module which is used for splitting the task lineThe block is used for splitting the data analysis operation requirement into a locally calculated task line MkLAnd one or more cloud computing task lines MkC1,MkC2...MkCn...; wherein the local computing task line MkLCorresponding to an operation data amount of DkLOne or more cloud computing task lines MkC1,MkC2...MkCn.., the total corresponding operation data amount is DkC；
Wherein Dk＝DkL+(1-)DkC；
The splitting ratio of local calculation and cloud calculation is expressed, and is more than or equal to 0 and less than or equal to 1;
wherein the local computing task line MkLIs calculated with a time delay of tkLCloud computing task line MkC1,MkC2...MkCn.., the calculated time delays are tkC1,tkC2...tkCn...；
The way of calculating the split ratio is: when the constraint condition is satisfied, i.e. t is more than or equal to 0kL,tkC1,tkC2...tkCn...≤tmaxLet max (t)kL,tkC1,tkC2...tkCn) At a minimum; wherein t ismaxIs the maximum allowed time delay;
on the basis of the determination, determining a task line M of cloud computing splitting according to the number of available cloud serverskC1,MkC2...MkCn...
7. The system of claim 5,
the coordinator comprises a cloud server calculation amount evaluation module, and the cost of local calculation is defined as
Where n is the number of the local terminal,
represents the total overhead of the nth local terminal,
a weight of the temporal overhead is represented,
a time overhead is represented by the time of day,
wherein
Is the number of calculation threads that can be invoked by the local terminal in local unit time, dnCan be expressed in units of the calculated amount of a single thread per unit time,
representing the weight of the overhead of the computational resource,
is the overhead of the computational resources and is,
wherein
Is the computational resource overhead for each computational thread,
computational resource overhead in units of data size, bnIs the amount of data processed by the local terminal;
the cost of the cloud server for carrying out cloud computing on the computing task uploaded by the nth local terminal is
Wherein the content of the first and second substances,
representing the time to upload a computing task from the local terminal to the cloud server,
representing the computation time of the cloud server,
wherein C is the calculation thread number, pi, which can be used by the cloud server in unit timenRepresenting a profit value obtained by the cloud server after completing a calculation task;
the comprehensive utility of the cloud server is
Wherein R represents the comprehensive utility of the cloud server, K is a task line corresponding to the cloud server, and K task lines are calculated by the cloud server; n is a radical ofkFor the number of successfully completed calculations, π, within the utility statistics window in the kth task linekThe profit value pr obtained by the cloud server after the kth task line completes calculation successfully every timebCost, pr, representing unit data computation of cloud serverdCost of the unit calculation thread number of the cloud server, bkRepresenting the amount of data computation incurred by each successful completion of the computation by the kth task line, dkRepresents the number of calculation threads, g, opened for each successful completion of the calculation of the kth task linekRepresenting the cost of the cloud server after the kth task line successfully completes the calculation each time;
further, local terminal overhead is set
Overhead greater than cloud server
Taking the maximum comprehensive utility R of the cloud server as a target for constraint conditions,solving for bk，dkAnd pik；
Reference bk，dkAnd pikAnd distributing the calculation tasks to the cloud server, namely acquiring the data calculation amount and the calculation thread number born by the cloud server for the kth task line according to the maximization of the comprehensive utility R of the cloud server, and determining the calculation born by the cloud server according to the data calculation amount and the calculation thread number.
8. The system of claim 5,
the coordinator also comprises a cloud task line distribution module, and the cloud task line distribution module determines the data calculation amount and the calculation thread number born by the cloud server for the cloud server; and determining the queuing period for allocating the task lines to the cloud server according to the data calculation amount and the calculation thread number born by the cloud server for each cloud task line split according to the data analysis calculation requirements, and allocating the cloud task lines to the appropriate cloud server under the condition that the local calculation task lines and the cloud calculation task lines are maximally parallel.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010225913.9A CN111539863B (en) | 2020-03-26 | 2020-03-26 | Intelligent city operation method and system based on multi-source task line |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010225913.9A CN111539863B (en) | 2020-03-26 | 2020-03-26 | Intelligent city operation method and system based on multi-source task line |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111539863A true CN111539863A (en) | 2020-08-14 |
| CN111539863B CN111539863B (en) | 2021-03-19 |
Family
ID=71974823
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010225913.9A Active CN111539863B (en) | 2020-03-26 | 2020-03-26 | Intelligent city operation method and system based on multi-source task line |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111539863B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112104996A (en) * | 2020-09-08 | 2020-12-18 | 电子科技大学 | Low-cost crowd sensing calculation method applied to shared traffic system |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2017176542A1 (en) * | 2016-04-08 | 2017-10-12 | Alcatel-Lucent Usa Inc. | Optimal dynamic cloud network control |
| CN108170523A (en) * | 2017-12-28 | 2018-06-15 | 合肥工业大学 | A kind of Random Task sequence dispatching method of mobile cloud computing |
| CN108540406A (en) * | 2018-07-13 | 2018-09-14 | 大连理工大学 | A kind of network discharging method based on mixing cloud computing |
| CN108600299A (en) * | 2018-03-02 | 2018-09-28 | 中国科学院上海微系统与信息技术研究所 | Calculating task discharging method and system between distributed multi-user |
| CN109474681A (en) * | 2018-11-05 | 2019-03-15 | 安徽大学 | Resource allocation methods, system and the server system of mobile edge calculations server |
| CN109669768A (en) * | 2018-12-11 | 2019-04-23 | 北京工业大学 | A kind of resource allocation and method for scheduling task towards side cloud combination framework |
| CN109819032A (en) * | 2019-01-24 | 2019-05-28 | 中山大学 | A kind of base station selected cloud robot task distribution method with computation migration of joint consideration |
| CN110418416A (en) * | 2019-07-26 | 2019-11-05 | 东南大学 | Resource allocation methods based on multiple agent intensified learning in mobile edge calculations system |
-
2020
- 2020-03-26 CN CN202010225913.9A patent/CN111539863B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2017176542A1 (en) * | 2016-04-08 | 2017-10-12 | Alcatel-Lucent Usa Inc. | Optimal dynamic cloud network control |
| CN108170523A (en) * | 2017-12-28 | 2018-06-15 | 合肥工业大学 | A kind of Random Task sequence dispatching method of mobile cloud computing |
| CN108600299A (en) * | 2018-03-02 | 2018-09-28 | 中国科学院上海微系统与信息技术研究所 | Calculating task discharging method and system between distributed multi-user |
| CN108540406A (en) * | 2018-07-13 | 2018-09-14 | 大连理工大学 | A kind of network discharging method based on mixing cloud computing |
| CN109474681A (en) * | 2018-11-05 | 2019-03-15 | 安徽大学 | Resource allocation methods, system and the server system of mobile edge calculations server |
| CN109669768A (en) * | 2018-12-11 | 2019-04-23 | 北京工业大学 | A kind of resource allocation and method for scheduling task towards side cloud combination framework |
| CN109819032A (en) * | 2019-01-24 | 2019-05-28 | 中山大学 | A kind of base station selected cloud robot task distribution method with computation migration of joint consideration |
| CN110418416A (en) * | 2019-07-26 | 2019-11-05 | 东南大学 | Resource allocation methods based on multiple agent intensified learning in mobile edge calculations system |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112104996A (en) * | 2020-09-08 | 2020-12-18 | 电子科技大学 | Low-cost crowd sensing calculation method applied to shared traffic system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111539863B (en) | 2021-03-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Abd Elaziz et al. | Advanced optimization technique for scheduling IoT tasks in cloud-fog computing environments | |
| CN110096349B (en) | Job scheduling method based on cluster node load state prediction | |
| CN109324875B (en) | Data center server power consumption management and optimization method based on reinforcement learning | |
| US20080263559A1 (en) | Method and apparatus for utility-based dynamic resource allocation in a distributed computing system | |
| CN102724103B (en) | Proxy server, hierarchical network system and distributed workload management method | |
| CN110389820B (en) | Private cloud task scheduling method for resource prediction based on v-TGRU model | |
| CN104881325A (en) | Resource scheduling method and resource scheduling system | |
| CN104239144A (en) | Multilevel distributed task processing system | |
| CN103338228A (en) | Cloud calculating load balancing scheduling algorithm based on double-weighted least-connection algorithm | |
| CN103401939A (en) | Load balancing method adopting mixing scheduling strategy | |
| CN114816721B (en) | Multitask optimization scheduling method and system based on edge calculation | |
| CN104657205A (en) | Virtualization-based video content analyzing method and system | |
| CN113553160A (en) | Task scheduling method and system for edge computing node of artificial intelligence Internet of things | |
| CN108132840A (en) | Resource regulating method and device in a kind of distributed system | |
| CN111539863B (en) | Intelligent city operation method and system based on multi-source task line | |
| CN114064261A (en) | Multi-dimensional heterogeneous resource quantification method and device based on industrial edge computing system | |
| Falah et al. | Comparison of cloud computing providers for development of big data and internet of things application | |
| CN117472587A (en) | Resource scheduling system of AI intelligent computation center | |
| CN111131447A (en) | Load balancing method based on intermediate node task allocation | |
| More et al. | Energy-aware VM migration using dragonfly–crow optimization and support vector regression model in Cloud | |
| CN117349026B (en) | Distributed computing power scheduling system for AIGC model training | |
| CN116402318B (en) | Multi-stage computing power resource distribution method and device for power distribution network and network architecture | |
| CN113656150A (en) | Deep learning computing power virtualization system | |
| CN111160283B (en) | Data access method, device, equipment and medium | |
| CN116048821B (en) | High-utilization AI server and resource allocation method thereof |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20210210 Address after: 400010 no.50-6, 19 dapingzheng street, Yuzhong District, Chongqing Applicant after: Light control tesilian (Chongqing) Information Technology Co.,Ltd. Address before: 401329 no.2-1, building 6, No.39 Xinggu Road, Jiulongpo District, Chongqing Applicant before: CHONGQING TERMINUS TECHNOLOGIES Co.,Ltd. |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |