CN116633932A - Dynamic scheduling system for cloud computing resource pool - Google Patents
Dynamic scheduling system for cloud computing resource pool Download PDFInfo
- Publication number
- CN116633932A CN116633932A CN202310897504.7A CN202310897504A CN116633932A CN 116633932 A CN116633932 A CN 116633932A CN 202310897504 A CN202310897504 A CN 202310897504A CN 116633932 A CN116633932 A CN 116633932A
- Authority
- CN
- China
- Prior art keywords
- subdomain
- scheduling
- task
- cloud computing
- sub
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000010276 construction Methods 0.000 claims abstract description 13
- 238000000034 method Methods 0.000 claims abstract description 10
- 238000004364 calculation method Methods 0.000 claims description 51
- 238000009826 distribution Methods 0.000 claims description 46
- 238000004891 communication Methods 0.000 claims description 30
- 238000012360 testing method Methods 0.000 claims description 28
- 230000004044 response Effects 0.000 claims description 24
- 238000000354 decomposition reaction Methods 0.000 claims description 12
- 238000010586 diagram Methods 0.000 claims description 10
- 238000012544 monitoring process Methods 0.000 claims description 10
- 238000012545 processing Methods 0.000 claims description 7
- 238000013475 authorization Methods 0.000 claims description 6
- 238000012423 maintenance Methods 0.000 claims description 6
- 230000002776 aggregation Effects 0.000 claims description 4
- 238000004220 aggregation Methods 0.000 claims description 4
- 238000004458 analytical method Methods 0.000 claims description 3
- 238000013507 mapping Methods 0.000 claims description 3
- 239000011159 matrix material Substances 0.000 claims description 3
- 238000012163 sequencing technique Methods 0.000 claims description 3
- 238000003860 storage Methods 0.000 claims description 3
- 238000005192 partition Methods 0.000 claims 1
- 238000000638 solvent extraction Methods 0.000 claims 1
- 238000005516 engineering process Methods 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 238000010420 art technique Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000009877 rendering Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000007480 spreading Effects 0.000 description 1
- 238000003892 spreading Methods 0.000 description 1
- 108020001568 subdomains Proteins 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/08—Configuration management of networks or network elements
- H04L41/0803—Configuration setting
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/30—Decision processes by autonomous network management units using voting and bidding
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
- Multi Processors (AREA)
Abstract
The application provides a dynamic scheduling system for a cloud computing resource pool, and relates to the technical field of resource scheduling. The application realizes the automatic construction of the dynamic dispatching system through the platform management unit, and can carry out self-adaptive subdomain division, node election and deployment according to the condition of the existing resource pool, thereby providing dispatching cooperation support for sporadic cloud computing servers; when the system is constructed, the used platform management contract has the advantages of flexible, quick and convenient deployment, and can be automatically executed after being deployed on the cloud computing platform without supervision; through the system, a user can obtain a cloud computing resource pool with sufficient computing power and low cost, and all sporadic cloud computing servers added into a platform can be comprehensively combined to obtain a resource pool with strong computing power, so that the full utilization of computing power is realized; therefore, the method has practical and popularization values.
Description
Technical Field
The application relates to the technical field of resource scheduling, in particular to a cloud computing resource pool dynamic scheduling system.
Background
With the development of technology and age, more and more application scenes provide service support through cloud computing, wherein cloud computing (cloud computing) is one of distributed computing, and is characterized in that huge data computing processing programs are decomposed into countless small programs through a network 'cloud', and then the small programs are processed and analyzed through a system formed by a plurality of servers to obtain results and returned to users.
However, the cloud computing service at the present stage is often provided by a large network service provider, and the special cloud computing device is maintained through the special IDC machine room to provide corresponding cloud, so that the cloud computing service has the advantages of high speed and stable service, but also has the problems of high operation and maintenance cost and high service cost. For some services (such as video transcoding, etc.) which are heavy and do not require strict quality of service, but are sensitive to cost requirements, users are difficult to pay high service costs.
Today, there are many sporadically arranged cloud computing servers in our life, which are not fully utilized; the reason is that the service processing types, service computing power, online time and free time of the servers are all in dynamic state, so that the operation coordination is difficult to be carried out through a fixed cloud computing resource pool scheduling system, and the problems of difficult maintenance, complex deployment and large network load are caused.
Therefore, it is desirable to provide a dynamic scheduling system for cloud computing resource pools to solve the above technical problems.
Disclosure of Invention
In order to solve the technical problems, the cloud computing resource pool dynamic scheduling system provided by the application is deployed on a cloud computing platform, wherein the cloud computing platform comprises a plurality of cloud computing servers and is divided into a global scheduling node, a subdomain scheduling node and computing execution equipment according to purposes; the system comprises a platform management unit, a global scheduling unit, a subdomain scheduling unit and a response execution unit;
the platform management units are distributed and deployed on the cloud computing platform in an intelligent contract form, automatically execute platform management contracts and complete the construction of a dynamic scheduling system of the resource pool; wherein the platform management contracts include subdomain division contracts, node election contracts and authorization construction contracts; when the platform management contract is automatically executed, firstly executing a subdomain division contract to finish subdomain division of the cloud computing platform; then executing node election is about, completing subdomain scheduling node election, and electing the whole domain scheduling node in each subdomain scheduling node, and the rest cloud computing servers are used as computing executing equipment; finally executing an authorization construction contract, authorizing and deploying the global scheduling unit to the global scheduling node, authorizing and deploying the subzone scheduling unit to the subzone scheduling node, and deploying the response execution unit to the computing execution equipment;
the global scheduling node is used for receiving the calculation tasks, performing task downloading and data summarizing and comprises a service response module, a task scheduling module and a list management module; the subzone scheduling unit is used for dynamic scheduling of subzone computing tasks and comprises a task receiving module, a task decomposing module, a load monitoring module and a task scheduling module; the response execution unit is used for sub-domain calculation task response execution and comprises an instruction response module, a task execution module and a state monitoring module.
As a further solution, the sub-domain division contract divides the sub-domain scope by:
s1: acquiring IP network addresses of all cloud computing servers, and carrying out IP network address analysis to obtain server position information;
s2: acquiring the calculation force load capacity of each cloud computing server, and matching with the position information of the server to obtain calculation force distribution information;
s3: mapping the calculation force distribution information point to point on the regional position map, and calculating the calculation force aggregation degree of the regional position points to obtain a calculation force distribution thermodynamic diagram;
s4: dividing a sub-domain main body in a computational power distribution thermodynamic diagram according to a preset dividing threshold value, and acquiring a region position point set which is not lower than the dividing threshold value to obtain the sub-domain main body;
s5: the cloud computing servers positioned on the regional position point set are attributed to the current sub-domain main body, and a sub-domain main body server set is obtained;
s6: selecting subzone scheduling nodes in the subzone main body server set to obtain subzone scheduling nodes of the current subzone main body;
s7: searching neighboring nodes of a cloud computing server which does not belong to, and searching a subdomain scheduling node set in a nearby area according to server position information;
s8: the current cloud computing server respectively carries out PING subdomain scheduling on IP network addresses of node sets and carries out node communication test;
s9: selecting a subzone scheduling node with an optimal communication test result, and adding the current cloud computing server as a subzone edge server;
s10: counting attributive sub-domain edge servers by each sub-domain scheduling node to obtain sub-domain edges and sub-domain edge server sets;
the operations S1 to S10 are repeated to complete all division of subfields by constituting the subfield range of the current subfield by the subfield body and the subfield edges.
As a further solution, the calculation force distribution information is a gaussian kernel density value with calculation force weighting, and a specific calculation formula is as follows:
；
wherein ,gaussian kernel density value->Representing the nuclear density estimation point, +.>Calculating the total amount of servers for the cloud, < >>Representing the current cloud computing server number, +.>Representing cloud computing server->Location point (s)/(s)>Representing a core Density estimation Point to cloud computing Server->Distance of location point, +.>Representing cloud computing server->Is a calculation force weight parameter of ∈10->Representing bandwidth parameters->Representing the standard deviation of the data.
As a further solution, the node election contract proceeds by:
acquiring a subdomain main body server set and an IP network address; wherein the sub-domain body servers collectively compriseNAn element;
performing inter-communication test and counting communication test results to obtain a communication scoring matrixS(i, j)，i, j∈NAnd is also provided withi≠j；
Counting intercommunication scores of each sub-domain main body serverSi，Si=ΣS(i, j)； wherein ,inumber the current scoring server, andi∈N；jis a sum numbered variable, andjrespectively take 1 toNMiddle removingiNumbering outside;
solving the highest intercommunication score in the main body of the current subdomain, wherein the highest intercommunication score passesargmax（Si) Acquiring; wherein,irespectively take 1 toN；
Setting a subzone main body server corresponding to the highest intercommunication score as a subzone scheduling node, and finishing subzone scheduling node election;
after the subzone scheduling node election is completed, electing the global scheduling node in all subzone scheduling nodes of the cloud computing platform;
wherein, global scheduling node election: and performing intercommunication test on all the subzone scheduling nodes, and selecting the subzone scheduling node with the highest intercommunication score in the cloud computing platform as the global scheduling node.
As a further solution, the list management module collects related data, gathers the related data into a corresponding data list, and performs storage and maintenance; the data list comprises a subdomain service list, a subdomain network list and a task allocation list; the subdomain service list is used for summarizing the service types processed by each subdomain and the corresponding service computing power, and the subdomain network list is used for summarizing the IP network address of each subdomain scheduling node and the subdomain network formed by each subdomain and the neighborhood thereof; the task allocation list is used for summarizing service allocation conditions.
As a further solution, the subdomain service list is constructed by: the subdomain scheduling node sends a resource sensing instruction to each computing executing device, and each computing executing device receives the resource sensing instruction and feeds back the processed service type and the corresponding service computing power through the instruction response module; the subdomain dispatching nodes count the service types and service computing power in the subdomain scope, and the subdomain dispatching nodes summarize the service types and the service computing power of each subdomain to obtain a subdomain service list.
As a further solution, the subdomain network list is constructed by: acquiring server position information of scheduling nodes of all subdomains, inquiring adjacent subdomains through the position information, and setting the adjacent subdomains as adjacent domains; after carrying out neighborhood query on all the subdomain scheduling nodes, obtaining a neighborhood set of each subdomain; and carrying out inter-communication test on the neighborhood set, counting communication test results, and carrying out good-bad sequencing on the communication test results, and summarizing the communication test results into a subdomain network list of the current subdomain.
As a still further solution, the information of the task allocation list record includes: the method comprises the steps of decomposing a computing task into task decomposition logic of a plurality of subtasks and task allocation information of each subtask; after the subtasks are processed, the subtask results are combined through task allocation information and task decomposition logic to obtain calculation task results.
As a further solution, when a user needs to execute a computing task, firstly, sending a service request to the service response module and providing task description information; the description information comprises user equipment IP, a request service type and a request service computing power;
the global dispatching node analyzes the user address information according to the user equipment IP, searches the latest subdomain meeting the processing capacity of the request service type in the subdomain service list, and provides the IP network address corresponding to the subdomain dispatching node for the user; issuing a task receiving command to a subdomain scheduling node through a task scheduling module;
the subdomain scheduling node obtains a task receiving command and receives a calculation task data packet sent by a user; and decomposing the computing task into a plurality of subtasks through a task decomposition module, and recording and uploading task decomposition logic.
As a further solution, the task scheduling module performs scheduling distribution on each subtask by:
the subdomain scheduling nodes acquire the load condition of the current subdomain through a load monitoring module to acquire the current available calculation load capacity;
and (3) performing local receiving judgment: if the current available calculation power load meets the calculation power requirement of the request service, locally receiving all subtasks and updating a task allocation list to finish task scheduling and distribution; otherwise, receiving a corresponding number of sub-tasks according to the current available calculation load capacity, partially updating a task allocation list, and performing next distribution operation;
and carrying out neighborhood receiving judgment: acquiring unscheduled distribution subtasks, and requesting each neighborhood to receive the subtasks according to the sequence of the subdomain network list; each neighborhood carries out local receiving judgment, receives the corresponding number of sub-tasks and partially updates a task allocation list; if all the subtasks are received, completing task scheduling and distribution; otherwise, carrying out the next distributing operation;
and (3) performing expansion receiving judgment: each neighborhood acquires unscheduled distribution subtasks and respectively carries out neighborhood receiving judgment; requesting a sub-domain network of each neighborhood to receive sub-tasks; each second-order neighborhood carries out local receiving judgment, receives corresponding number of sub-tasks and partially updates a task allocation list; if all the subtasks are received, completing task scheduling and distribution; otherwise, the extended receiving judgment is carried out again until all the subtasks are received.
Compared with the related art, the cloud computing resource pool dynamic scheduling system provided by the application has the following beneficial effects:
the application realizes the automatic construction of the dynamic dispatching system through the platform management unit, and can carry out self-adaptive subdomain division, node election and deployment according to the condition of the existing resource pool, thereby providing dispatching cooperation support for sporadic cloud computing servers; when the system is constructed, the used platform management contract has the advantages of flexible, quick and convenient deployment, and can be automatically executed after being deployed on the cloud computing platform without supervision; through the system, a user can obtain a cloud computing resource pool with sufficient computing power and low cost, and all sporadic cloud computing servers added into a platform can be comprehensively combined to obtain a resource pool with strong computing power, so that the full utilization of computing power is realized; therefore, the method has practical and popularization values.
Drawings
FIG. 1 is a schematic diagram of a monitoring system according to the present application;
FIG. 2 is a flow chart of system construction provided in an embodiment of the present application;
FIG. 3 is a flow chart of a sub-domain allocation provided in an embodiment of the present application;
FIG. 4 is a flow chart of scheduling distribution provided by an embodiment of the present application;
fig. 5 is a schematic view of region division according to an embodiment of the present application.
Detailed Description
The application will be further described with reference to the drawings and embodiments.
As shown in fig. 1, the dynamic scheduling system for cloud computing resource pools provided in this embodiment is deployed on a cloud computing platform, where the cloud computing platform includes a plurality of cloud computing servers and is divided into a global scheduling node, a subzone scheduling node and a computing execution device according to purposes; the system comprises a platform management unit, a global scheduling unit, a subdomain scheduling unit and a response execution unit;
the platform management units are distributed and deployed on the cloud computing platform in an intelligent contract form, automatically execute platform management contracts and complete the construction of a dynamic scheduling system of the resource pool; wherein the platform management contracts include subdomain division contracts, node election contracts and authorization construction contracts; when the platform management contract is automatically executed, firstly executing a subdomain division contract to finish subdomain division of the cloud computing platform; then executing node election is about, completing subdomain scheduling node election, and electing the global scheduling nodes in each subdomain scheduling node, and the rest cloud computing servers are used as computing executing equipment; finally executing an authorization construction contract, authorizing and deploying the global scheduling unit to the global scheduling node, authorizing and deploying the subzone scheduling unit to the subzone scheduling node, and deploying the response execution unit to the computing execution equipment;
the global scheduling node is used for receiving the calculation tasks, performing task downloading and data summarizing and comprises a service response module, a task scheduling module and a list management module; the subzone scheduling unit is used for dynamic scheduling of subzone computing tasks and comprises a task receiving module, a task decomposing module, a load monitoring module and a task scheduling module; the response execution unit is used for sub-domain calculation task response execution and comprises an instruction response module, a task execution module and a state monitoring module.
It should be noted that: today, there are many cloud computing servers that are not fully utilized, often without a fixed working state, and are difficult to integrate and utilize; and users suffer from the technology of non-strict scene service in a cloud computing resource pool with insufficient computational effort and low cost.
Therefore, the embodiment aims to construct a flexible deployment, low maintenance cost, multi-party and multi-source dynamic cloud resource pool system, as shown in fig. 2, the automatic construction of a dynamic scheduling system is realized through a platform management unit, and self-adaptive subdomain division, node election and deployment can be performed according to the condition of the existing resource pool, so that scheduling collaborative support is provided for sporadic cloud computing servers; when the system is constructed, the used platform management contract has the advantages of flexible, quick and convenient deployment, and can be automatically executed after being deployed on the cloud computing platform without supervision; through the system, a user can obtain a cloud computing resource pool with sufficient computing power and low cost, all sporadic cloud computing servers added into the platform can be comprehensively arranged together to obtain a resource pool with strong computing power, so that the full utilization of computing power is realized, and a server provider can also cooperate with a charging system to realize idle computing resource rendering, so that multi-party win-win is realized.
As a further solution, the sub-domain division contract divides the sub-domain scope by:
s1: acquiring IP network addresses of all cloud computing servers, and carrying out IP network address analysis to obtain server position information;
s2: acquiring the calculation force load capacity of each cloud computing server, and matching with the position information of the server to obtain calculation force distribution information;
s3: mapping the calculation force distribution information point to point on the regional position map, and calculating the calculation force aggregation degree of the regional position points to obtain a calculation force distribution thermodynamic diagram;
s4: dividing a sub-domain main body in a computational power distribution thermodynamic diagram according to a preset dividing threshold value, and acquiring a region position point set which is not lower than the dividing threshold value to obtain the sub-domain main body;
s5: the cloud computing servers positioned on the regional position point set are attributed to the current sub-domain main body, and a sub-domain main body server set is obtained;
s6: selecting subzone scheduling nodes in the subzone main body server set to obtain subzone scheduling nodes of the current subzone main body;
s7: searching neighboring nodes of a cloud computing server which does not belong to, and searching a subdomain scheduling node set in a nearby area according to server position information;
s8: the current cloud computing server respectively carries out PING subdomain scheduling on IP network addresses of node sets and carries out node communication test;
s9: selecting a subzone scheduling node with an optimal communication test result, and adding the current cloud computing server as a subzone edge server;
s10: counting attributive sub-domain edge servers by each sub-domain scheduling node to obtain sub-domain edges and sub-domain edge server sets;
the operations S1 to S10 are repeated to complete all division of subfields by constituting the subfield range of the current subfield by the subfield body and the subfield edges.
It should be noted that: since the present embodiment is a dynamically deployed scheduling system, the distribution situation of each server is not fixed, so we need to dynamically divide the sub-domains according to the server distribution situation. In the embodiment, when dividing the sub-domain, the sub-domain range is formed by adopting a two-layer structure of the sub-domain main body and the sub-domain edges, the sub-domain main body needs to ensure that the calculation power of the sub-domain main body is high, and all servers are densely distributed, so that the sub-domain main body is disconnected or online at some servers, other servers can be quickly started up, and the sub-domain main body is not greatly influenced; therefore, we need to screen out these areas and select subzone scheduling nodes in these servers to better perform task scheduling; for the server outside the sub-domain body, the server is loose, so the server is arranged as the edge of the sub-domain, and the online and offline of the server only affect the server and cannot affect the whole sub-domain.
As a further solution, the calculation force distribution information is a gaussian kernel density value with calculation force weighting, and a specific calculation formula is as follows:
；
wherein ,gaussian kernel density value->Representing the nuclear density estimation point, +.>Calculating the total amount of servers for the cloud, < >>Representing the current cloud computing server number, +.>Representing cloud computing server->Location point (s)/(s)>Representing a core Density estimation Point to cloud computing Server->Distance of location point, +.>Representing cloud computing server->Is a calculation force weight parameter of ∈10->Representing bandwidth parameters->Representing the standard deviation of the data.
It should be noted that: the kernel density estimation of the Gaussian model is an important non-parametric inspection method, and the space aggregation characteristic of point events is reflected by generating a smooth density surface in a research area; as shown in fig. 5, the left side a is a specific cloud computing server distribution diagram, the right side b is a computational force distribution thermodynamic diagram obtained through estimating the density of computational force kernels, the formula is a gaussian kernel density value with computational force weighting, so that the situation of high computational force occupation ratio and dense distribution can be reflected, in the computational force distribution thermodynamic diagram, a circle part is a sub-domain main body obtained through dividing by a preset dividing threshold, and other servers can perform sub-domain attribution through communication test results.
As a further solution, the node election contract proceeds by:
acquiring a subdomain main body server set and an IP network address; wherein the sub-domain body servers collectively compriseNAn element;
performing inter-communication test and counting communication test results to obtain a communication scoring matrixS(i, j)，i, j∈NAnd is also provided withi≠j；
Counting intercommunication scores of each sub-domain main body serverSi，Si=ΣS(i, j)； wherein ,inumber the current scoring server, andi∈N；jis a sum numbered variable, andjrespectively take 1 toNMiddle removingiNumbering outside;
solving the highest intercommunication score in the main body of the current subdomain, wherein the highest intercommunication score passesargmax（Si) Acquiring; wherein,irespectively take 1 toN；
Setting a subzone main body server corresponding to the highest intercommunication score as a subzone scheduling node, and finishing subzone scheduling node election;
after the subzone scheduling node election is completed, electing the global scheduling node in all subzone scheduling nodes of the cloud computing platform;
wherein, global scheduling node election: and performing intercommunication test on all the subzone scheduling nodes, and selecting the subzone scheduling node with the highest intercommunication score in the cloud computing platform as the global scheduling node.
It should be noted that: when selecting a subdomain scheduling node, the subdomain main body affects the subdomain most; therefore, the server set of the sub-domain main body is selected, and the server with the highest intercommunication quality is selected as a sub-domain scheduling node, so that data interaction can be rapidly performed when task scheduling is performed.
In addition, because the embodiment adopts a two-level distributed scheduling structure, the global scheduling node does not need to be responsible for concrete service scheduling of the subdomains, and only needs to carry out related summarizing work; therefore, the subzone scheduling node with the highest intercommunication score in the cloud computing platform is selected as the global scheduling node, so that the global rapid coordination is realized.
As a further solution, the list management module collects related data, gathers the related data into a corresponding data list, and performs storage and maintenance; the data list comprises a subdomain service list, a subdomain network list and a task allocation list; the subdomain service list is used for summarizing the service types processed by each subdomain and the corresponding service computing power, and the subdomain network list is used for summarizing the IP network address of each subdomain scheduling node and the subdomain network formed by each subdomain and the neighborhood thereof; the task allocation list is used for summarizing service allocation conditions.
It should be noted that: the global scheduling node assists in the co-processing of the tasks through the list management module.
As a further solution, the subdomain service list is constructed by: the subdomain scheduling node sends a resource sensing instruction to each computing executing device, and each computing executing device receives the resource sensing instruction and feeds back the processed service type and the corresponding service computing power through the instruction response module; the subdomain dispatching nodes count the service types and service computing power in the subdomain scope, and the subdomain dispatching nodes summarize the service types and the service computing power of each subdomain to obtain a subdomain service list.
As a further solution, the subdomain network list is constructed by: acquiring server position information of scheduling nodes of all subdomains, inquiring adjacent subdomains through the position information, and setting the adjacent subdomains as adjacent domains; after carrying out neighborhood query on all the subdomain scheduling nodes, obtaining a neighborhood set of each subdomain; and carrying out inter-communication test on the neighborhood set, counting communication test results, and carrying out good-bad sequencing on the communication test results, and summarizing the communication test results into a subdomain network list of the current subdomain.
As a still further solution, the information of the task allocation list record includes: the method comprises the steps of decomposing a computing task into task decomposition logic of a plurality of subtasks and task allocation information of each subtask; after the subtasks are processed, the subtask results are combined through task allocation information and task decomposition logic to obtain calculation task results.
As a further solution, when a user needs to execute a computing task, firstly, sending a service request to the service response module and providing task description information; the description information comprises user equipment IP, a request service type and a request service computing power;
the global dispatching node analyzes the user address information according to the user equipment IP, searches the latest subdomain meeting the processing capacity of the request service type in the subdomain service list, and provides the IP network address corresponding to the subdomain dispatching node for the user; issuing a task receiving command to a subdomain scheduling node through a task scheduling module;
the subdomain scheduling node obtains a task receiving command and receives a calculation task data packet sent by a user; decomposing the computing task into a plurality of subtasks through a task decomposition module, and recording and uploading task decomposition logic; and acquiring the load condition of the current subdomain through a load monitoring module, and scheduling and distributing each subtask through a task scheduling module according to the load condition.
It should be noted that: as shown in fig. 3, because the architecture of the system in this embodiment is relatively loose, we need to select the nearest subdomain as much as possible for processing and distributing the computing task, so as to avoid the burden of the network increasing due to the introduction of multiple parties; therefore, the global scheduling node does not accept the concrete calculation data, receives only the description information and distributes the sub-domain of the corresponding process.
As a further solution, the task scheduling module performs scheduling distribution on each subtask by:
acquiring the load condition of the current subdomain to obtain the current available calculation load capacity;
and (3) performing local receiving judgment: if the current available calculation power load meets the calculation power requirement of the request service, locally receiving all subtasks and updating a task allocation list to finish task scheduling and distribution; otherwise, receiving a corresponding number of sub-tasks according to the current available calculation load capacity, partially updating a task allocation list, and performing next distribution operation;
and carrying out neighborhood receiving judgment: acquiring unscheduled distribution subtasks, and requesting each neighborhood to receive the subtasks according to the sequence of the subdomain network list; each neighborhood carries out local receiving judgment, receives the corresponding number of sub-tasks and partially updates a task allocation list; if all the subtasks are received, completing task scheduling and distribution; otherwise, carrying out the next distributing operation;
and (3) performing expansion receiving judgment: each neighborhood acquires unscheduled distribution subtasks and respectively carries out neighborhood receiving judgment; requesting a sub-domain network of each neighborhood to receive sub-tasks; each second-order neighborhood carries out local receiving judgment, receives corresponding number of sub-tasks and partially updates a task allocation list; if all the subtasks are received, completing task scheduling and distribution; otherwise, the extended receiving judgment is carried out again until all the subtasks are received.
It should be noted that: the existing dispatching distribution method is characterized in that a central dispatcher decomposes and evenly distributes computing tasks to all cloud computing servers, load balance of all servers is guaranteed, all cloud computing servers are integrated to the central dispatcher after sub-task computing is completed, task computing results are returned to users, and therefore effective utilization of computing resources is achieved, and stable operation of a system is guaranteed.
However, this approach can present a significant challenge to cloud computing system networks; such as: the cloud computing users submit computing tasks to a central dispatcher arranged in Beijing, the specific task execution is executed by a computing server in Shanghai and Guangzhou, and in the long-distance data distribution and summarization process, a plurality of relay devices are required to be involved for data transmission and forwarding, so that the overall network load of the cloud computing system is increased.
Therefore, as shown in fig. 4, the present embodiment performs task claim distribution by a multi-layer inquiry method, spreads each neighborhood from the nearest sub-domain, spreads as concentric circles until the task distribution is completed, and ensures that communication switching is small and communication burden is small when spreading is performed because the local is the center.
Note that: the application is a large system application, and has more contents for some common technical means and modules, and is not repeated; if the person skilled in the art knows that the redundant setting needs to be performed on each node server, the person skilled in the art knows that the front-end page needs to be constructed so as to facilitate the user to submit tasks and allocate the tasks without sense, and the person skilled in the art knows how to construct corresponding contract codes according to contract functions, etc. Accordingly, the application is not to be considered in part as being carried out using prior art techniques; which aims to protect the technical content of the system main body.
The foregoing is only illustrative of the present application and is not to be construed as limiting the scope of the application, and all equivalent structures or equivalent flow modifications which may be made by the teachings of the present application and the accompanying drawings or which may be directly or indirectly employed in other related art are within the scope of the application.
Claims (10)
1. The cloud computing platform comprises a plurality of cloud computing servers and is divided into a global scheduling node, a subdomain scheduling node and computing execution equipment according to purposes; the system is characterized by comprising a platform management unit, a global scheduling unit, a subdomain scheduling unit and a response execution unit;
the platform management units are distributed and deployed on the cloud computing platform in an intelligent contract form, automatically execute platform management contracts and complete the construction of a dynamic scheduling system of the resource pool; wherein the platform management contracts include subdomain division contracts, node election contracts and authorization construction contracts; when the platform management contract is automatically executed, firstly executing a subdomain division contract to finish subdomain division of the cloud computing platform; then executing node election is about, completing subdomain scheduling node election, and electing the whole domain scheduling node in each subdomain scheduling node, and the rest cloud computing servers are used as computing executing equipment; finally executing an authorization construction contract, authorizing and deploying the global scheduling unit to the global scheduling node, authorizing and deploying the subzone scheduling unit to the subzone scheduling node, and deploying the response execution unit to the computing execution equipment;
the global scheduling node is used for receiving the calculation tasks, performing task downloading and data summarizing and comprises a service response module, a task scheduling module and a list management module; the subzone scheduling unit is used for dynamic scheduling of subzone computing tasks and comprises a task receiving module, a task decomposing module, a load monitoring module and a task scheduling module; the response execution unit is used for sub-domain calculation task response execution and comprises an instruction response module, a task execution module and a state monitoring module.
2. The cloud computing resource pool dynamic scheduling system of claim 1, wherein the subdomain partitioning contract partitions subdomain ranges by:
s1: acquiring IP network addresses of all cloud computing servers, and carrying out IP network address analysis to obtain server position information;
s2: acquiring the calculation force load capacity of each cloud computing server, and matching with the position information of the server to obtain calculation force distribution information;
s3: mapping the calculation force distribution information point to point on the regional position map, and calculating the calculation force aggregation degree of the regional position points to obtain a calculation force distribution thermodynamic diagram;
s4: dividing a sub-domain main body in a computational power distribution thermodynamic diagram according to a preset dividing threshold value, and acquiring a region position point set which is not lower than the dividing threshold value to obtain the sub-domain main body;
s5: the cloud computing servers positioned on the regional position point set are attributed to the current sub-domain main body, and a sub-domain main body server set is obtained;
s6: selecting subzone scheduling nodes in the subzone main body server set to obtain subzone scheduling nodes of the current subzone main body;
s7: searching neighboring nodes of a cloud computing server which does not belong to, and searching a subdomain scheduling node set in a nearby area according to server position information;
s8: the current cloud computing server respectively carries out PING subdomain scheduling on IP network addresses of node sets and carries out node communication test;
s9: selecting a subzone scheduling node with an optimal communication test result, and adding the current cloud computing server as a subzone edge server;
s10: counting attributive sub-domain edge servers by each sub-domain scheduling node to obtain sub-domain edges and sub-domain edge server sets;
the operations S1 to S10 are repeated to complete all division of subfields by constituting the subfield range of the current subfield by the subfield body and the subfield edges.
3. The cloud computing resource pool dynamic scheduling system according to claim 2, wherein the computing force distribution information is a gaussian kernel density value with computing force weighting, and the specific calculation formula is:
；
wherein ,gaussian kernel density value->Representing the nuclear density estimation point, +.>Calculating the total amount of servers for the cloud, < >>Representing the current cloud computing server number, +.>Representing cloud computing server->Location point (s)/(s)>Representing a core Density estimation Point to cloud computing Server->Distance of location point, +.>Representing cloud computing server->Is a calculation force weight parameter of ∈10->Representing bandwidth parameters->Representing the standard deviation of the data.
4. The cloud computing resource pool dynamic scheduling system of claim 1, wherein the node election contract is performed by:
acquiring a subdomain main body server set and an IP network address; wherein the sub-domain body servers collectively compriseNAn element;
performing inter-communication test and counting communication test results to obtain a communication scoring matrixS(i, j )，i, j∈NAnd is also provided withi≠j；
Counting intercommunication scores of each sub-domain main body serverSi，Si =ΣS(i, j )； wherein ,inumber the current scoring server, andi∈N；jis a sum numbered variable, andjrespectively take 1 toNMiddle removingiNumbering outside;
solving the highest intercommunication score in the main body of the current subdomain, wherein the highest intercommunication score passesargmax（Si) Acquiring; wherein,irespectively take 1 toN；
Setting a subzone main body server corresponding to the highest intercommunication score as a subzone scheduling node, and finishing subzone scheduling node election;
after the subzone scheduling node election is completed, electing the global scheduling node in all subzone scheduling nodes of the cloud computing platform;
wherein, global scheduling node election: and performing intercommunication test on all the subzone scheduling nodes, and selecting the subzone scheduling node with the highest intercommunication score in the cloud computing platform as the global scheduling node.
5. The cloud computing resource pool dynamic scheduling system of claim 1, wherein the list management module collects related data, gathers the related data into a corresponding data list, and performs storage and maintenance; the data list comprises a subdomain service list, a subdomain network list and a task allocation list; the subdomain service list is used for summarizing the service types processed by each subdomain and the corresponding service computing power, and the subdomain network list is used for summarizing the IP network address of each subdomain scheduling node and the subdomain network formed by each subdomain and the neighborhood thereof; the task allocation list is used for summarizing service allocation conditions.
6. The cloud computing resource pool dynamic scheduling system of claim 5, wherein the subdomain service list is constructed by: the subdomain scheduling node sends a resource sensing instruction to each computing executing device, and each computing executing device receives the resource sensing instruction and feeds back the processed service type and the corresponding service computing power through the instruction response module; the subdomain dispatching nodes count the service types and service computing power in the subdomain scope, and the subdomain dispatching nodes summarize the service types and the service computing power of each subdomain to obtain a subdomain service list.
7. The cloud computing resource pool dynamic scheduling system of claim 5, wherein said subzone network list is constructed by: acquiring server position information of scheduling nodes of all subdomains, inquiring adjacent subdomains through the position information, and setting the adjacent subdomains as adjacent domains; after carrying out neighborhood query on all the subdomain scheduling nodes, obtaining a neighborhood set of each subdomain; and carrying out inter-communication test on the neighborhood set, counting communication test results, and carrying out good-bad sequencing on the communication test results, and summarizing the communication test results into a subdomain network list of the current subdomain.
8. The cloud computing resource pool dynamic scheduling system of claim 5, wherein said task allocation list recorded information comprises: the method comprises the steps of decomposing a computing task into task decomposition logic of a plurality of subtasks and task allocation information of each subtask; after the subtasks are processed, the subtask results are combined through task allocation information and task decomposition logic to obtain calculation task results.
9. The cloud computing resource pool dynamic scheduling system of claim 6, wherein when a user needs to perform a computing task, firstly sending a service request to the service response module and providing task description information; the description information comprises user equipment IP, a request service type and a request service computing power;
the global dispatching node analyzes the user address information according to the user equipment IP, searches the latest subdomain meeting the processing capacity of the request service type in the subdomain service list, and provides the IP network address corresponding to the subdomain dispatching node for the user; issuing a task receiving command to a subdomain scheduling node through a task scheduling module;
the subdomain scheduling node obtains a task receiving command and receives a calculation task data packet sent by a user; and decomposing the computing task into a plurality of subtasks through a task decomposition module, and recording and uploading task decomposition logic.
10. A cloud computing resource pool dynamic scheduling system as claimed in any of claims 5 and 9, wherein the task scheduling module schedules and distributes sub-tasks by:
and (3) performing local receiving judgment: the subdomain scheduling nodes acquire the load condition of the current subdomain through a load monitoring module to acquire the current available calculation load capacity; if the current available calculation power load meets the calculation power requirement of the request service, locally receiving all subtasks and updating a task allocation list to finish task scheduling and distribution; otherwise, receiving a corresponding number of sub-tasks according to the current available calculation load capacity, partially updating a task allocation list, and performing next distribution operation;
and carrying out neighborhood receiving judgment: acquiring unscheduled distribution subtasks, and requesting each neighborhood to receive the subtasks according to the sequence of the subdomain network list; each neighborhood carries out local receiving judgment, receives the corresponding number of sub-tasks and partially updates a task allocation list; if all the subtasks are received, completing task scheduling and distribution; otherwise, carrying out the next distributing operation;
and (3) performing expansion receiving judgment: each neighborhood acquires unscheduled distribution subtasks and respectively carries out neighborhood receiving judgment; requesting a sub-domain network of each neighborhood to receive sub-tasks; each second-order neighborhood carries out local receiving judgment, receives corresponding number of sub-tasks and partially updates a task allocation list; if all the subtasks are received, completing task scheduling and distribution; otherwise, the extended receiving judgment is carried out again until all the subtasks are received.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310897504.7A CN116633932B (en) | 2023-07-21 | 2023-07-21 | Dynamic scheduling system for cloud computing resource pool |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310897504.7A CN116633932B (en) | 2023-07-21 | 2023-07-21 | Dynamic scheduling system for cloud computing resource pool |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116633932A true CN116633932A (en) | 2023-08-22 |
| CN116633932B CN116633932B (en) | 2023-09-19 |
Family
ID=87642144
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310897504.7A Active CN116633932B (en) | 2023-07-21 | 2023-07-21 | Dynamic scheduling system for cloud computing resource pool |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116633932B (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140282944A1 (en) * | 2013-03-12 | 2014-09-18 | Stackinsider Technology LLC | Methods and systems of deploying cloud computing platforms |
| US20150271251A1 (en) * | 2012-10-23 | 2015-09-24 | Telefonaktiebolaget L M. Ericsson (Publ) | Method and System for Cloud Service Deployment |
| CN105959138A (en) * | 2016-04-29 | 2016-09-21 | 深圳前海大数点科技有限公司 | Micro-service dynamic disposition system and method based on cloud calculation |
| CN110837408A (en) * | 2019-09-16 | 2020-02-25 | 中国科学院软件研究所 | High-performance server-free computing method and system based on resource cache |
| US20220163942A1 (en) * | 2020-11-20 | 2022-05-26 | Hitachi, Ltd. | Distributed System and Data Transmission Method |
| CN114650320A (en) * | 2022-03-10 | 2022-06-21 | 腾讯科技（深圳）有限公司 | Task scheduling method and device, storage medium and electronic equipment |
-
2023
- 2023-07-21 CN CN202310897504.7A patent/CN116633932B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20150271251A1 (en) * | 2012-10-23 | 2015-09-24 | Telefonaktiebolaget L M. Ericsson (Publ) | Method and System for Cloud Service Deployment |
| US20140282944A1 (en) * | 2013-03-12 | 2014-09-18 | Stackinsider Technology LLC | Methods and systems of deploying cloud computing platforms |
| CN105959138A (en) * | 2016-04-29 | 2016-09-21 | 深圳前海大数点科技有限公司 | Micro-service dynamic disposition system and method based on cloud calculation |
| CN110837408A (en) * | 2019-09-16 | 2020-02-25 | 中国科学院软件研究所 | High-performance server-free computing method and system based on resource cache |
| US20220163942A1 (en) * | 2020-11-20 | 2022-05-26 | Hitachi, Ltd. | Distributed System and Data Transmission Method |
| CN114650320A (en) * | 2022-03-10 | 2022-06-21 | 腾讯科技（深圳）有限公司 | Task scheduling method and device, storage medium and electronic equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116633932B (en) | 2023-09-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107241319B (en) | Distributed network crawler system based on VPN and scheduling method | |
| CN104038540B (en) | Method and system for automatically selecting application proxy server | |
| CN110519090B (en) | Method and system for allocating accelerator cards of FPGA cloud platform and related components | |
| CN113342510B (en) | Water and power basin emergency command cloud-side computing resource cooperative processing method | |
| CN109085999A (en) | data processing method and processing system | |
| CN112148492A (en) | Service deployment and resource allocation method considering multi-user mobility | |
| CN110071965A (en) | A kind of data center management system based on cloud platform | |
| CN109343945A (en) | A kind of multitask dynamic allocation method based on contract net algorithm | |
| Cao et al. | Towards tenant demand-aware bandwidth allocation strategy in cloud datacenter | |
| Ahvar et al. | NACER: a Network-Aware Cost-Efficient Resource allocation method for processing-intensive tasks in distributed clouds | |
| CN102929693B (en) | Performance evaluation method and device for servers of whole equipment cabinet | |
| CN117149445B (en) | Cross-cluster load balancing method and device, equipment and storage medium | |
| CN116633932B (en) | Dynamic scheduling system for cloud computing resource pool | |
| CN113242304A (en) | Edge side multi-energy data acquisition scheduling control method, device, equipment and medium | |
| JP5969340B2 (en) | Resource management system, resource management method, and resource management program | |
| CN116708446A (en) | Network performance comprehensive weight decision-based computing network scheduling service method and system | |
| CN109302302A (en) | Service scalable method, system and the computer readable storage medium of network element | |
| CN111722783B (en) | Data storage method and device | |
| CN109600251B (en) | Virtual network deployment method, system and device | |
| CN102929605A (en) | Cloud-computing-based open interface of data mining system | |
| DE102013207603B4 (en) | Run jobs efficiently in a shared pool of resources | |
| CN107294765B (en) | Network function virtualization self-adaptive trust management method | |
| CN111328107A (en) | Multi-cloud heterogeneous mobile edge computing system architecture and energy optimization design method | |
| WO2016184521A1 (en) | Method for providing locations for performing tasks of moving objects | |
| CN116074322B (en) | High-throughput task scheduling method, system and medium based on intelligent message segmentation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |