CN117573335A - Method, device, equipment and computer readable storage medium for settling account of computing power resources - Google Patents
Method, device, equipment and computer readable storage medium for settling account of computing power resources Download PDFInfo
- Publication number
- CN117573335A CN117573335A CN202311454301.7A CN202311454301A CN117573335A CN 117573335 A CN117573335 A CN 117573335A CN 202311454301 A CN202311454301 A CN 202311454301A CN 117573335 A CN117573335 A CN 117573335A
- Authority
- CN
- China
- Prior art keywords
- computing
- resource
- task
- settlement
- target server
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 45
- 238000003860 storage Methods 0.000 title claims abstract description 20
- 238000004364 calculation method Methods 0.000 claims abstract description 21
- 238000009826 distribution Methods 0.000 claims description 6
- 238000001914 filtration Methods 0.000 claims description 5
- 238000012544 monitoring process Methods 0.000 claims description 3
- 238000010586 diagram Methods 0.000 description 12
- 238000004891 communication Methods 0.000 description 7
- 238000004590 computer program Methods 0.000 description 7
- 230000006870 function Effects 0.000 description 6
- 238000005259 measurement Methods 0.000 description 6
- 230000008569 process Effects 0.000 description 5
- 238000012545 processing Methods 0.000 description 5
- 230000009471 action Effects 0.000 description 4
- 238000011161 development Methods 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 4
- 238000012549 training Methods 0.000 description 4
- 238000004088 simulation Methods 0.000 description 3
- 241001481833 Coryphaena hippurus Species 0.000 description 2
- 238000004422 calculation algorithm Methods 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 230000010354 integration Effects 0.000 description 2
- 238000007726 management method Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000004075 alteration Effects 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000012790 confirmation Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 239000003814 drug Substances 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000002688 persistence Effects 0.000 description 1
- 238000013468 resource allocation Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 239000000126 substance Substances 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
- 239000002699 waste material Substances 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
- G06F9/5044—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals considering hardware capabilities
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
- G06F9/5038—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals considering the execution order of a plurality of tasks, e.g. taking priority or time dependency constraints into consideration
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The application discloses a method, a device and equipment for settling account of computing power resources and a computer-readable storage medium, and belongs to the technical field of big data cloud computing. The method comprises the steps of: distributing the current computing task to a target server by utilizing a computing resource pool; acquiring the running time length, the resource occupancy rate and the settlement rule of the target server of the current calculation task; and determining the real-time resource cost of the current computing task based on the running time length, the resource occupancy rate and the settlement rule. By the method for settling the computing power resources, the utilization rate of the computing power resources is optimized, the flexibility of the use of the computing power resources is improved, the utilization rate of the computing power resources is improved, and the economic cost for computing tasks by using the computing resources is greatly reduced.
Description
Technical Field
The present disclosure relates to the field of big data cloud computing technologies, and in particular, to a method, an apparatus, a device, and a computer readable storage medium for settling a computing power resource.
Background
The Chinese communication institute issues white paper book of the development index of Chinese computing power (2022) in 2022, 11 months, which indicates that the total computing power scale in 2021 of China reaches 202 EFlips, the global proportion is about 33%, the intelligent computing power is rapidly increased, the speed is increased by 85%, and the computing power proportion in China is more than 50%. By the end of 2021, the application distribution of the computing industry in China is mainly the Internet, government enterprises, finance and others. In the future, the power-calculating scale of China will continue to expand, and the structure of power-calculating service will also continue to develop and optimize.
In view of the extremely limited computational power of devices such as personal computers, workstations, and the like, some tasks such as large model training, simulation, and the like need to rely on large computational power devices such as supercomputers of supercomputers, and computing resources required for computing tasks are acquired from the supercomputers. In the process of the computing power service, the computing power public platform centralizes the idle computing power resources of different data centers and sells the idle computing power resources to users of enterprises, universities and the like. For the resource allocation link in the current computing power service, the cloud computing mode locks the used computing and storage resources through the resource virtualization technology, namely, the fixed computing resource amount is bound with the computing task, however, the utilization rate of the computing resource is not high, a plurality of computing resources are in an idle state, and the computing power use cost is usually settled according to the fixed computing resource amount, so that the economic cost for using the computing resource to perform the computing task is high.
In view of this, a new solution is needed to solve the above-mentioned problems.
Disclosure of Invention
The main purpose of the present application is to provide a method, a device and a computer readable storage medium for settling account of computing resources, which aims to solve the technical problem of high economic cost of computing tasks using computing resources in the current computing service.
In order to achieve the above object, the present application provides a method for settling a computing resource, the method for settling a computing resource comprising the steps of:
distributing the current computing task to a target server by utilizing a computing resource pool;
acquiring the running time length, the resource occupancy rate and the settlement rule of the target server of the current calculation task;
and determining the real-time resource cost of the current computing task based on the running time length, the resource occupancy rate and the settlement rule.
Optionally, before the step of allocating the current computing task to the target server using the computing resource pool, the method further comprises:
and adding the target server into a computing resource pool by using the Zookeeper cluster, and storing the resource information of the target server by using a MySQL database.
Optionally, the step of allocating the current computing task to the target server by using the computing resource pool includes:
determining workflow data of a current computing task by utilizing a computing resource pool;
dividing the workflow data into a plurality of work units by using a main server;
and distributing the working units to the corresponding target servers.
Optionally, the step of assigning the working unit to a corresponding target server includes:
determining the task type of the working unit;
and distributing the working unit to a target server corresponding to the task type.
Optionally, the step of assigning the working unit to a corresponding target server further includes:
load information of a plurality of servers is obtained;
determining a target server in a plurality of servers according to the load information;
and distributing the working units to the corresponding target servers.
Optionally, the step of determining a target server of the plurality of servers according to the load information includes:
filtering servers with load information smaller than a preset load threshold to determine a target server in a plurality of servers; or (b)
Determining the distribution priority among a plurality of servers according to the load information;
a target server of the plurality of servers is determined based on the allocation priority.
Optionally, the settlement rules include resource unit price; the step of determining the real-time resource cost of the current computing task based on the running time length, the resource occupancy rate and the settlement rule comprises the following steps:
and determining a product result obtained by multiplying the running time, the resource occupancy rate and the resource unit price as the real-time resource cost of the current computing task.
In addition, in order to achieve the above object, the present application further provides a computing power resource settlement apparatus, including:
the task allocation module is used for allocating the current computing task to the target server by utilizing the computing resource pool;
the task monitoring module is used for acquiring the running time of the current calculation task, the resource occupancy rate and the settlement rule of the target server;
and the expense settlement module is used for determining the real-time resource expense of the current calculation task based on the operation time length, the resource occupancy rate and the settlement rule.
The application also provides a computing power resource settlement device, which comprises a processor, a memory and a computing power resource settlement program stored on the memory and executable by the processor, wherein the computing power resource settlement program realizes the steps of the computing power resource settlement method when being executed by the processor.
The present application also provides a computer-readable storage medium having stored thereon a computing resource settlement program, wherein the computing resource settlement program, when executed by a processor, implements the steps of the computing resource settlement method as described above.
According to the computing power resource settlement method in the technical scheme, a computing resource pool is utilized to distribute a current computing task to a target server; acquiring the running time length, the resource occupancy rate and the settlement rule of the target server of the current calculation task; and determining the real-time resource cost of the current computing task based on the running time length, the resource occupancy rate and the settlement rule. According to the embodiment of the application, the computing and storage resources which are locked and used in the original cloud computing mode through the resource virtualization technology are improved to be used and released in the form of real-time tasks, the utilization rate of the computing resources is optimized, the flexibility of the computing resources is improved, the cost settlement is carried out in the process of task execution and after the task execution based on the actual operation condition of the task, and therefore the idle computing resources are not required to be bound to a fixed user, but are distributed by the specific tasks operated by actual demands, the utilization rate of the computing resources is improved, and the economic cost of the computing resources for computing tasks is greatly reduced.
Drawings
FIG. 1 is a schematic diagram of a hardware operating environment of a computing power resource settlement apparatus according to an embodiment of the present application;
FIG. 2 is a flowchart of a first embodiment of a computing resource settlement method according to the present application;
FIG. 3 is a detailed flowchart of step S10 of an embodiment of the computing resource settlement method of the present application;
FIG. 4 is a simplified schematic diagram of an overall process involved in the settlement method of computing resources of the present application;
fig. 5 is a schematic diagram of a frame structure of the computing resource settlement apparatus of the present application.
The realization, functional characteristics and advantages of the present application will be further described with reference to the embodiments, referring to the attached drawings.
Detailed Description
It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the present application.
The embodiment of the application provides a computing power resource settlement device which can be a data processing terminal such as a computer, a server and the like.
As shown in fig. 1, fig. 1 is a schematic structural diagram of a hardware running environment of a computing power resource settlement apparatus according to an embodiment of the present application.
As shown in fig. 1, the computing power resource settlement apparatus may include: a processor 1001, such as a CPU, a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Wherein the communication bus 1002 is used to enable connected communication between these components. The user interface 1003 may include a Display (Display), an input unit such as a control panel, and the optional user interface 1003 may also include a standard wired interface, a wireless interface. Network interface 1004 may optionally include a standard wired interface, a wireless interface (e.g., a WIFI interface). The memory 1005 may be a high-speed RAM memory or a stable memory (non-volatile memory), such as a disk memory. The memory 1005 may also optionally be a storage device separate from the processor 1001 described above. A computing power resource settlement program may be included in the memory 1005 as a computer storage medium.
Those skilled in the art will appreciate that the hardware configuration shown in fig. 1 does not constitute a limitation of the apparatus, and may include more or fewer components than shown, or may combine certain components, or may be arranged in different components.
With continued reference to fig. 1, the memory 1005 in fig. 1, which is a computer-readable storage medium, may include an operating device, a user interface module, a network communication module, and a computing resource settlement program.
In fig. 1, the network communication module is mainly used for connecting with a server and performing data communication with the server; and the processor 1001 may call the computing power resource settlement program stored in the memory 1005 and perform the following operations:
distributing the current computing task to a target server by utilizing a computing resource pool;
acquiring the running time length, the resource occupancy rate and the settlement rule of the target server of the current calculation task;
and determining the real-time resource cost of the current computing task based on the running time length, the resource occupancy rate and the settlement rule.
Further, the processor 1001 may call the computing power resource settlement program stored in the memory 1005, and further perform the following operations:
and adding the target server into a computing resource pool by using the Zookeeper cluster, and storing the resource information of the target server by using a MySQL database.
Further, the processor 1001 may call the computing power resource settlement program stored in the memory 1005, and further perform the following operations:
determining workflow data of a current computing task by utilizing a computing resource pool;
dividing the workflow data into a plurality of work units by using a main server;
and distributing the working units to the corresponding target servers.
Further, the processor 1001 may call the computing power resource settlement program stored in the memory 1005, and further perform the following operations:
determining the task type of the working unit;
and distributing the working unit to a target server corresponding to the task type.
Further, the processor 1001 may call the computing power resource settlement program stored in the memory 1005, and further perform the following operations:
load information of a plurality of servers is obtained;
determining a target server in a plurality of servers according to the load information;
and distributing the working units to the corresponding target servers.
Further, the processor 1001 may call the computing power resource settlement program stored in the memory 1005, and further perform the following operations:
filtering servers with load information smaller than a preset load threshold to determine a target server in a plurality of servers; or (b)
Determining the distribution priority among a plurality of servers according to the load information;
a target server of the plurality of servers is determined based on the allocation priority.
Further, the processor 1001 may call the computing power resource settlement program stored in the memory 1005, and further perform the following operations:
and determining a product result obtained by multiplying the running time, the resource occupancy rate and the resource unit price as the real-time resource cost of the current computing task.
Based on the hardware structure of the computing power resource settlement device, various embodiments of the computing power resource settlement method are provided.
The embodiment of the application provides a settlement method for computing power resources.
Referring to fig. 2, fig. 2 is a flow chart of a first embodiment of the computing resource settlement method of the present application; in a first embodiment of the present application, the method for settling a computing resource includes the steps of:
step S10, distributing a current computing task to a target server by utilizing a computing resource pool;
in order to implement the technical solution of the present embodiment from the code layer, in the early stage work of the present embodiment, it may include:
1. the method comprises the steps of modifying a Dolphin scheduler to run a working example and changing a task example code flow, realizing the creation and state change of a task example associated code, synchronously generating an order for charging according to the amount by starting the running workflow example, and increasing the support for generating the order.
2. And the development order list display function is used for creating an order entity, associating task instance numbers, completing list inquiry of orders and state change interface development, and recording the cost generated by task operation in real time.
3. Developing a charging rule configuration function, creating a charging configuration entity and a resource entity, associating the computing resources in a charging configuration table, inquiring the charging configuration of the computing resources through the resource numbers, and completing the monovalent configuration of the computing resources through an editing interface of the charging configuration to realize the monovalent configuration of the computing resources with different brands/specifications.
In this embodiment, various servers with different types of computing resources may be uniformly registered in a computing resource pool, and when an operation of a certain computing task needs to be executed, the computing resource pool allocates a current computing task to an appropriate target server, and specifically, may be allocated to a target server corresponding to the task type according to the task type of the computing task. The target server may be one or more. The task type of the computing task can be defined according to actual needs, for example, the task type can be divided into network model training for image recognition, network model training for natural language, simulation and simulation operation, AI large model training and the like, the classification mode is only one of the task types, and the task type can be further subdivided into types in more specific fields. Task types can also be classified according to industries, such as aerospace, automobiles, new energy sources, chemical industry, biological medicine and the like, and are not described in detail herein.
In addition, for the creation of the current computing task, the type of task and components required may be selected at the time of creating the computing task, defining a workflow by drag and configuration parameters to create any one of the computing tasks, and designating the computing task to run in a pool of computing resources that contains the target server.
In an embodiment, before the step S10, the method further includes:
and a step a, adding the target server into a computing resource pool by using the Zookeeper cluster, and storing the resource information of the target server by using a MySQL database.
In this embodiment, mySQL relational databases are taken as an example for illustration, but other suitable databases may be selected. The MySQL database is used as a persistence layer of the business logic, and a data storage layer of the business logic is constructed by utilizing the characteristics of association relation, structural inquiry and the like of the relational database. The computing node (server) of the computing power service is registered in the Zookeeper cluster, and server resources can synchronize server related data from the Zookeeper to a resource management list of a MySQL database during charging, and server resource information is stored through the MySQL database. The specific operation is that each server is added into a computational resource pool (computational resource pool) through a Zookeeper cluster, and after the completion, the number, the name, the specification, the system architecture, the IP address, the deployment time, the running state (such as CPU, GPU load and the like) and other resource information of the server are saved through a MySQL database.
According to the embodiment, CPU/GPU, memory and other computing resources of a plurality of servers are uniformly accessed to a computing resource pool through service registration, so that the computing resources can be matched with corresponding users, and the safety problems of inconsistent information formats, non-uniform measurement units, acquisition, transmission and storage of information under different management domains and the like of interoperable heterogeneous resources among cross-management-domain charging main bodies can be solved.
Referring to fig. 3, in an embodiment, the step S10 includes:
step S11, determining workflow data of a current computing task by utilizing a computing resource pool;
step S12, the main server is utilized to divide the workflow data into a plurality of working units;
and step S13, the working units are distributed to the corresponding target servers.
After defining task workflow by using a computing resource pool, storing workflow metadata into a database by calling an API (Application Program Interface ) interface, acquiring the workflow data from the database by a main server, splitting the workflow data into a plurality of subtasks (working units), enabling each workflow to be split by only one main server through the realization of a distributed lock among the plurality of main servers, and distributing the split tasks (working units) to corresponding target servers (a plurality of subtasks can be distributed to a plurality of target servers, and the target servers can be executed in parallel so as to improve the task computing efficiency). After receiving the subtask, the target server returns acknowledgement information ack and response to the main server, which indicates that the task has been received. After the confirmation information is sent, the task is started to be executed, the target server executes the task and returns the execution progress and the result to the main server, the main server writes the execution state of the corresponding task into a database and feeds back the execution state to a UI interface through an API interface to display the execution state of each subtask, the maximum optimization of the performance of the server selected by the task operation is achieved by distributing the pressure of the server through a linear weighting algorithm, the cost generated by the task operation is ensured to be accumulated always with the optimal resource utilization rate, the algorithm can report the load information of the target server to a registry (computing resource pool) at intervals, and the load information can comprise: CPU/GPU load values and remaining physical memory.
In one embodiment, the step S13 includes:
step b, determining the task type of the working unit;
and c, distributing the working unit to a target server corresponding to the task type.
Different servers can be collocated with different hardware specifications, combined architectures and operating environments, so that different computing power levels and specific task types can be different. Therefore, after the workflow data is segmented into a plurality of working units, a matched target server is determined for the workflow data according to the task types corresponding to the working units, and the working units are distributed to the target server, and the target server can operate the subtasks with better performance, so that each subtask can obtain better calculation efficiency, the operation speed is improved, and the cost of using calculation resources is reduced.
In an embodiment, the step S13 further includes:
step d, load information of a plurality of servers is obtained;
step e, determining a target server in a plurality of servers according to the load information;
and f, distributing the working unit to a corresponding target server.
Load information, such as CPU/GPU load values and remaining physical memory, for each server in the computing resource pool may be obtained. Based on the load information of different servers, one or more target servers with better load information can be determined through comparison, so that working units (subtasks) are distributed to the corresponding target servers, and therefore, better operation efficiency of each calculation subtask is ensured to be maintained.
In an embodiment, the step e, determining a target server from the plurality of servers according to the load information, includes:
filtering servers with load information smaller than a preset load threshold to determine a target server in a plurality of servers; or (b)
The corresponding load threshold values can be respectively configured and preset for hardware such as a CPU, a GPU, a memory and the like, and the server can be filtered out without serving as a target server as long as the load information of one piece of hardware is smaller than the corresponding preset load threshold value. Accordingly, if the load information of each hardware is greater than or equal to the corresponding preset load threshold, the server is reserved as the target server, so as to ensure that the target server keeps a better performance state to execute the computing task.
Determining the distribution priority among a plurality of servers according to the load information;
a target server of the plurality of servers is determined based on the allocation priority.
Or comparing load information of each server managed by the computing resource pool, determining one or more target servers with better performance than the plurality of servers, particularly, prioritizing each server through the comparison of the load information, taking the server or servers with the highest priority as the target servers, and for determining the target server with the highest priority, describing that the computing task can be completed by only executing and operating on the server, and for determining the target servers with the higher priority, determining the number of the target servers depends on the number of subtasks, for example, 5 subtasks, and then determining the server with the highest priority rank as the target server.
Step S20, acquiring the running time of the current computing task, the resource occupancy rate and the settlement rule of the target server;
the running time of the current computing task and the resource occupancy rate (which can be averaged) can be monitored in real time, and the computing resources used by the computing task can be settled by configuring and utilizing the settlement rules of the target server.
One embodiment of a settlement rule for a particular target server:
the charging configuration table is newly added in the MySQL database and the computing power resource is associated through the charging configuration table, so that the charging unit price of the resource can be input at the client to complete the charging rule configuration, and the associated computing power resource information is stored in the MySQL database through the charging configuration table. The user (charging main body, user) triggers the metering start by using the metering resource through a certain action (behavior), so as to form a metering entry; the metering entry should be the benchmark object of measurement for the entire metering and billing system; this concept is described in a more precise language:
metering item = metering body + action + quantifiable resource
The amount metered can be said to be how much the metering item is used, precisely described as: measurement value = amount of measurement item.
Based on the rules, the servers with various specifications/different models/different instruction sets can be converted into standard measurement units on the basis, and measurement standards for different resources are provided according to the characteristics of different calculation objects and the use modes of users. The metering transaction may begin with the start of the action of the metering body and end with the end of the action. For example, in a task execution scenario: when a user submits a computing task, the transaction should be created once the task is captured by the system, then the computing task starts waiting in the queue until an instant of executing the metering transaction starts, and when the task is completed or failed or cancelled (in any case, the computing resource used by the application and the application corresponding to the computing task of the user reaches an end state), the metering transaction ends.
And step S30, determining the real-time resource cost of the current computing task based on the running time length, the resource occupancy rate and the settlement rule.
When the calculation task is started, the corresponding order information is generated at the main server according to the task type or the task state and is stored in the MySQL database, and the task creation time is synchronized to generate order information, wherein the order statistics content can comprise numbers, creation time, running state, accumulated time length, accumulated amount, resource occupancy rate and the like. When a computing node (server) included in a task runs a computing task, inquiring the current charging configuration of the server through a resource number, and assuming that the unit price is set to be 50 yuan, calculating the accumulated amount of the running task at the price of 50 yuan/min, synchronizing the amount of the running task to the accumulated amount of the order for display, wherein the running time of the task is the actual running time of the task, and the charging time is equal to the total running time of the task.
Specifically, the settlement rules include resource unit price; the step S30 includes:
and g, determining a product result obtained by multiplying the operation time length, the resource occupancy rate and the resource unit price as the real-time resource cost of the current calculation task.
When a plurality of tasks (can come from the same computing task or different computing tasks) run on the same server at the same time, the computing formula is as follows: resource unit price time of task operation time, resource occupation ratio, when the task is finished, automatically closing the order and recording the finally generated amount.
Aiming at the development trend of the network computing integration in the future, the settlement and charging mode can realize the optimal scheduling of resources, and the various unsmoothness computing power resources are segmented and refined, so that the requirements on the network resources are met, and various demands such as computing and storing are also included. Meanwhile, network computing integration refined charging can be carried out based on a Service Level Agreement (SLA), and the requirements of diversified networks and computing resources of future industry users are met.
According to the computing power resource settlement method in the technical scheme, a computing resource pool is utilized to distribute a current computing task to a target server; acquiring the running time length, the resource occupancy rate and the settlement rule of the target server of the current calculation task; and determining the real-time resource cost of the current computing task based on the running time length, the resource occupancy rate and the settlement rule. According to the embodiment of the application, the computing and storage resources which are locked and used in the original cloud computing mode through the resource virtualization technology are improved to be used and released in the form of real-time tasks, the utilization rate of the computing resources is optimized, the flexibility of the computing resources is improved, the cost settlement is carried out in the process of task execution and after the task execution based on the actual operation condition of the task, and therefore the idle computing resources are not required to be bound to a fixed user, but are distributed by the specific tasks operated by actual demands, the utilization rate of the computing resources is improved, and the economic cost of the computing resources for computing tasks is greatly reduced.
For further understanding of the whole flow of the embodiments of the present application, please refer to fig. 4, fig. 4 is a simplified schematic diagram of the whole flow related to the computing resource settlement method of the present application.
And creating a computing task through the Dolphin scheduler, and configuring corresponding computing requirements according to the task.
And performing resource matching on the established tasks through a computing resource pool, and distributing idle resources according to task configuration requirements.
And starting the execution task and configuring the charging rule according to the resource type/task type.
And configuring a charging rule according to the resource type, generating an order, tracking and recording the cost of the resource according to the task running time and the actual resource utilization rate, counting the accumulated amount in real time, and finally generating and outputting a consumption detail to a user.
According to the technical scheme, the required cost is calculated by acquiring the memory/CPU/GPU resource proportion occupied by the calculation task when running on the server and the task running time (including the total time), so that the calculation task can be counted in a single task running state on the premise of fixed resource unit price, the calculation resource can be taken and released in real time according to the actual condition of the task, the utilization rate of the calculation resource is effectively improved, and the resource waste is reduced.
Referring to fig. 5, fig. 5 is a schematic diagram of a frame structure of the computing resource settlement apparatus of the present application. The application also provides a computing power resource settlement device, which comprises:
a task allocation module a10, configured to allocate a current computing task to a target server by using a computing resource pool;
the task monitoring module A20 is used for acquiring the running time of the current calculation task, the resource occupancy rate and the settlement rule of the target server;
and a fee settlement module A30 for determining the real-time resource fee of the current computing task based on the running time length, the resource occupancy rate and the settlement rules.
Optionally, the task allocation module a10 is further configured to:
and adding the target server into a computing resource pool by using the Zookeeper cluster, and storing the resource information of the target server by using a MySQL database.
Optionally, the task allocation module a10 is further configured to:
determining workflow data of a current computing task by utilizing a computing resource pool;
dividing the workflow data into a plurality of work units by using a main server;
and distributing the working units to the corresponding target servers.
Optionally, the task allocation module a10 is further configured to:
determining the task type of the working unit;
and distributing the working unit to a target server corresponding to the task type.
Optionally, the task allocation module a10 is further configured to:
load information of a plurality of servers is obtained;
determining a target server in a plurality of servers according to the load information;
and distributing the working units to the corresponding target servers.
Optionally, the task allocation module a10 is further configured to:
filtering servers with load information smaller than a preset load threshold to determine a target server in a plurality of servers; or (b)
Determining the distribution priority among a plurality of servers according to the load information;
a target server of the plurality of servers is determined based on the allocation priority.
Optionally, the fee settlement module a30 is further configured to:
and determining a product result obtained by multiplying the running time, the resource occupancy rate and the resource unit price as the real-time resource cost of the current computing task.
The specific implementation manner of the computing power resource settlement device is basically the same as the above embodiments of the computing power resource settlement method, and will not be repeated here.
Furthermore, the application also provides a computer readable storage medium. The computer readable storage medium stores a settlement program for the computing power resources, wherein the settlement program for the computing power resources realizes the steps of the settlement method for the computing power resources when being executed by a processor.
The method implemented when the computing resource settlement program is executed may refer to various embodiments of the computing resource settlement method of the present application, which are not described herein.
It will be apparent to those skilled in the art that embodiments of the present application may be provided as a method, apparatus, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (devices), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
It should be noted that in the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The application may be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by one and the same item of hardware. The use of the words first, second, third, etc. do not denote any order. These words may be interpreted as names.
While preferred embodiments of the present application have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. It is therefore intended that the following claims be interpreted as including the preferred embodiments and all such alterations and modifications as fall within the scope of the application.
The foregoing description is only of the preferred embodiments of the present application and is not intended to limit the scope of the claims, and all equivalent structural changes made in the present application and the accompanying drawings or direct/indirect application in other related technical fields are included in the scope of the present application.
Claims (10)
1. The method for settling the computing power resource is characterized by comprising the following steps of:
distributing the current computing task to a target server by utilizing a computing resource pool;
acquiring the running time length, the resource occupancy rate and the settlement rule of the target server of the current calculation task;
and determining the real-time resource cost of the current computing task based on the running time length, the resource occupancy rate and the settlement rule.
2. The computing power resource settlement method of claim 1, wherein prior to said step of assigning a current computing task to a target server using a pool of computing resources, said method further comprises:
and adding the target server into a computing resource pool by using the Zookeeper cluster, and storing the resource information of the target server by using a MySQL database.
3. The computing power resource settlement method of claim 1, wherein the step of allocating the current computing task to the target server using the computing resource pool comprises:
determining workflow data of a current computing task by utilizing a computing resource pool;
dividing the workflow data into a plurality of work units by using a main server;
and distributing the working units to the corresponding target servers.
4. The computing power resource settlement method of claim 3, wherein said step of assigning said work units to corresponding target servers comprises:
determining the task type of the working unit;
and distributing the working unit to a target server corresponding to the task type.
5. The computing power resource settlement method of claim 3, wherein said step of assigning said work units to corresponding target servers further comprises:
load information of a plurality of servers is obtained;
determining a target server in a plurality of servers according to the load information;
and distributing the working units to the corresponding target servers.
6. The computing power resource settlement method of claim 5, wherein said step of determining a target server of a plurality of servers from said load information comprises:
filtering servers with load information smaller than a preset load threshold to determine a target server in a plurality of servers; or (b)
Determining the distribution priority among a plurality of servers according to the load information;
a target server of the plurality of servers is determined based on the allocation priority.
7. The method of settlement of computing power resources of claim 1, wherein the settlement rules include resource unit price; the step of determining the real-time resource cost of the current computing task based on the running time length, the resource occupancy rate and the settlement rule comprises the following steps:
and determining a product result obtained by multiplying the running time, the resource occupancy rate and the resource unit price as the real-time resource cost of the current computing task.
8. An apparatus for settling a computing power resource, the apparatus comprising:
the task allocation module is used for allocating the current computing task to the target server by utilizing the computing resource pool;
the task monitoring module is used for acquiring the running time of the current calculation task, the resource occupancy rate and the settlement rule of the target server;
and the expense settlement module is used for determining the real-time resource expense of the current calculation task based on the operation time length, the resource occupancy rate and the settlement rule.
9. A computing power resource settlement device comprising a processor, a memory, and a computing power resource settlement program stored on the memory that is executable by the processor, wherein the computing power resource settlement program, when executed by the processor, implements the steps of the computing power resource settlement method according to any one of claims 1 to 7.
10. A computer readable storage medium, wherein a computing resource settlement program is stored on the computer readable storage medium, wherein the computing resource settlement program, when executed by a processor, implements the steps of the computing resource settlement method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311454301.7A CN117573335A (en) | 2023-11-03 | 2023-11-03 | Method, device, equipment and computer readable storage medium for settling account of computing power resources |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311454301.7A CN117573335A (en) | 2023-11-03 | 2023-11-03 | Method, device, equipment and computer readable storage medium for settling account of computing power resources |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117573335A true CN117573335A (en) | 2024-02-20 |
Family
ID=89890799
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311454301.7A Pending CN117573335A (en) | 2023-11-03 | 2023-11-03 | Method, device, equipment and computer readable storage medium for settling account of computing power resources |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117573335A (en) |
-
2023
- 2023-11-03 CN CN202311454301.7A patent/CN117573335A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Narayanan et al. | {Heterogeneity-Aware} cluster scheduling policies for deep learning workloads | |
| Van den Bossche et al. | Online cost-efficient scheduling of deadline-constrained workloads on hybrid clouds | |
| Zhu et al. | Scheduling stochastic multi-stage jobs to elastic hybrid cloud resources | |
| CN114741207B (en) | GPU resource scheduling method and system based on multi-dimensional combination parallelism | |
| CN109117252B (en) | Method and system for task processing based on container and container cluster management system | |
| CN112685153A (en) | Micro-service scheduling method and device and electronic equipment | |
| US20240111586A1 (en) | Multi-policy intelligent scheduling method and apparatus oriented to heterogeneous computing power | |
| CN109857535B (en) | Spark JDBC-oriented task priority control implementation method and device | |
| CN109254836B (en) | Deadline constraint cost optimization scheduling method for priority dependent tasks of cloud computing system | |
| CN115543615A (en) | Resource allocation method and device, electronic equipment and storage medium | |
| CN113946431B (en) | Resource scheduling method, system, medium and computing device | |
| CN111861412A (en) | Completion time optimization-oriented scientific workflow scheduling method and system | |
| CN115134371A (en) | Scheduling method, system, equipment and medium containing edge network computing resources | |
| CN115934362B (en) | Deep learning-oriented server non-perception computing cluster scheduling method and product | |
| CN115292016A (en) | Task scheduling method based on artificial intelligence and related equipment | |
| Singh et al. | Scheduling algorithm with load balancing in cloud computing | |
| CN113407343A (en) | Service processing method, device and equipment based on resource allocation | |
| Choi et al. | Gpsf: general-purpose scheduling framework for container based on cloud environment | |
| CN116820714A (en) | Scheduling method, device, equipment and storage medium of computing equipment | |
| CN111597035A (en) | Simulation engine time advancing method and system based on multiple threads | |
| Cui et al. | A scheduling algorithm for multi-tenants instance-intensive workflows | |
| CN115952054A (en) | Simulation task resource management method, device, equipment and medium | |
| CN117573335A (en) | Method, device, equipment and computer readable storage medium for settling account of computing power resources | |
| CN108429704B (en) | Node resource allocation method and device | |
| Awasare et al. | Survey and comparative study on resource allocation strategies in cloud computing environment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |