CN114661462A - Resource allocation method, system, computer readable storage medium and electronic device - Google Patents

Resource allocation method, system, computer readable storage medium and electronic device Download PDF

Info

Publication number
CN114661462A
CN114661462A CN202210212325.0A CN202210212325A CN114661462A CN 114661462 A CN114661462 A CN 114661462A CN 202210212325 A CN202210212325 A CN 202210212325A CN 114661462 A CN114661462 A CN 114661462A
Authority
CN
China
Prior art keywords
target
job
computing
computing resource
resource
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210212325.0A
Other languages
Chinese (zh)
Inventor
王敏
贺荣徽
何万青
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Alibaba China Co Ltd
Original Assignee
Alibaba China Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Alibaba China Co Ltd filed Critical Alibaba China Co Ltd
Priority to CN202210212325.0A priority Critical patent/CN114661462A/en
Publication of CN114661462A publication Critical patent/CN114661462A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5061—Partitioning or combining of resources
    • G06F9/5072—Grid computing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/48—Program initiating; Program switching, e.g. by interrupt
    • G06F9/4806—Task transfer initiation or dispatching
    • G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
    • G06F9/4881—Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
    • G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
    • G06F9/5038—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals considering the execution order of a plurality of tasks, e.g. taking priority or time dependency constraints into consideration
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
    • G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
    • G06F9/5044—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals considering hardware capabilities
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
    • G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
    • G06F9/5055—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals considering software capabilities, i.e. software resources associated or available to the machine

Abstract

The application discloses a resource allocation method, a system, a computer readable storage medium and an electronic device. Wherein, the method comprises the following steps: acquiring operation characteristic information of target operation to be executed by a target computing cluster; determining a target computing resource type based on the job characteristic information; and allocating target computing resources for the target computing cluster, wherein the target computing resources are computing resources required by the target computing cluster to execute the target job, and the computing resource type of the target computing resources is the target computing resource type. The method and the device solve the technical problem of unreasonable resource allocation caused by selecting similar computing resources for all jobs in the prior art.

Description

Resource allocation method, system, computer readable storage medium and electronic device
Technical Field
The present application relates to the field of artificial intelligence, and in particular, to a resource allocation method, system, computer-readable storage medium, and electronic device.
Background
The cloud computing platform is also called a cloud platform, and is a service based on hardware resources and software resources, and provides computing, network and storage capabilities. With the development of cloud computing and artificial intelligence, the cloud on a cloud computing service platform is more and more in demand, the computing specification is also required to be diversified, and the scale of a single cluster is also larger and larger.
At present, in the process of processing the operation, the related cloud computing service platform selects the same kind of computing resources for all the operations to perform unified processing, so that the resource allocation is unreasonable, and the advantages of the computing resources on the cloud cannot be exerted.
In view of the above problems, no effective solution has been proposed.
Disclosure of Invention
Embodiments of the present application provide a resource allocation method, a system, a computer-readable storage medium, and an electronic device, so as to at least solve the technical problem in the prior art that resource allocation is unreasonable because similar computing resources are selected for all jobs.
According to an aspect of an embodiment of the present application, there is provided a resource allocation method, including: acquiring operation characteristic information of a target operation to be executed by a target computing cluster; determining a target computing resource type based on the job characteristic information; and allocating target computing resources for the target computing cluster, wherein the target computing resources are computing resources required by the target computing cluster to execute the target job, and the computing resource type of the target computing resources is the target computing resource type.
According to another aspect of the embodiments of the present application, there is also provided a resource allocation method, including: responding to a job creating instruction, creating a target job and a target computing cluster for executing the target job, and displaying job characteristic information of the target job; displaying a target computing resource type determined based on the job characteristic information; and responding to the resource allocation instruction, and displaying the target computing resources allocated to the target computing cluster, wherein the target computing resources are computing resources required by the target computing cluster to execute the target job, and the computing resource type of the target computing resources is the target computing resource type.
According to another aspect of the embodiments of the present application, there is also provided a resource allocation apparatus, including: the acquisition module is used for acquiring the operation characteristic information of the target operation to be executed by the target computing cluster; a determination module to determine a target computing resource type based on the job characteristic information; and the allocation module is used for allocating target computing resources to the target computing cluster, wherein the target computing resources are computing resources required by the target computing cluster to execute the target job, and the computing resource types of the target computing resources are target computing resource types.
According to another aspect of the embodiments of the present application, there is also provided a resource allocation apparatus, including: the first response module is used for responding to the job creating instruction, creating a target job and executing a target computing cluster of the target job, and displaying job characteristic information of the target job; the display module is used for displaying the target computing resource type determined based on the job characteristic information; and the second response module is used for responding to the resource allocation instruction and displaying the target computing resources allocated to the target computing cluster, wherein the target computing resources are computing resources required by the target computing cluster to execute the target job, and the computing resource types of the target computing resources are target computing resource types.
According to another aspect of the embodiments of the present application, there is also provided a resource allocation system, including: a scheduler for receiving a target job submitted by a target object; the characteristic collection component is used for acquiring the job characteristic information of a target job to be executed by the target computing cluster and determining the type of the target computing resource based on the job characteristic information; the resource management component is used for determining target computing resources according to the types of the target computing resources, creating a task queue for the scheduler, and then adding the target computing resources to the scheduler based on the task queue; the scheduler also allocates the target computing resource to a task queue corresponding to the computing node of the target computing cluster based on a preset scheduling strategy so that the computing node in the target computing cluster executes the target job.
According to another aspect of the embodiments of the present application, there is also provided a computer-readable storage medium, in which a computer program is stored, where the computer program is configured to execute the above resource allocation method when running.
According to another aspect of embodiments of the present application, there is also provided an electronic device, including one or more processors; a memory for storing one or more programs which, when executed by the one or more processors, cause the one or more processors to carry out the method for executing a program, wherein the program is arranged to carry out the method for allocating resources as described above when executed.
In the embodiment of the invention, a mode of allocating different types of computing resources to each job based on job characteristic information of the job is adopted, and the target computing resources are allocated to the target computing cluster by acquiring the job characteristic information of the target job to be executed by the target computing cluster and then determining the type of the target computing resources based on the job characteristic information, wherein the target computing resources are computing resources required by the target computing cluster to execute the target job, and the type of the computing resources of the target computing resources is the type of the target computing resources.
In the process, the target computing resources are distributed for the target computing cluster, processing of all jobs based on different types of computing resources is achieved, and the phenomenon that resources are wasted or insufficient when similar computing resources are selected for all jobs is avoided, so that the reasonability of resource distribution is improved, the operation efficiency is improved, and the user experience is improved. In addition, in the application, the target computing resource type is determined based on the job characteristic information, so that the computing resource type of the most appropriate computing resource for processing each target job is determined, the difference and inaccuracy of manual selection of computing resources are avoided, and the job operation efficiency is further improved.
Therefore, the scheme provided by the application achieves the purpose of distributing different types of computing resources for each job based on the job characteristic information of the job, thereby realizing the technical effect of improving the rationality of resource distribution and further solving the technical problem of unreasonable resource distribution caused by selecting similar computing resources for all jobs in the prior art.
Drawings
The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the application and together with the description serve to explain the application and not to limit the application. In the drawings:
FIG. 1 is a block diagram of an alternative electronic device (or mobile device) according to an embodiment of the present application;
fig. 2 is an embodiment of an alternative electronic device (or mobile device) as a transmitting end according to an embodiment of the present application;
FIG. 3 is a flow chart of an alternative resource allocation method according to an embodiment of the present application;
FIG. 4 is a schematic diagram of a resource allocation method according to the prior art;
FIG. 5 is a schematic diagram of a resource allocation method according to the prior art;
FIG. 6 is a timing diagram of an alternative resource allocation method according to an embodiment of the present application;
FIG. 7 is a schematic view of an alternative feature collection assembly according to an embodiment of the present application;
FIG. 8 is a flow chart of an alternative resource allocation method according to an embodiment of the present application;
FIG. 9 is a schematic diagram of an alternative human-computer interaction operation according to an embodiment of the application;
FIG. 10 is a schematic diagram of an alternative resource allocation apparatus according to an embodiment of the present application;
FIG. 11 is a schematic diagram of an alternative resource allocation apparatus according to an embodiment of the present application;
FIG. 12 is a schematic diagram of an alternative resource allocation system according to an embodiment of the present application;
fig. 13 is a block diagram of an alternative electronic device according to an embodiment of the present application.
Detailed Description
In order to make the technical solutions better understood by those skilled in the art, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only partial embodiments of the present application, but not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
It should be noted that the terms "first," "second," and the like in the description and claims of this application and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It should be understood that the data so used may be interchanged under appropriate circumstances such that embodiments of the application described herein may be implemented in sequences other than those illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
Example 1
There is also provided, in accordance with an embodiment of the present application, an embodiment of a method for resource allocation, where it is noted that the steps illustrated in the flowchart of the drawings may be performed in a computer system such as a set of computer-executable instructions, and that while a logical order is illustrated in the flowchart, in some cases the steps illustrated or described may be performed in an order different than here.
The method provided by the first embodiment of the present application may be executed in a mobile terminal, an electronic device, or a similar computing device. Fig. 1 shows a hardware configuration block diagram of an electronic device (or mobile device) for implementing the resource allocation method. As shown in fig. 1, the electronic device 10 (or mobile device 10) may include one or more processors (shown as 102a, 102b, … …, 102n in the figures) which may include, but are not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA, a memory 104 for storing data, and a transmission module 106 for communication functions. Besides, the method can also comprise the following steps: a display, an input/output interface (I/O interface), a Universal Serial BUS (USB) port (which may be included as one of the ports of the BUS), a network interface, a power source, and/or a camera. It will be understood by those skilled in the art that the structure shown in fig. 1 is only an illustration and is not intended to limit the structure of the electronic device. For example, electronic device 10 may also include more or fewer components than shown in FIG. 1, or have a different configuration than shown in FIG. 1.
It should be noted that the one or more processors and/or other data processing circuitry described above may be referred to generally herein as "data processing circuitry". The data processing circuitry may be embodied in whole or in part in software, hardware, firmware, or any combination thereof. Further, the data processing circuitry may be a single, stand-alone processing module, or incorporated in whole or in part into any of the other elements in the electronic device 10 (or mobile device). As referred to in the embodiments of the application, the data processing circuit acts as a processor control (e.g. selection of a variable resistance termination path connected to the interface).
The memory 104 may be used to store software programs and modules of application software, such as program instructions/data storage devices corresponding to the resource allocation method in the embodiment of the present application, and the processor executes various functional applications and data processing by running the software programs and modules stored in the memory 104, so as to implement the above-mentioned resource allocation method. The memory 104 may include high speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include memory remotely located from the processor, which may be connected to the electronic device 10 over a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The transmission device 106 is used for receiving or transmitting data via a network. Specific examples of the network described above may include a wireless network provided by a communication provider of the electronic device 10. In one example, the transmission device 106 includes a Network adapter (NIC) that can be connected to other Network devices through a base station to communicate with the internet. In one example, the transmission device 106 can be a Radio Frequency (RF) module, which is used to communicate with the internet in a wireless manner.
The display may be, for example, a touch screen type Liquid Crystal Display (LCD) that may enable a user to interact with a user interface of the electronic device 10 (or mobile device).
Fig. 1 shows a block diagram of a hardware structure, which may be taken as an exemplary block diagram of not only the electronic device 10 (or the mobile device) described above, but also an exemplary block diagram of the server described above, and in an alternative embodiment, fig. 2 shows an embodiment that uses the electronic device 10 (or the mobile device) shown in fig. 1 as a sending end in a block diagram. As shown in fig. 2, the electronic device 10 (or mobile device) may be connected or electronically connected to one or more servers 108 via a data network, wherein the aforementioned servers may be servers in the same server cluster, and optionally, the server cluster may be a resource cluster, a computing cluster, or the like server cluster, and the server may be a resource server, a computing server, a security server, or a game server, or the like server. In an alternative embodiment, the electronic device 10 (or mobile device) may be a mobile phone, a tablet, a computer, or a smart wearable device. The data network connection may be a local area network connection, a wide area network connection, an internet connection, or other type of data network connection. The electronic device 10 (or mobile device) may execute to connect to a network service executed by a server (e.g., a computing server) or a group of servers. The web server 110 is a web-based user service, such as social networking, cloud resources, email, online payment, or other online application, and may also be a cloud server.
Under the operating environment, the application provides a resource allocation method as shown in fig. 3. Fig. 3 is a flowchart of an alternative resource allocation method according to an embodiment of the present application. In the present application, a resource allocation system is taken as an execution subject of the embodiment, and the resource allocation system at least includes a resource scheduling component, a feature collecting component, and a resource management component, where the resource scheduling component, the feature collecting component, and the resource management component are connected to each other respectively.
Step S202, acquiring the job characteristic information of the target job to be executed by the target computing cluster.
In step S202, job feature information of a target job to be executed by a target computing cluster may be acquired by an electronic device, a server, an application system, or the like. The resource allocation system can generate a target job based on the expected operations such as query vocabulary entry, browsing page, switching program and the like executed by a user in the target application on the terminal equipment such as a mobile phone, a computer, a tablet and the like, and then the feature extraction component extracts the job feature information of the target job. Optionally, the job characteristic information of the target job at least includes job information and system characteristic information; the target Computing cluster may serve as a cloud Computing service platform, and the target Computing cluster may be an HPC (High Performance Computing) cluster, which may be created by a user based on a resource allocation system before the target Computing resource is allocated, and includes a plurality of Computing nodes, each Computing node being a server.
It should be noted that, by obtaining the job characteristic information, it is possible to facilitate subsequent determination of the target computing resource type corresponding to the target job.
In step S204, a target computing resource type is determined based on the job characteristic information.
In step S204, the job feature information may be compared with preset feature information in a feature database, the internet, a cloud server or other storage areas based on the feature collection component, so as to determine a target computing resource type according to a computing resource type corresponding to the preset feature information that most matches the job feature information of the target job. Optionally, the feature collection component may also compare the job feature information with each preset computing resource type directly, so as to determine the target computing resource type according to the comparison result. Optionally, the feature collection component may further input the job feature information into a trained web learning model to determine the target computing resource type based on the output of the web learning model.
It should be noted that, by determining the target computing resource type based on the job feature information, the determination of the computing resource type of the computing resource which is relatively most suitable for processing each target job is realized, and the difference and inaccuracy of manually selecting the computing resource are avoided, so that each job is processed based on different types of computing resources in the following process, and the processing efficiency of each job is ensured.
Step S206, allocating target computing resources to the target computing cluster, where the target computing resources are computing resources required by the target computing cluster to execute the target job, and the computing resource types of the target computing resources are target computing resource types.
In step S206, target computing resources may be allocated for the target computing cluster based on a resource scheduling component in the resource allocation system. Optionally, before allocating the target computing resource to the target computing cluster, the resource management component in the resource allocation system may query, based on the type of the target computing resource, the computing resource corresponding to the type of the target computing resource in a resource database, the internet, a cloud server, or another storage area, so as to send the queried computing resource as the target computing resource to the resource scheduling component, so that the resource scheduling component allocates the target computing resource to the target computing cluster.
It should be noted that, by allocating target computing resources to the target computing cluster, processing of each job based on different types of computing resources is achieved, and a phenomenon of resource waste or shortage in selecting similar computing resources for all jobs is avoided, so that rationality of resource allocation is improved, job operation efficiency is improved, and user experience is improved.
In the prior art, there is an automatic scaling scheme on a public cloud, as shown in fig. 4, which performs automatic scaling according to overall loads such as a CPU and a memory to adjust the computing scale of a computing cluster. Similarly, in another automated scaling scheme for HPC solutions, which is bound to a single HPC cluster, as shown in fig. 5, the computing resources are scaled to the cloud for the load of the HPC cluster to adjust the computing size of the computing cluster. However, in the two schemes, the scheduler resource elasticity of the HPC cluster is determined according to a system workload or a job queue, that is, the foregoing process is only general processing according to the job queue, and often processes all jobs uniformly, and selects similar computing resources, and does not consider the demands of the application characteristics of the jobs on the computing resources, and does not collect and analyze the job characteristics, and selects the computing resources in a linkage manner according to the analysis result, thereby causing an unreasonable allocation problem.
Based on the schemes defined in steps S202 to S206, it can be known that, in the embodiment of the present invention, a manner of allocating different types of computing resources to each job based on job feature information of the job is adopted, and a target computing resource type is determined based on job feature information by obtaining job feature information of a target job to be executed by a target computing cluster, so as to allocate a target computing resource to the target computing cluster, where the target computing resource is a computing resource required by the target computing cluster to execute the target job, and the computing resource type of the target computing resource is the target computing resource type.
It is easy to notice that in the above process, the target computing resources are allocated to the target computing cluster, so that each job is processed based on different types of computing resources, and the phenomenon of resource waste or shortage in selecting similar computing resources for all jobs is avoided, thereby improving the rationality of resource allocation, further improving the job operation efficiency, and improving the user experience. In addition, in the application, the target computing resource type is determined based on the job characteristic information, so that the computing resource type of the most appropriate computing resource for processing each target job is determined, the difference and inaccuracy of manually selecting the computing resource are avoided, and the job operation efficiency is further improved.
Therefore, the scheme provided by the application achieves the purpose of distributing different types of computing resources for each job based on the job characteristic information of the job, thereby realizing the technical effect of improving the rationality of resource distribution and further solving the technical problem of unreasonable resource distribution caused by selecting the same type of computing resources for all jobs in the prior art.
In an optional embodiment, in the process of obtaining job feature information of a target job to be executed by the target computing cluster, the feature collection component may obtain job information of the target job from a job list corresponding to the resource scheduling component, and then determine system feature information corresponding to the target job based on the target computing cluster. The resource scheduling component is configured to receive a target job submitted by a target object, and allocate the target job to a computing node of a target computing cluster, where the job information at least includes: container dependency information of target operation running time, and system characteristic information at least comprises: and calculating the network bandwidth and the memory occupancy rate of the nodes.
Optionally, as shown in fig. 6, after the resource scheduling component acquires the target job submitted by the user, the resource scheduling component may allocate the target job to the computing node of the target computing cluster. Then, the feature collection component may obtain job information of the target job from a job list corresponding to the resource scheduling component, where the job information at least includes container dependency information and job name when the target job runs, job detail information, software information, a queue, and the like, where the job detail information may include information such as size and content of the target job. The feature collection component may then collect operational data of each computing node in the target computing cluster to obtain system feature information to determine job feature information based on the job information and the system feature information. The operation data represents the operation condition of each computing node in the target computing cluster, and the system characteristic information at least comprises information such as network bandwidth and memory occupancy rate of the computing nodes.
It should be noted that, by acquiring the job information and the system characteristic information of the target job, the job characteristic information of the target job is accurately determined, so that the target computing resource type determined subsequently based on the job characteristic information is more accurate, thereby efficiently using computing resources and improving job operation efficiency.
In an alternative embodiment, in determining a target computing resource type based on job characterization information, the characterization collection component may perform a computing resource type query from a pre-defined characterization database based on the job characterization information, obtain a query result, and when the query result represents that the computing resource type corresponding to the operation characteristic information of the target operation exists in the preset characteristic database, determining the computing resource type corresponding to the operation characteristic information of the target operation in the preset characteristic database as a target computing resource type, when the query result represents that the computing resource type corresponding to the operation characteristic information of the target operation does not exist in the preset characteristic database, and comparing the operation characteristic information of the target operation with target operation characteristic information corresponding to a preset computing resource type, and determining the target computing resource type according to the comparison result. The preset characteristic database at least stores job characteristic information and a calculation resource type of historical jobs, and an association relation between the job characteristic information and the calculation resource type of the historical jobs, wherein the calculation resource type is a type of calculation resources required by a target calculation cluster to execute the historical jobs.
Optionally, as shown in fig. 7, the feature collection component may compare the job feature information of the currently running target job with the job feature information of all historical jobs in the preset feature database to determine the job feature information of the historical jobs whose similarity to the job feature information of the target job exceeds a preset threshold, so as to obtain the query result. Optionally, the feature collection component may also compare the job feature information of the currently running target job with the job feature information of the historical jobs in the same industry in the preset feature database, so as to more quickly determine the job feature information of the historical jobs similar to the job feature information of the target job.
Further, when the query result indicates that the computing resource type corresponding to the job feature information of the target job exists in the preset feature database, that is, when the similarity between the job feature information of the historical job and the job feature information of the target job exists in the preset feature database exceeds a threshold, the feature collection component may determine the computing resource type corresponding to the job feature information of the historical job with the highest similarity to the job feature information of the target job as the computing resource type corresponding to the job feature information of the target job.
When the query result indicates that the computing resource type corresponding to the operation feature information of the target operation does not exist in the preset feature database, that is, when the similarity between the operation feature information of all historical operations in the preset feature database and the operation feature information of the target operation does not exceed the threshold, the feature collection component can compare the operation feature information of the target operation with the target operation feature information corresponding to the preset computing resource type, and determine the target computing resource type according to the comparison result. The preset computing resource types at least comprise a containerized instance computing resource type, a super computing cluster computing resource type, a physical machine computing resource type and a virtual machine computing resource type.
It should be noted that, by determining the target computing resource type by using the job feature information of the historical job and the computing resource type, a more accurate determination of the target computing resource type corresponding to each target job is achieved. In addition, under the condition that the target computing resource type cannot be determined based on the historical operation, the target computing resource type is determined based on the preset target operation characteristic information, so that the applicability of the application is improved, and the phenomenon that the target computing resource type corresponding to the target operation cannot be acquired is avoided.
In an optional embodiment, in the process of comparing the job characteristic information of the target job with target job characteristic information corresponding to a preset computing resource type and determining the target computing resource type according to the comparison result, the characteristic collection component may determine the job running form of the target job according to container dependency information of the target job in running, and determine the target computing resource type as a containerized instance computing resource type when the job running form characterizes that the target job runs in a container, wherein the containerized instance computing resource type characterizes that the target job runs in dependency on the container.
Alternatively, as shown in fig. 7, the feature collection component may determine the running form of the target job according to the container dependency information of the target job running, and determine whether the target job runs in a container based on the running form of the target job, that is, whether the target job runs in a container, in this embodiment, the container may be a docker or a singleton or other container. The docker is an open-source application container engine, which enables developers to package their applications and dependency packages into a portable mirror image, and then to publish the image to any operating system machine, so as to implement virtualization; singulate is an open source container platform, specifically one optimized for HPC workloads, allowing untrusted users to run untrusted containers in a trusted manner. Further, the feature collection component may determine that the target computing resource type is a containerized instance computing resource type when the run form of the target job characterizes the target job as running in the container.
It should be noted that, determining whether the target computing resource type is a containerized instance computing resource type based on the container dependency information can accurately determine the association relationship between the target computing resource type and the containerized instance computing resource type, thereby facilitating acquisition of the accurate target computing resource type.
In an optional embodiment, when the job running form indicates that the target job is not running in the container, the feature collection component may detect whether a real-time maximum network bandwidth of the computing node is greater than a first preset bandwidth, and whether an average bandwidth corresponding to the computing node is greater than a second preset bandwidth, and determine that the target computing resource type is a super computing cluster computing resource type when the real-time maximum network bandwidth is greater than the first preset bandwidth or the average bandwidth is greater than the second preset bandwidth, where the super computing cluster computing resource type indicates that the running of the target job is influenced by the network bandwidth to the greatest extent.
Optionally, as shown in fig. 7, when the job running form indicates that the target job is not running in the container, the feature collection component may monitor the network bandwidth of the computing node corresponding to the target job, and detect whether a real-time maximum bandwidth (i.e., a real-time maximum network bandwidth) of a network bottleneck of the computing node corresponding to the target job is greater than a first preset bandwidth and whether an average bandwidth (i.e., an average bandwidth) of a network load pressure of the computing node corresponding to the target job is greater than a second preset bandwidth. When the real-time maximum network bandwidth of the computing node is greater than a first preset bandwidth a (i.e., a threshold a in fig. 7) or the average bandwidth is greater than a second preset bandwidth b (i.e., a threshold b in fig. 7), the feature collection component may determine that the target computing resource type corresponding to the target job is the super computing cluster computing resource type.
It should be noted that, determining whether the target computing resource type is the super computing cluster computing resource type based on the real-time maximum network bandwidth and the average bandwidth of the computing node corresponding to the target job can accurately determine the association relationship between the target computing resource type and the super computing cluster computing resource type, thereby facilitating acquisition of the accurate target computing resource type.
In an alternative embodiment, when the real-time maximum network bandwidth is less than or equal to a first preset bandwidth and the average bandwidth is less than or equal to a second preset bandwidth, the feature collection component may detect whether the memory occupancy rate of the computing node is greater than a preset occupancy rate, and determine that the target computing resource type is the physical computer computing resource type when the memory occupancy rate is greater than the preset occupancy rate, and determine that the target computing resource type is the virtual computer computing resource type when the memory occupancy rate is less than or equal to the preset occupancy rate.
Optionally, as shown in fig. 7, when the real-time maximum network bandwidth is less than or equal to the first preset bandwidth and the average bandwidth is less than or equal to the second preset bandwidth, the feature collection component may monitor a Central Processing Unit (CPU) occupancy of the computing node corresponding to the target job, that is, a memory occupancy, and detect whether the CPU occupancy is greater than the preset occupancy. Optionally, the feature collection component may detect whether the average occupancy rate of the CPU is greater than a preset occupancy rate, or may detect whether the maximum occupancy rate of the CPU is greater than the preset occupancy rate.
Further, the feature collection component may determine that the target computing resource type is a physical machine computing resource type when the memory occupancy is greater than a preset occupancy. Conversely, when the memory occupancy is less than or equal to the preset occupancy, the feature collection component may determine that the target computing resource type is a VM (Virtual Machine) computing resource type.
It should be noted that, determining whether the target computing resource type is a physical computer computing resource type or a virtual machine computing resource type based on the memory occupancy rate corresponding to the target job can accurately determine the association relationship between the target computing resource type and the physical computer computing resource type and the virtual machine computing resource type, thereby facilitating acquisition of an accurate target computing resource type.
It is emphasized that the foregoing determination of whether a target computing resource type is a containerized instance computing resource type, or a supercomputing cluster computing resource type, or a physical computer computing resource type, or a virtual machine computing resource type may be ordered based on actual demand.
In an optional embodiment, after comparing the job feature information of the target job with the target job feature information corresponding to the preset computing resource type and determining the target computing resource type according to the comparison result, the feature collection component may store the job detail information and the system feature information corresponding to the target job into the preset feature database.
Optionally, as shown in fig. 7, the feature collection component may store job detail information and system feature information corresponding to the target job and an association relationship between the target job and the target computing resource type in a preset feature database, so as to enrich data in the preset feature database and improve diversity of data in the feature database, so that when the feature collection component acquires job feature information of a new target job next time, the corresponding target computing resource type may be acquired more quickly. The job detail information at least includes information such as size and content corresponding to the target job.
In an optional embodiment, in the process of allocating the target computing resource to the target computing cluster, the resource allocation system may apply for the target computing resource corresponding to the target computing resource type from the cloud server based on the resource management component, then create a task queue to the resource scheduling component based on the resource management component, and add the target computing resource to the resource scheduling component based on the task queue, so that the target computing resource is allocated to the task queue corresponding to the computing node of the target computing cluster based on the scheduling policy corresponding to the resource scheduling component. The resource management component is used for managing the life cycle of the computing resources required by the target operation, and the cloud server is used for carrying out production management on the computing resources required by the target operation.
Specifically, as shown in FIGS. 6 and 7, when the feature collection component determines the target computing resource type, the feature collection component sends a resource production request to the resource management component and sends the target computing resource type to the resource management component. Optionally, the feature collection component may also send the target computing resource type to the resource management component after determining the target computing resource type and receiving the message of the target computing cluster extension. And then, applying a target computing resource corresponding to the target computing resource type to a cloud server by a resource management component, wherein the cloud server at least stores the computing resource corresponding to the containerized instance computing resource type, the computing resource corresponding to the super computing cluster computing resource type, the computing resource corresponding to the physical computer computing resource type and the computing resource corresponding to the virtual computer computing resource type.
Further, as shown in fig. 6, after the target computing resource application is successful, the resource management component may create a task queue to the resource scheduling component, add the target computing resource to the resource scheduling component based on the task queue, and maintain the life cycle of all the computing resources in the target computing cluster. At least before adding the target computing resource to the resource scheduling component based on the task queue, the resource management component needs to create a target computing cluster, that is, a computing node is added to the resource scheduling component, so as to ensure that the job can be successfully run.
Further, the resource scheduling component may allocate the target computing resources and the target jobs to the task queues corresponding to the computing nodes of the target computing cluster based on its own scheduling policy, so that the target computing cluster can run the corresponding target jobs based on the respective target computing resources.
It should be noted that, by creating a task queue to the resource scheduling component based on the resource management component and allocating a corresponding task queue to the computing node of the target computing cluster based on the resource scheduling component, the target computing cluster can process the corresponding target job based on each target computing resource, thereby avoiding the phenomenon of resource waste or shortage in selecting similar computing resources for all jobs, and improving the rationality of resource allocation.
In an alternative embodiment, after allocating the target computing resource for the target computing cluster and after the resource management component detects that the resource scheduling component completes the target job, the resource management component may remove the target computing resource from the task queue and send a resource release message to the cloud server to release the target computing resource.
Optionally, as shown in fig. 6, when any target job in the resource scheduling component is executed, the resource scheduling component may send information of the execution completion to the resource management component, or the resource management component may obtain related information from the resource scheduling component, so that the resource management component determines that the resource scheduling component completes the target job. Thereafter, the resource management component may remove the target computing resource corresponding to the completed target job from the task queue, and send a resource release message to the cloud server to release the target computing resource.
It should be noted that, by releasing the target computing resource corresponding to the target job after the target job is completed, the situation that the target computing resource cannot be used successfully when the target computing resource needs to be used next time can be avoided, so that the utilization rate of each computing resource is improved.
In an alternative embodiment, after determining the target computing resource type based on the job characteristic information, the resource management component may create a plurality of task queues according to the job characteristic information of the plurality of target jobs when the number of target jobs is multiple, and determine the target computing resource corresponding to each target job according to the computing resource type corresponding to the plurality of task queues, so as to execute the plurality of target jobs based on the target computing resource. Wherein each task queue corresponds to a computing resource of a different computing resource type.
Specifically, when there are a plurality of target jobs, the resource management component may create a plurality of task queues based on categories of job feature information of the plurality of target jobs, that is, job feature information with relatively high similarity is regarded as the same category, job feature information with relatively low similarity is regarded as a different category, and the number of task queues is equal to the number of categories of job feature information. Each task queue corresponds to a computing resource of a different computing resource type, so that the target computing resource corresponding to each target job can be determined based on the computing resource types corresponding to the multiple task queues, and the corresponding target job can be executed based on the target computing resource. Optionally, the resource management component may determine the job characteristic information of a plurality of target jobs based on the foregoing method, or may substantially determine the industry nature and characteristics of each target job based on tests and experiences, and use them as job characteristic information.
By creating a plurality of task queues according to the job feature information of a plurality of target jobs and determining target computing resources based on the plurality of task queues, the work efficiency and the running efficiency of each target job can be improved when the number of the target jobs is large.
It should be noted that, in the application, the expansion or elasticity requirement is fully considered, the resource decision is made according to the operation characteristic information of the target operation, and the computing resource is automatically released after the target operation is executed, so that the phenomena that the cloud timeliness cannot be reflected by statically distributed computing resources, the resource waste or deficiency exists, and the scene that the operation needs elasticity cannot be met in the prior art are avoided. Meanwhile, the computing resource type is decided based on the operation characteristic information, so that the difference caused by manual selection of computing resources can be avoided, and the user experience is improved.
Therefore, the scheme provided by the application achieves the purpose of distributing different types of computing resources for each job based on the job characteristic information of the job, thereby realizing the technical effect of improving the rationality of resource distribution and further solving the technical problem of unreasonable resource distribution caused by selecting the same type of computing resources for all jobs in the prior art.
It should be noted that, for simplicity of description, the above-mentioned method embodiments are described as a series of acts or combination of acts, but those skilled in the art will recognize that the present application is not limited by the order of acts described, as some steps may occur in other orders or concurrently depending on the application. Further, those skilled in the art should also appreciate that the embodiments described in the specification are preferred embodiments and that the acts and modules referred to are not necessarily required in this application.
Through the above description of the embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but the former is a better implementation mode in many cases. Based on such understanding, the technical solutions of the present application may be embodied in the form of a software product, which is stored in a storage medium (e.g., ROM/RAM, magnetic disk, optical disk) and includes instructions for enabling a terminal device (e.g., a mobile phone, a computer, a server, or a network device) to execute the method of the embodiments of the present application.
Example 2
Under the operating environment, the present application provides a resource allocation method as shown in fig. 8. Fig. 8 is a flowchart of an alternative resource allocation method according to an embodiment of the present application.
Step S302, in response to the job creation instruction, creates a target job and a target computing cluster for executing the target job, and displays job feature information of the target job.
In step S302, as shown in fig. 9, the aforementioned resource allocation system may create a target job in response to a job creation instruction input by a user, and then may acquire the target job by the resource scheduling component and acquire the target job from the resource scheduling component by the feature collection component to collect job feature information of the target job. Meanwhile, the resource management component in the resource allocation system can also create a target computing cluster for executing the target job.
Further, after the feature collection component finishes collecting the job feature information of the target job, the resource allocation system can display the job feature information of the target job to a background operator or a user through a display interface so as to be viewed by the background operator or the user.
It should be noted that, by responding to the job creation instruction of the user, the target job and the target computing cluster are created to ensure normal feedback on the operation of the user. Meanwhile, by displaying the operation characteristic information, a background operator can better supervise the resource distribution system, and target operation execution errors caused by wrong instructions or wrong operation characteristic information are prevented, so that the stability of target operation execution is improved. The erroneous job characteristic information may be blank job characteristic information or the like.
Step S304, displaying the target computing resource type determined based on the job characteristic information.
In step S304, as shown in fig. 9, after the feature collection component determines the target computing resource type based on the aforementioned method, the resource allocation system may display each target job and its corresponding target computing resource type to a background operator or user through the display interface.
It should be noted that, by displaying the target computing resource type, a background operator can further monitor the resource allocation system, and prevent a target job and a target computing resource type from being incorrectly paired, or a target job execution error caused by the wrong target computing resource type, thereby improving the stability of target job execution. The wrong target computing resource type may be a blank computing resource type or a computing resource type that does not exist in a storage area of a cloud server, a database, or the like.
Step S306, responding to the resource allocation instruction, and displaying the target computing resource allocated to the target computing cluster, where the target computing resource is a computing resource required by the target computing cluster to execute the target job, and the computing resource type of the target computing resource is the target computing resource type.
In step S306, as shown in fig. 9, the resource allocation system may allocate a target computing resource to the target computing cluster in response to a resource allocation instruction input by a background operator or a user, and display the allocated target computing resource for the target computing cluster (i.e., display an allocation result in fig. 9).
It should be noted that, by displaying the target computing resources allocated to the target computing cluster, it is further convenient for a background operator to supervise the resource allocation system, so as to avoid the occurrence of a phenomenon that the resource allocation system allocates the same type of target computing resources to all target jobs or allocates a plurality of target computing resources to the same target job, thereby improving the stability of target job execution.
It should be noted that, for simplicity of description, the above-mentioned method embodiments are described as a series of acts or combination of acts, but those skilled in the art will recognize that the present application is not limited by the order of acts described, as some steps may occur in other orders or concurrently depending on the application. Further, those skilled in the art should also appreciate that the embodiments described in the specification are preferred embodiments and that the acts and modules referred to are not necessarily required in this application.
Through the above description of the embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but the former is a better implementation mode in many cases. Based on such understanding, the technical solutions of the present application or portions thereof that contribute to the prior art may be embodied in the form of a software product, where the computer software product is stored in a storage medium (such as a ROM/RAM, a magnetic disk, and an optical disk), and includes several instructions for enabling a terminal device (which may be a mobile phone, a computer, a server, or a network device) to execute the method of the embodiments of the present application.
Example 3
According to an embodiment of the present invention, there is also provided a resource allocation apparatus for implementing the resource allocation method, as shown in fig. 10, the apparatus includes:
an obtaining module 402, configured to obtain job feature information of a target job to be executed by a target computing cluster;
a determination module 404 for determining a target computing resource type based on job characterization information;
an allocating module 406, configured to allocate a target computing resource to a target computing cluster, where the target computing resource is a computing resource required by the target computing cluster to execute a target job, and a computing resource type of the target computing resource is a target computing resource type.
It should be noted here that the acquiring module 402, the determining module 404 and the allocating module 406 correspond to steps S202 to S206 in embodiment 1, and the three modules are the same as the corresponding steps in the implementation example and application scenario, but are not limited to the disclosure in embodiment 1. It should be noted that the above modules may be operated in the electronic device 10 provided in embodiment 1 as a part of the apparatus.
Optionally, the obtaining module further includes: the sub-acquisition module is used for acquiring the job information of the target job from the job list corresponding to the resource scheduling component, wherein the resource scheduling component is used for receiving the target job submitted by the target object and distributing the target job to the computing nodes of the target computing cluster; the first sub-determination module is configured to determine, based on the target computing cluster, system characteristic information corresponding to a target job, where the job characteristic information at least includes job information and system characteristic information, and the job information at least includes: container dependency information of target operation running time, and system characteristic information at least comprises: and calculating the network bandwidth and the memory occupancy rate of the nodes.
Optionally, the determining module further includes: the query module is used for querying the computing resource types from a preset feature database based on the job feature information to obtain a query result, wherein the preset feature database at least stores the job feature information and the computing resource types of historical jobs and the incidence relation between the job feature information and the computing resource types of the historical jobs, and the computing resource types are the types of computing resources required by a target computing cluster to execute the historical jobs; the second sub-determination module is used for determining that the computing resource type corresponding to the job characteristic information of the target job in the preset characteristic database is the target computing resource type when the query result represents that the computing resource type corresponding to the job characteristic information of the target job exists in the preset characteristic database; and the first processing module is used for comparing the operation characteristic information of the target operation with the target operation characteristic information corresponding to the preset computing resource type when the query result represents that the computing resource type corresponding to the operation characteristic information of the target operation does not exist in the preset characteristic database, and determining the target computing resource type according to the comparison result.
Optionally, the first processing module further includes: the third sub-determination module is used for determining the operation running form of the target operation according to the container dependence information of the target operation; and the fourth sub-determination module is used for determining the target computing resource type as a containerized instance computing resource type when the job running form indicates that the target job runs in the container, wherein the containerized instance computing resource type indicates that the target job is dependent on the container during running.
Optionally, the resource allocation apparatus further includes: the second processing module is used for detecting whether the real-time maximum network bandwidth of the computing node is larger than a first preset bandwidth or not and whether the average bandwidth corresponding to the computing node is larger than a second preset bandwidth or not when the operation mode represents that the target operation is not operated in the container; and the fifth sub-determining module is used for determining that the target computing resource type is a super computing cluster computing resource type when the real-time maximum network bandwidth is larger than the first preset bandwidth or the average bandwidth is larger than the second preset bandwidth, wherein the super computing cluster computing resource type represents that the running of the target operation is influenced by the network bandwidth to the maximum extent.
Optionally, the resource allocation apparatus further includes: the third processing module is used for detecting whether the memory occupancy rate of the computing node is greater than the preset occupancy rate or not when the real-time maximum network bandwidth is less than or equal to the first preset bandwidth and the average bandwidth is less than or equal to the second preset bandwidth; the sixth sub-determination module is used for determining that the target computing resource type is the computing resource type of the physical machine when the memory occupancy rate is greater than the preset occupancy rate; and the seventh sub-determining module is used for determining the target computing resource type as the virtual machine computing resource type when the memory occupancy rate is less than or equal to the preset occupancy rate.
Optionally, the resource allocation apparatus further includes: and the storage module is used for storing the operation detail information and the system characteristic information corresponding to the target operation into a preset characteristic database.
Optionally, the allocation module further includes: the fourth processing module is used for applying for target computing resources corresponding to the target computing resource types to the cloud server based on the resource management component, wherein the resource management component is used for managing the life cycle of the computing resources required by the target operation, and the cloud server is used for carrying out production management on the computing resources required by the target operation; the fifth processing module is used for creating a task queue to the resource scheduling component based on the resource management component and adding a target computing resource to the resource scheduling component based on the task queue; and the sub-allocation module is used for allocating the target computing resources to the task queues corresponding to the computing nodes of the target computing cluster based on the scheduling strategies corresponding to the resource scheduling components.
Optionally, the resource allocation apparatus further includes: and the sixth processing module is used for removing the target computing resource from the task queue and sending a resource release message to the cloud server to release the target computing resource after the resource management component detects that the resource scheduling component completes the target job.
Optionally, the resource allocation apparatus further includes: the system comprises a creating module, a processing module and a processing module, wherein the creating module is used for creating a plurality of task queues according to the job characteristic information of a plurality of target jobs when the number of the target jobs is multiple, and each task queue corresponds to computing resources of different computing resource types; the eighth sub-determining module is used for determining target computing resources corresponding to each target job according to the computing resource types corresponding to the plurality of task queues; a seventh processing module to execute a plurality of target jobs based on the target computing resources.
Example 4
According to an embodiment of the present invention, there is also provided a resource allocation apparatus for implementing the resource allocation method, as shown in fig. 11, the apparatus includes:
a first response module 502, configured to respond to the job creation instruction, create a target job and execute a target computing cluster of the target job, and display job feature information of the target job;
a display module 504 for displaying the target computing resource type determined based on the job characteristic information;
a second response module 506, configured to respond to the resource allocation instruction, and display a target computing resource allocated to the target computing cluster, where the target computing resource is a computing resource required by the target computing cluster to execute the target job, and a computing resource type of the target computing resource is a target computing resource type.
It should be noted here that the first response module 502, the display module 504, and the second response module 506 correspond to steps S302 to S306 in embodiment 2, and the three modules are the same as the corresponding steps in the implementation example and application scenario, but are not limited to the disclosure in embodiment 2. It should be noted that the above modules may be implemented in the electronic device 10 provided in the first embodiment as a part of an apparatus.
Example 5
According to an embodiment of the present invention, there is also provided a resource allocation system for implementing the resource allocation method, as shown in fig. 12, the system includes:
and the scheduler is used for receiving the target job submitted by the target object.
Optionally, the scheduler is responsible for receiving target jobs submitted by the user, allocating the target jobs to specific computing resources of the target computing cluster for computing according to different scheduling policies, and monitoring the running state of each target job to obtain a computing execution result corresponding to each target job. The target jobs are calculation tasks of the target calculation cluster, different target jobs can be configured with different parameters such as resource requirements, priorities, execution times and the like, and the scheduler can adopt different scheduling strategies according to the configuration parameters of the different target jobs so that the target calculation cluster can better execute the target jobs. When the target jobs corresponding to the computing resources are more, the scheduler can queue and manage the target jobs. The scheduler may also monitor the result of the computation execution of the target job, and resubmit execution of the target job whose execution has failed in the representation of the result of the computation execution.
And the characteristic collection component is used for acquiring the job characteristic information of the target job to be executed by the target computing cluster and determining the type of the target computing resource based on the job characteristic information.
Optionally, the feature collection component is connected to the scheduler, and the feature collection component may collect system feature information, such as container dependency information, network bandwidth, memory occupancy rate, and the like during operation of a target job, on the target computing cluster side by accessing the scheduler and the target computing cluster, determine a computing resource type corresponding to the application according to a certain policy, and send a resource application message to the resource management component. It should be noted that, unlike the conventional performance collection tool, the feature collection component can be combined with the resource management component, the scheduler, and other units on the cloud to perform resource flexibility and management together. In addition, the characteristic collection component can collect the operation characteristic information, fully exert the advantages of data on the cloud, and finally fully prejudge the type of the computing resource of the needed computing resource.
And the resource management component is used for determining the target computing resource according to the type of the target computing resource, creating a task queue for the scheduler, and then adding the target computing resource to the scheduler based on the task queue.
Optionally, the resource management component is connected to the feature collection component and the scheduler, and the resource management component is configured to manage a life cycle of the computing resource required by the target job, and specifically, the resource management component is configured to perform life cycle management of creation, deletion, addition, exit, and the like of the computing resource of the entire target computing cluster. The resource management component can apply for the determined target computing resource from the cloud server, interact with the scheduler, and inform the scheduler of the addition and the exit of the computing resource, so that the scheduler can update the scheduling policy of the scheduler, and after the corresponding computing resource in the scheduler is used, the resource manager can also release the computing resource to the cloud server. Unlike the traditional composition of related systems, since the instances on the cloud can be flexibly applied and released, the system of the present application includes the part of the special resource management component for the life cycle management of the cluster computing resource. The resource management component may be a separately deployed system, or may be deployed with the scheduler or as an internal component of the scheduler.
The scheduler also allocates the target computing resource to a task queue corresponding to the computing node of the target computing cluster based on a preset scheduling strategy so that the computing node in the target computing cluster executes the target job.
Optionally, the scheduler is further configured to allocate the target computing resource to a task queue corresponding to a relevant server in the target computing cluster after applying for the corresponding target computing resource, so that the relevant server executes the corresponding target job.
In an optional embodiment, the resource allocation system may further include a cloud server, where the cloud server is connected to the resource management component and is configured to perform production management on the computing resources required by the target job, where at least the computing resources corresponding to the containerized instance computing resource type, the computing resources corresponding to the super computing cluster computing resource type, the computing resources corresponding to the physical computer computing resource type, and the computing resources corresponding to the virtual machine computing resource type are stored in the cloud server.
It should be noted here that the scheduler, the feature collection component, and the resource management component can be used to implement the method provided in embodiment 1, and the example and application scenario implemented by the scheduler, the feature collection component, and the resource management component are the same as those implemented in embodiment 1, but are not limited to the disclosure in embodiment 1.
Example 6
Embodiments of the present application may provide an electronic device, which may be any one of electronic devices in a group of electronic devices. Alternatively, in this embodiment, the electronic device may be replaced with a terminal device such as a mobile terminal.
Optionally, in this embodiment, the electronic device may be located in at least one network device of a plurality of network devices of a computer network.
In this embodiment, the electronic device may execute the program code of the following steps in the resource allocation method: acquiring operation characteristic information of a target operation to be executed by a target computing cluster; determining a target computing resource type based on the job characteristic information; and allocating target computing resources for the target computing cluster, wherein the target computing resources are computing resources required by the target computing cluster to execute the target job, and the computing resource type of the target computing resources is the target computing resource type.
Optionally, fig. 13 is a block diagram of an electronic device according to an embodiment of the present application. As shown in fig. 13, the electronic device 10 may include: one or more (only one shown) processors 102, memory 104, and a memory controller.
The memory may be configured to store software programs and modules, such as program instructions/modules corresponding to the resource allocation method and apparatus in the embodiments of the present application, and the processor executes various functional applications and data processing by running the software programs and modules stored in the memory, so as to implement the resource allocation method. The memory may include high speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memories may further include a memory located remotely from the processor, which may be connected to the terminal 10 via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The processor can call the information and application program stored in the memory through the transmission device to execute the following steps: acquiring operation characteristic information of a target operation to be executed by a target computing cluster; determining a target computing resource type based on the job characteristic information; and allocating target computing resources for the target computing cluster, wherein the target computing resources are computing resources required by the target computing cluster to execute the target job, and the computing resource type of the target computing resources is the target computing resource type.
Optionally, the processor may further execute the program code of the following steps: acquiring the job information of a target job from a job list corresponding to a resource scheduling component, wherein the resource scheduling component is used for receiving the target job submitted by a target object and distributing the target job to a computing node of a target computing cluster; determining system characteristic information corresponding to a target job based on a target computing cluster, wherein the job characteristic information at least comprises job information and system characteristic information, and the job information at least comprises: container dependency information of target operation running time, and system characteristic information at least comprises: and calculating the network bandwidth and the memory occupancy rate of the nodes.
Optionally, the processor may further execute the program code of the following steps: performing computing resource type query from a preset feature database based on the job feature information to obtain a query result, wherein the preset feature database at least stores job feature information and computing resource types of historical jobs and an incidence relation between the job feature information and the computing resource types of the historical jobs, and the computing resource types are the types of computing resources required by a target computing cluster to execute the historical jobs; when the query result represents that the computing resource type corresponding to the operation characteristic information of the target operation exists in the preset characteristic database, determining the computing resource type corresponding to the operation characteristic information of the target operation in the preset characteristic database as the target computing resource type; and when the query result represents that the computing resource type corresponding to the operation characteristic information of the target operation does not exist in the preset characteristic database, comparing the operation characteristic information of the target operation with the target operation characteristic information corresponding to the preset computing resource type, and determining the target computing resource type according to the comparison result.
Optionally, the processor may further execute the program code of the following steps: determining the operation running form of the target operation according to the container dependence information of the target operation running; when the job run form characterizes that the target job runs in the container, determining that the target computing resource type is a containerized instance computing resource type, wherein the containerized instance computing resource type characterizes that the target job runs in dependence on the container.
Optionally, the processor may further execute the program code of the following steps: when the operation running form represents that the target operation is not run in the container, detecting whether the real-time maximum network bandwidth of the computing node is larger than a first preset bandwidth or not, and whether the average bandwidth corresponding to the computing node is larger than a second preset bandwidth or not; and when the real-time maximum network bandwidth is larger than a first preset bandwidth or the average bandwidth is larger than a second preset bandwidth, determining that the target computing resource type is a super computing cluster computing resource type, wherein the super computing cluster computing resource type represents that the operation of the target operation is influenced by the network bandwidth to the maximum extent.
Optionally, the processor may further execute the program code of the following steps: when the real-time maximum network bandwidth is smaller than or equal to a first preset bandwidth and the average bandwidth is smaller than or equal to a second preset bandwidth, detecting whether the memory occupancy rate of the computing node is larger than the preset occupancy rate; when the memory occupancy rate is greater than the preset occupancy rate, determining that the target computing resource type is the physical computer computing resource type; and when the memory occupancy rate is less than or equal to the preset occupancy rate, determining the target computing resource type as the virtual machine computing resource type.
Optionally, the processor may further execute the program code of the following steps: and after comparing the operation characteristic information of the target operation with the target operation characteristic information corresponding to the preset computing resource type and determining the target computing resource type according to the comparison result, storing the operation detail information and the system characteristic information corresponding to the target operation into a preset characteristic database.
Optionally, the processor may further execute the program code of the following steps: applying for target computing resources corresponding to the target computing resource types to a cloud server based on a resource management component, wherein the resource management component is used for managing the life cycle of the computing resources required by the target operation, and the cloud server is used for carrying out production management on the computing resources required by the target operation; creating a task queue to the resource scheduling component based on the resource management component, and adding a target computing resource to the resource scheduling component based on the task queue; and allocating the target computing resources to the task queues corresponding to the computing nodes of the target computing cluster based on the scheduling strategies corresponding to the resource scheduling components.
Optionally, the processor may further execute the program code of the following steps: after allocating the target computing resources for the target computing cluster, after the resource management component detects that the resource scheduling component completes the target job, removing the target computing resources from the task queue, and sending a resource release message to the cloud server to release the target computing resources.
Optionally, the processor may further execute the program code of the following steps: after determining the target computing resource types based on the job characteristic information, when the number of the target jobs is multiple, creating a plurality of task queues according to the job characteristic information of the multiple target jobs, wherein each task queue corresponds to computing resources of different computing resource types; determining a target computing resource corresponding to each target operation according to the computing resource types corresponding to the plurality of task queues; a plurality of target jobs are executed based on the target computing resources.
Optionally, the processor may further execute the program code of the following steps: responding to a job creating instruction, creating a target job and a target computing cluster for executing the target job, and displaying job characteristic information of the target job; displaying a target computing resource type determined based on the job characteristic information; and responding to the resource allocation instruction, and displaying the target computing resources allocated to the target computing cluster, wherein the target computing resources are computing resources required by the target computing cluster to execute the target job, and the computing resource type of the target computing resources is the target computing resource type.
By adopting the embodiment of the application, the electronic equipment for implementing the resource allocation method in the foregoing embodiment is provided. Different types of computing resources are allocated to each job based on job characteristic information of the jobs, so that the purpose of improving the rationality of resource allocation is achieved, and the technical problem of unreasonable resource allocation caused by the fact that similar computing resources are selected for all jobs in the prior art is solved.
It can be understood by those skilled in the art that the structure shown in fig. 13 is only an illustration, and the electronic device may also be a terminal device such as a smart phone (e.g., an Android phone, an iOS phone, etc.), a tablet computer, a palmtop computer, a Mobile Internet Device (MID), a PAD, and the like. Fig. 13 is a diagram illustrating a structure of the electronic device. For example, the electronic device 10 may also include more or fewer components (e.g., network interfaces, display devices, etc.) than shown in FIG. 13, or have a different configuration than shown in FIG. 13.
Those skilled in the art will appreciate that all or part of the steps in the methods of the above embodiments may be implemented by a program instructing hardware associated with the terminal device, where the program may be stored in a computer-readable storage medium, and the storage medium may include: flash disks, Read-Only memories (ROMs), Random Access Memories (RAMs), magnetic or optical disks, and the like.
Example 7
Embodiments of the present application also provide a computer-readable storage medium. Optionally, in this embodiment, the computer-readable storage medium may be configured to store the program code executed by the resource allocation method provided in the first embodiment.
Optionally, in this embodiment, the computer-readable storage medium may be located in any one of electronic devices in an electronic device group in a computer network, or in any one of mobile terminals in a mobile terminal group.
Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: acquiring operation characteristic information of a target operation to be executed by a target computing cluster; determining a target computing resource type based on the job characteristic information; and allocating target computing resources for the target computing cluster, wherein the target computing resources are computing resources required by the target computing cluster to execute the target job, and the computing resource type of the target computing resources is the target computing resource type.
Optionally, the computer readable storage medium may further store program code for performing the steps of: acquiring job information of a target job from a job list corresponding to a resource scheduling component, wherein the resource scheduling component is used for receiving the target job submitted by a target object and distributing the target job to a computing node of a target computing cluster; determining system characteristic information corresponding to a target job based on a target computing cluster, wherein the job characteristic information at least comprises job information and system characteristic information, and the job information at least comprises: container dependency information of target operation running time, and system characteristic information at least comprises: and calculating the network bandwidth and the memory occupancy rate of the nodes.
Optionally, the computer readable storage medium may further store program code for performing the steps of: performing computing resource type query from a preset feature database based on the job feature information to obtain a query result, wherein the preset feature database at least stores job feature information and computing resource types of historical jobs and an incidence relation between the job feature information and the computing resource types of the historical jobs, and the computing resource types are the types of computing resources required by a target computing cluster to execute the historical jobs; when the query result represents that the computing resource type corresponding to the operation characteristic information of the target operation exists in the preset characteristic database, determining the computing resource type corresponding to the operation characteristic information of the target operation in the preset characteristic database as the target computing resource type; and when the query result represents that the computing resource type corresponding to the operation characteristic information of the target operation does not exist in the preset characteristic database, comparing the operation characteristic information of the target operation with the target operation characteristic information corresponding to the preset computing resource type, and determining the target computing resource type according to the comparison result.
Optionally, the computer readable storage medium may further store program code for performing the steps of: determining the operation running form of the target operation according to the container dependence information of the target operation running; and when the job running form represents that the target job runs in the container, determining that the target computing resource type is a containerized instance computing resource type, wherein the containerized instance computing resource type represents that the target job runs in dependence on the container.
Optionally, the computer readable storage medium may further store program code for performing the steps of: when the operation running form represents that the target operation is not run in the container, detecting whether the real-time maximum network bandwidth of the computing node is larger than a first preset bandwidth or not, and whether the average bandwidth corresponding to the computing node is larger than a second preset bandwidth or not; and when the real-time maximum network bandwidth is larger than a first preset bandwidth or the average bandwidth is larger than a second preset bandwidth, determining that the target computing resource type is a super computing cluster computing resource type, wherein the super computing cluster computing resource type represents that the operation of the target operation is influenced by the network bandwidth to the maximum extent.
Optionally, the computer readable storage medium may further store program code for performing the steps of: when the real-time maximum network bandwidth is less than or equal to a first preset bandwidth and the average bandwidth is less than or equal to a second preset bandwidth, detecting whether the memory occupancy rate of the computing node is greater than the preset occupancy rate; when the memory occupancy rate is greater than the preset occupancy rate, determining that the target computing resource type is the physical computer computing resource type; and when the memory occupancy rate is less than or equal to the preset occupancy rate, determining the target computing resource type as the virtual machine computing resource type.
Optionally, the computer readable storage medium may further store program code for performing the steps of: and after comparing the operation characteristic information of the target operation with the target operation characteristic information corresponding to the preset computing resource type and determining the target computing resource type according to the comparison result, storing the operation detail information and the system characteristic information corresponding to the target operation into a preset characteristic database.
Optionally, the computer readable storage medium may further store program code for performing the steps of: applying a target computing resource corresponding to the target computing resource type to a cloud server based on a resource management component, wherein the resource management component is used for managing the life cycle of the computing resource required by the target operation, and the cloud server is used for carrying out production management on the computing resource required by the target operation; creating a task queue to the resource scheduling component based on the resource management component, and adding a target computing resource to the resource scheduling component based on the task queue; and allocating the target computing resources to the task queues corresponding to the computing nodes of the target computing cluster based on the scheduling strategies corresponding to the resource scheduling components.
Optionally, the computer readable storage medium may further store program code for performing the steps of: after allocating the target computing resources for the target computing cluster, after the resource management component detects that the resource scheduling component completes the target job, removing the target computing resources from the task queue, and sending a resource release message to the cloud server to release the target computing resources.
Optionally, the computer readable storage medium may further store program code for performing the steps of: after determining the target computing resource types based on the job characteristic information, when the number of the target jobs is multiple, creating a plurality of task queues according to the job characteristic information of the multiple target jobs, wherein each task queue corresponds to computing resources of different computing resource types; determining a target computing resource corresponding to each target operation according to the computing resource types corresponding to the plurality of task queues; a plurality of target jobs are executed based on the target computing resources.
Optionally, the computer readable storage medium may further store program code for performing the steps of: responding to a job creating instruction, creating a target job and a target computing cluster for executing the target job, and displaying job characteristic information of the target job; displaying a target computing resource type determined based on the job characteristic information; and responding to the resource allocation instruction, and displaying the target computing resources allocated to the target computing cluster, wherein the target computing resources are computing resources required by the target computing cluster to execute the target job, and the computing resource type of the target computing resources is the target computing resource type.
The above-mentioned serial numbers of the embodiments of the present application are merely for description and do not represent the merits of the embodiments.
In the above embodiments of the present application, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.
In the embodiments provided in the present application, it should be understood that the disclosed technology can be implemented in other ways. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units is only one type of division of logical functions, and there may be other divisions when actually implemented, for example, a plurality of units or components may be combined or may be integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, units or modules, and may be in an electrical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application may be substantially implemented or contributed to by the prior art, or all or part of the technical solution may be embodied in a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a removable hard disk, a magnetic or optical disk, and other various media capable of storing program codes.
The foregoing is only a preferred embodiment of the present application and it should be noted that those skilled in the art can make several improvements and modifications without departing from the principle of the present application, and these improvements and modifications should also be considered as the protection scope of the present application.

Claims (14)

1. A method for resource allocation, comprising:
acquiring operation characteristic information of a target operation to be executed by a target computing cluster;
determining a target computing resource type based on the job characterization information;
allocating target computing resources to the target computing cluster, wherein the target computing resources are computing resources required by the target computing cluster to execute the target job, and the computing resource type of the target computing resources is the target computing resource type.
2. The method of claim 1, wherein obtaining job characteristics information for a target job to be executed by a target compute cluster comprises:
acquiring the job information of the target job from a job list corresponding to a resource scheduling component, wherein the resource scheduling component is used for receiving the target job submitted by a target object and distributing the target job to the computing nodes of the target computing cluster;
determining system characteristic information corresponding to the target job based on the target computing cluster, wherein the job characteristic information at least comprises the job information and the system characteristic information, and the job information at least comprises: the container dependency information of the target operation runtime, the system characteristic information at least includes: and calculating the network bandwidth and the memory occupancy rate of the node.
3. The method of claim 2, wherein determining a target computing resource type based on the job characterization information comprises:
performing computing resource type query from a preset feature database based on the job feature information to obtain a query result, wherein the preset feature database at least stores job feature information and computing resource types of historical jobs and an incidence relation between the job feature information of the historical jobs and the computing resource types, and the computing resource types are types of computing resources required by the target computing cluster to execute the historical jobs;
when the query result represents that the computing resource type corresponding to the operation feature information of the target operation exists in the preset feature database, determining that the computing resource type corresponding to the operation feature information of the target operation in the preset feature database is the target computing resource type;
and when the query result indicates that the computing resource type corresponding to the operation characteristic information of the target operation does not exist in the preset characteristic database, comparing the operation characteristic information of the target operation with target operation characteristic information corresponding to a preset computing resource type, and determining the target computing resource type according to the comparison result.
4. The method according to claim 3, wherein comparing the job feature information of the target job with target job feature information corresponding to a preset computing resource type, and determining the target computing resource type according to the comparison result comprises:
determining the operation running form of the target operation according to the container dependence information of the target operation running;
when the job run form characterizes that the target job runs in a container, determining that the target computing resource type is a containerized instance computing resource type, wherein the containerized instance computing resource type characterizes that the target job runs in dependence on the container.
5. The method of claim 4, further comprising:
when the operation running form represents that the target operation does not run in the container, detecting whether the real-time maximum network bandwidth of the computing node is larger than a first preset bandwidth or not, and whether the average bandwidth corresponding to the computing node is larger than a second preset bandwidth or not;
when the real-time maximum network bandwidth is larger than the first preset bandwidth, or the average bandwidth is larger than the second preset bandwidth, determining that the target computing resource type is a super computing cluster computing resource type, wherein the super computing cluster computing resource type represents that the running of the target job is influenced by the network bandwidth to the maximum extent.
6. The method of claim 5, further comprising:
when the real-time maximum network bandwidth is smaller than or equal to the first preset bandwidth and the average bandwidth is smaller than or equal to the second preset bandwidth, detecting whether the memory occupancy rate of the computing node is larger than a preset occupancy rate;
when the memory occupancy rate is greater than the preset occupancy rate, determining that the target computing resource type is a physical machine computing resource type;
and when the memory occupancy rate is less than or equal to the preset occupancy rate, determining that the target computing resource type is the virtual machine computing resource type.
7. The method according to any one of claims 4 to 6, wherein after comparing the job characteristic information of the target job with target job characteristic information corresponding to a preset computing resource type and determining the target computing resource type according to the comparison result, the method further comprises:
and storing the operation detail information corresponding to the target operation and the system characteristic information into the preset characteristic database.
8. The method of claim 1, wherein allocating target computing resources for the target computing cluster comprises:
applying for a target computing resource corresponding to the target computing resource type to a cloud server based on a resource management component, wherein the resource management component is used for managing the life cycle of the computing resource required by the target job, and the cloud server is used for performing production management on the computing resource required by the target job;
creating a task queue to a resource scheduling component based on the resource management component, and adding the target computing resource to the resource scheduling component based on the task queue;
and allocating the target computing resource to a task queue corresponding to a computing node of the target computing cluster based on a scheduling policy corresponding to the resource scheduling component.
9. The method of claim 8, wherein after allocating target computing resources for the target computing cluster, the method further comprises:
after the resource management component detects that the resource scheduling component completes the target job, the target computing resource is removed from the task queue, and a resource release message is sent to the cloud server to release the target computing resource.
10. The method of claim 1, wherein after determining a target computing resource type based on the job characterization information, the method further comprises:
when the number of the target jobs is multiple, creating a plurality of task queues according to job characteristic information of the target jobs, wherein each task queue corresponds to computing resources of different computing resource types;
determining a target computing resource corresponding to each target operation according to the computing resource types corresponding to the plurality of task queues;
executing a plurality of the target jobs based on the target computing resources.
11. A method for resource allocation, comprising:
responding to a job creating instruction, creating a target job, executing a target computing cluster of the target job, and displaying job characteristic information of the target job;
displaying a target computing resource type determined based on the job characteristic information;
and responding to a resource allocation instruction, and displaying target computing resources allocated to the target computing cluster, wherein the target computing resources are computing resources required by the target computing cluster to execute the target job, and the computing resource type of the target computing resources is the target computing resource type.
12. A resource allocation system, comprising:
the scheduler is used for receiving a target job submitted by a target object;
the characteristic collection component is used for acquiring the job characteristic information of a target job to be executed by the target computing cluster and determining the type of the target computing resource based on the job characteristic information;
the resource management component is used for determining target computing resources according to the types of the target computing resources, creating a task queue for the scheduler, and then adding the target computing resources to the scheduler based on the task queue;
the scheduler further allocates the target computing resource to a task queue corresponding to a computing node of the target computing cluster based on a preset scheduling policy, so that the computing node in the target computing cluster executes the target job.
13. A computer-readable storage medium, in which a computer program is stored, wherein the computer program is arranged to perform the resource allocation method of any one of claims 1 to 11 when executed.
14. An electronic device, wherein the electronic device comprises one or more processors; memory for storing one or more programs which, when executed by the one or more processors, cause the one or more processors to implement a method for running a program, wherein the program is arranged to, when running, perform the resource allocation method of any of claims 1 to 11.
CN202210212325.0A 2022-03-04 2022-03-04 Resource allocation method, system, computer readable storage medium and electronic device Pending CN114661462A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210212325.0A CN114661462A (en) 2022-03-04 2022-03-04 Resource allocation method, system, computer readable storage medium and electronic device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210212325.0A CN114661462A (en) 2022-03-04 2022-03-04 Resource allocation method, system, computer readable storage medium and electronic device

Publications (1)

Publication Number Publication Date
CN114661462A true CN114661462A (en) 2022-06-24

Family

ID=82026515

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210212325.0A Pending CN114661462A (en) 2022-03-04 2022-03-04 Resource allocation method, system, computer readable storage medium and electronic device

Country Status (1)

Country Link
CN (1) CN114661462A (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115495249A (en) * 2022-10-31 2022-12-20 上海楷领科技有限公司 Task execution method of cloud cluster
CN116302452A (en) * 2023-05-18 2023-06-23 苏州浪潮智能科技有限公司 Job scheduling method, system, device, communication equipment and storage medium
CN117472516A (en) * 2023-12-27 2024-01-30 苏州元脑智能科技有限公司 Virtual resource scheduling method, device, cluster system, electronic equipment and medium
WO2024037173A1 (en) * 2022-08-17 2024-02-22 华为技术有限公司 Scheduler, job scheduling method and related device

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2024037173A1 (en) * 2022-08-17 2024-02-22 华为技术有限公司 Scheduler, job scheduling method and related device
CN115495249A (en) * 2022-10-31 2022-12-20 上海楷领科技有限公司 Task execution method of cloud cluster
CN116302452A (en) * 2023-05-18 2023-06-23 苏州浪潮智能科技有限公司 Job scheduling method, system, device, communication equipment and storage medium
CN116302452B (en) * 2023-05-18 2023-08-22 苏州浪潮智能科技有限公司 Job scheduling method, system, device, communication equipment and storage medium
CN117472516A (en) * 2023-12-27 2024-01-30 苏州元脑智能科技有限公司 Virtual resource scheduling method, device, cluster system, electronic equipment and medium
CN117472516B (en) * 2023-12-27 2024-03-29 苏州元脑智能科技有限公司 Virtual resource scheduling method, device, cluster system, electronic equipment and medium

Similar Documents

Publication Publication Date Title
CN114661462A (en) Resource allocation method, system, computer readable storage medium and electronic device
US10956424B2 (en) Application recommending method and system, and server
CN110427252B (en) Task scheduling method, device and storage medium based on task dependency relationship
CN110308980A (en) Batch processing method, device, equipment and the storage medium of data
CN111625331B (en) Task scheduling method, device, platform, server and storage medium
CN109478147B (en) Adaptive resource management in distributed computing systems
CN105049268A (en) Distributed computing resource allocation system and task processing method
CN104954478A (en) System and method for realizing automatic longitudinal scaling of server in cloud computing platform
CN112579304A (en) Resource scheduling method, device, equipment and medium based on distributed platform
CN104780146A (en) Resource manage method and device
CN111338774A (en) Distributed timing task scheduling system and computing device
US9535749B2 (en) Methods for managing work load bursts and devices thereof
US20100042723A1 (en) Method and system for managing load in a network
CN116541134B (en) Method and device for deploying containers in multi-architecture cluster
CN109614227A (en) Task resource concocting method, device, electronic equipment and computer-readable medium
CN112395075A (en) Resource processing method and device and resource scheduling system
CN111258746A (en) Resource allocation method and service equipment
CN115033352A (en) Task scheduling method, device and equipment for multi-core processor and storage medium
Hanafy et al. A new infrastructure elasticity control algorithm for containerized cloud
CN115658301A (en) Storage resource scheduling method, device, storage medium and electronic equipment
CN114816709A (en) Task scheduling method, device, server and readable storage medium
CN114221908A (en) Dynamic current-limiting fusing processing method and device, computer equipment and storage medium
CN109697117B (en) Terminal control method, terminal control device and computer-readable storage medium
CN105550025A (en) Distributed IaaS (Infrastructure as a Service) scheduling method and system
CN117311973A (en) Computing device scheduling method and device, nonvolatile storage medium and electronic device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination