CN117435337A - Resource allocation method, device, electronic equipment and storage medium - Google Patents

Resource allocation method, device, electronic equipment and storage medium Download PDF

Info

Publication number
CN117435337A
CN117435337A CN202311035476.4A CN202311035476A CN117435337A CN 117435337 A CN117435337 A CN 117435337A CN 202311035476 A CN202311035476 A CN 202311035476A CN 117435337 A CN117435337 A CN 117435337A
Authority
CN
China
Prior art keywords
physical machine
resource
node
machine nodes
resource index
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202311035476.4A
Other languages
Chinese (zh)
Inventor
刘春雨
程君陶
胡平
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Industrial and Commercial Bank of China Ltd ICBC
Original Assignee
Industrial and Commercial Bank of China Ltd ICBC
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Industrial and Commercial Bank of China Ltd ICBC filed Critical Industrial and Commercial Bank of China Ltd ICBC
Priority to CN202311035476.4A priority Critical patent/CN117435337A/en
Publication of CN117435337A publication Critical patent/CN117435337A/en
Pending legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/50Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5061Partitioning or combining of resources
    • G06F9/5077Logical partitioning of resources; Management or configuration of virtualized resources
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44Arrangements for executing specific programs
    • G06F9/455Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45533Hypervisors; Virtual machine monitors
    • G06F9/45558Hypervisor-specific management and integration aspects
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02DCLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00Energy efficient computing, e.g. low power processors, power management or thermal management

Landscapes

  • Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Debugging And Monitoring (AREA)

Abstract

The disclosure provides a resource allocation method, a device, an electronic device and a storage medium, which can be applied to the fields of computer technology, financial science and technology and intelligent operation and maintenance, and can be applied to the field of software testing. The method comprises the following steps: acquiring resource information of N first physical machine nodes included in a server cluster node; aiming at each resource index, according to the resource information of the N first physical machine nodes, standard deviation corresponding to the resource index and weight corresponding to each of the N first physical machine nodes are obtained; obtaining the resource index probabilities corresponding to the N first physical machine nodes respectively according to the standard deviation and the weights; determining a target physical machine node from the N first physical machine nodes according to the resource index probability; and creating a container on the target physical machine node.

Description

Resource allocation method, device, electronic equipment and storage medium
Technical Field
The present disclosure relates to the fields of computer technology, financial technology, and intelligent operation and maintenance, and may be used in the field of software testing, and in particular, to a resource allocation method, apparatus, device, medium, and program product.
Background
Related technology, the automatic scheduling of a container cluster management system k8s (kubernetes) is utilized to realize the automatic deployment of containers in a physical server cluster, and the dock is utilized to realize the isolation of resources such as CPU, memory and the like among a plurality of containers on the same physical server.
In the process of implementing the disclosed concept, the inventor finds that at least the following problems exist in the related art: in the case of a high density deployment container for a physical server cluster, a resource hot spot situation may occur.
Disclosure of Invention
In view of the foregoing, the present disclosure provides a resource allocation method, apparatus, device, medium, and program product.
According to a first aspect of the present disclosure, there is provided a resource allocation method, including:
acquiring resource information of N first physical machine nodes included in a server cluster node, wherein the resource information comprises a plurality of resource indexes, and N is an integer greater than or equal to 1;
for each resource index, according to the resource information of the N first physical machine nodes, obtaining standard deviation corresponding to the resource index and weights corresponding to the N first physical machine nodes respectively;
obtaining resource index probabilities corresponding to the N first physical machine nodes according to the standard deviation and the weights;
Determining a target physical machine node from the N first physical machine nodes according to the resource index probability; and
creating a container on the target physical machine node.
According to an embodiment of the present disclosure, the obtaining, for each of the resource indexes, a standard deviation corresponding to the resource index and weights corresponding to the N first physical machine nodes according to the resource information of the N first physical machine nodes includes:
acquiring resource index data corresponding to the N first physical machine nodes according to the resource information of the N first physical machine nodes, and obtaining N resource index data;
normalizing the N resource index data to obtain N normalized index data;
obtaining the standard deviation according to the N normalization index data;
sequencing the N resource index data to obtain serial numbers corresponding to the N first physical machine nodes respectively;
dividing the serial numbers corresponding to the N first physical machine nodes by N to obtain weights corresponding to the N first physical machine nodes.
According to an embodiment of the present disclosure, the plurality of resource indicators includes at least one of: CPU remaining amount, memory remaining amount, disk remaining amount, number of created containers as main library and number of created containers belonging to the same application as the above containers corresponding to the above N first physical machine nodes respectively.
According to an embodiment of the present disclosure, in a case where the container is a master, and the N resource index data are ordered, as the sequence numbers corresponding to the N first physical machine nodes increase, the CPU remaining amount increases, the memory remaining amount increases, the disk remaining amount increases, the number of created containers as master decreases, and the number of created containers belonging to the same application as the container decreases.
According to an embodiment of the present disclosure, in a case where the container is a standby, and the N resource index data are ordered, as the numbers corresponding to the N first physical machine nodes increase, the CPU remaining amount increases, the memory remaining amount increases, the disk remaining amount increases, the number of created containers as a master increases, and the number of created containers belonging to the same application as the container decreases.
According to an embodiment of the present disclosure, determining the target physical machine node from the N first physical machine nodes according to the resource indicator probability includes:
adding the resource index probabilities corresponding to the plurality of resource indexes respectively for each first physical machine node to obtain total resource index probabilities corresponding to the first physical machine node, and obtaining a plurality of total resource index probabilities;
Determining a maximum resource indicator probability from the plurality of total resource indicator probabilities;
and determining a first physical machine node corresponding to the maximum resource index probability from the N first physical machine nodes as the target physical machine node.
According to an embodiment of the present disclosure, the obtaining the resource information of N physical machine nodes included in the server cluster node includes:
acquiring attribute information of the container and resource information of M initial physical machine nodes included in the server cluster node, wherein M is an integer greater than or equal to N;
and screening the N first physical machine nodes with the storage capacity remaining quantity larger than a preset threshold value from the M initial physical machine nodes according to the preset threshold value, the attribute information and the resource information of the M initial physical machine nodes.
According to an embodiment of the present disclosure, the resource information includes a disk occupation amount of each created container, the attribute information includes an estimated disk application amount, and the method further includes:
obtaining the maximum disk occupation amount according to the disk occupation amount of the created container corresponding to each physical machine node;
and determining the maximum disk occupation amount as the estimated disk application amount.
A second aspect of the present disclosure provides a resource allocation apparatus, comprising:
the acquisition module is used for acquiring resource information of N first physical machine nodes included in the server cluster node, wherein the resource information comprises a plurality of resource indexes, and N is an integer greater than or equal to 1;
the first obtaining module is used for obtaining standard deviation corresponding to the resource indexes and weights corresponding to the N first physical machine nodes according to the resource information of the N first physical machine nodes for each resource index;
the second obtaining module is used for obtaining the resource index probabilities corresponding to the N first physical machine nodes respectively according to the standard deviation and the weights;
the determining module is used for determining a target physical machine node from the N first physical machine nodes according to the resource index probability; and
and the creation module is used for creating the container on the target physical machine node.
A third aspect of the present disclosure provides an electronic device, comprising: one or more processors; and a memory for storing one or more programs, wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to perform the method described above.
A fourth aspect of the present disclosure also provides a computer-readable storage medium having stored thereon executable instructions that, when executed by a processor, cause the processor to perform the above-described method.
A fifth aspect of the present disclosure also provides a computer program product comprising a computer program which, when executed by a processor, implements the above method.
According to the embodiment of the disclosure, by acquiring a plurality of resource indexes corresponding to each of N first physical machine nodes, then obtaining, for each resource index, a standard deviation corresponding to the resource index and a weight corresponding to each of N first physical machine nodes according to the resource information of each of N first physical machine nodes, obtaining a standard deviation capable of more accurately reflecting the importance of each of the resource indexes and a probability capable of more accurately reflecting each of the resource indexes according to each of the resource indexes, wherein the first physical machine node is determined as an optimal physical machine node, then obtaining, according to the standard deviation and the weight, a resource index probability corresponding to each of N first physical machine nodes, obtaining a probability of available resources capable of more accurately reflecting the degree of balance of the plurality of resource indexes of each of the first physical machine nodes, and then determining, according to the resource index probability, a target physical machine node with more balanced plurality of resource indexes from the N first physical machine nodes, so as to allocate a container to the target physical machine node with more balanced resource indexes, and more reasonably allocate the container.
Drawings
The foregoing and other objects, features and advantages of the disclosure will be more apparent from the following description of embodiments of the disclosure with reference to the accompanying drawings, in which:
FIG. 1 schematically illustrates an application scenario diagram of a resource allocation method according to an embodiment of the present disclosure;
FIG. 2 schematically illustrates a system architecture diagram of a resource allocation method according to an embodiment of the present disclosure;
FIG. 3 schematically illustrates a flow chart of a resource allocation method according to an embodiment of the disclosure;
FIG. 4 schematically illustrates a system architecture diagram for data acquisition and storage in accordance with an embodiment of the present disclosure;
FIG. 5 schematically illustrates a flowchart of screening N first physical machine nodes from M initial physical machine nodes that have a storage capacity remaining greater than a preset threshold according to an embodiment of the present disclosure;
FIG. 6 schematically illustrates a flow diagram for creating a container on a target physical machine node according to an embodiment of the disclosure;
FIG. 7 schematically illustrates a block diagram of a resource allocation apparatus according to an embodiment of the disclosure; and
fig. 8 schematically illustrates a block diagram of an electronic device adapted to implement a resource allocation method according to an embodiment of the disclosure.
Detailed Description
Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. It should be understood that the description is only exemplary and is not intended to limit the scope of the present disclosure. In the following detailed description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the embodiments of the present disclosure. It may be evident, however, that one or more embodiments may be practiced without these specific details. In addition, in the following description, descriptions of well-known structures and techniques are omitted so as not to unnecessarily obscure the concepts of the present disclosure.
The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. The terms "comprises," "comprising," and/or the like, as used herein, specify the presence of stated features, steps, operations, and/or components, but do not preclude the presence or addition of one or more other features, steps, operations, or components.
All terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art unless otherwise defined. It should be noted that the terms used herein should be construed to have meanings consistent with the context of the present specification and should not be construed in an idealized or overly formal manner.
Where expressions like at least one of "A, B and C, etc. are used, the expressions should generally be interpreted in accordance with the meaning as commonly understood by those skilled in the art (e.g.," a system having at least one of A, B and C "shall include, but not be limited to, a system having a alone, B alone, C alone, a and B together, a and C together, B and C together, and/or A, B, C together, etc.).
In the technical solution of the present disclosure, the related user information (including, but not limited to, user personal information, user image information, user equipment information, such as location information, etc.) and data (including, but not limited to, data for analysis, stored data, displayed data, etc.) are information and data authorized by the user or sufficiently authorized by each party, and the related data is collected, stored, used, processed, transmitted, provided, disclosed, applied, etc. and processed, all in compliance with the related laws and regulations and standards of the related country and region, necessary security measures are taken, no prejudice to the public order, and corresponding operation entries are provided for the user to select authorization or rejection.
In the traditional mode, a physical server deploys a container. For some application systems with less traffic pressure, this way of deploying containers results in a waste of a lot of physical server resources. Therefore, in order to reasonably utilize the physical server resources, the related technology can utilize a manual selection mode or automatic scheduling of k8s to realize automatic deployment of containers on a plurality of physical servers, namely, automatic deployment of containers on a physical server cluster.
Usually, the production environment has sufficient resources, and under the production environment, 6-8 containers are deployed by a single physical server. Under the test environment, the lift platform physical server is limited by the whole test resources, and 40-50 containers are deployed. Any one physical server included in the physical server cluster can be used as a physical machine node, and in the process of creating the container in the test environment, it is important to select a reasonable physical machine node to deploy the container. The unreasonable selection of the physical machine nodes can cause the problems of uneven resource distribution, local resource hot spots and the like, and the environment stability is affected.
Under the test environment, under the condition that the container is automatically deployed in the physical server cluster by utilizing a manual selection mode, whether the resources such as CPU, memory, disk space and the like remained in each physical machine node in the cluster are sufficient or not needs to be manually confirmed, and one (the container is in a single-library condition) or a plurality of (the container comprises a main and standby library condition) physical machine nodes conforming to the conditions are selected empirically. The method for selecting the physical machine node by using the manual experience has no unified standard and has the condition of unreasonable selection. And special personnel are required to be invested to finish the work, so that the end-to-end supply of the container cannot be realized. Under the condition that the containers are required to be deployed on a large scale, more manpower resources are required to be input, and the manpower cost is high.
Under the test environment, under the condition that the automatic scheduling of k8s is utilized to realize the automatic deployment of containers in the physical server cluster, one CPU and the physical machine node with the largest memory resource residue can be selected from the cluster according to the automatic scheduling strategy of k8 s. And as the automatic scheduling strategy of k8s is simpler, only the physical machine nodes with more resource residues are selected, and the resource hot spot condition can occur in the scene of high-density container deployment.
E.g., CPU, memory hot spot. The default scheduling policy of k8s cannot ensure that high-load containers are scattered and deployed on a plurality of different physical machine nodes, and when more high-load containers are deployed on the same physical machine node, the CPU and memory utilization rate of the physical machine node is higher, and even the physical machine node is crashed accidentally.
Such as a disk hot spot. The default scheduling strategy of k8s cannot ensure that a plurality of containers with more data volume are scattered and deployed on a plurality of different physical machine nodes, so that the disk utilization rate of part of the physical machine nodes is higher, a large number of disks remain in part of the physical machine nodes, and the disk space is wasted.
In order to at least partially solve the technical problems existing in the related art, embodiments of the present disclosure provide a resource allocation method, which can be applied to the field of computer technology, the field of financial technology, and the field of intelligent operation and maintenance, and can be used in the field of software testing.
The embodiment of the disclosure provides a resource allocation method, which comprises the following steps: acquiring resource information of N first physical machine nodes included in a server cluster node, wherein the resource information comprises a plurality of resource indexes, and N is an integer greater than or equal to 1; aiming at each resource index, according to the resource information of the N first physical machine nodes, standard deviation corresponding to the resource index and weight corresponding to each of the N first physical machine nodes are obtained; obtaining the resource index probabilities corresponding to the N first physical machine nodes respectively according to the standard deviation and the weights; determining a target physical machine node from the N first physical machine nodes according to the resource index probability; and creating a container on the target physical machine node.
Fig. 1 schematically illustrates an application scenario diagram of a resource allocation method according to an embodiment of the present disclosure.
As shown in fig. 1, an application scenario 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 is a medium used to provide a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, among others.
The user may interact with the server 105 through the network 104 using at least one of the first terminal device 101, the second terminal device 102, the third terminal device 103, to receive or send messages, etc. Various communication client applications, such as a shopping class application, a web browser application, a search class application, an instant messaging tool, a mailbox client, social platform software, etc. (by way of example only) may be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103.
The first terminal device 101, the second terminal device 102, the third terminal device 103 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smartphones, tablets, laptop and desktop computers, and the like.
The server 105 may be a server providing various services, such as a background management server (by way of example only) providing support for websites browsed by the user using the first terminal device 101, the second terminal device 102, and the third terminal device 103. The background management server may analyze and process the received data such as the user request, and feed back the processing result (e.g., the web page, information, or data obtained or generated according to the user request) to the terminal device.
It should be noted that, the resource allocation method provided by the embodiments of the present disclosure may be generally performed by the server 105. Accordingly, the resource allocation apparatus provided by the embodiments of the present disclosure may be generally disposed in the server 105. The resource allocation method provided by the embodiments of the present disclosure may also be performed by a server or a server cluster that is different from the server 105 and is capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and/or the server 105. Accordingly, the resource allocation apparatus provided by the embodiments of the present disclosure may also be provided in a server or a server cluster that is different from the server 105 and is capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103 and/or the server 105.
It should be understood that the number of terminal devices, networks and servers in fig. 1 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.
Fig. 2 schematically illustrates a system architecture diagram of a resource allocation method according to an embodiment of the present disclosure.
As shown in fig. 2, the system architecture 200 of the resource allocation method includes a task application module 210, a task execution module 220, an index collection and storage module 230, a Node (Node) selection module 240, a console 250, and a k8s cluster 260.
The system architecture 200 may complete the task of obtaining a target physical machine node, and then creating a container on the target physical machine node according to the resource allocation method provided by the embodiments of the present disclosure.
The task application module 210 provides a task application front-end interface for the client 201, and is configured to fill in attribute information required for creating a container, and generate a task work order after the front-end interface of the client 201 fills in attribute information corresponding to the container. The attribute information comprises application names, version numbers, the number of master libraries, the number of slave libraries, CUP occupation amount and memory occupation amount. The version number characterizes version information of the container.
The index collection and storage module 230 is configured to collect system layer and k8s layer basic data, generate resource indexes, and store the resource indexes in a database, so as to facilitate subsequent automatic selection of the optimal physical machine node.
The Node selection module 240 completes the selection of the optimal physical machine Node according to the resource index collected by the index collection and storage module 23.
Console 250, which generates container creation rules and then configures down onto k8s cluster 260.
And the task execution module 220 is used for completing task execution according to the attribute information included in the task work order and the optimal physical machine node, namely executing the task of creating the container on the optimal physical machine node, and feeding back an execution result.
The Node selection module 240 may complete the selection of the optimal physical machine Node using the resource allocation method shown in fig. 3.
Fig. 3 schematically illustrates a flow chart of a resource allocation method according to an embodiment of the disclosure.
As shown in fig. 3, the resource allocation method of this embodiment includes operations S310 to S350.
In operation S310, resource information of N first physical machine nodes included in the server cluster node is obtained, where the resource information includes a plurality of resource indexes, and N is an integer greater than or equal to 1.
According to an embodiment of the present disclosure, the resource information includes resource remaining amount information and attribute information of the created container.
According to an embodiment of the present disclosure, the plurality of resource indexes includes an index corresponding to the resource remaining amount information and an index corresponding to the attribute information of the created container.
According to an embodiment of the disclosure, the first physical machine node characterizes a physical machine server of which the remaining amount of resource indicators in the physical server cluster is within a preset threshold range.
In operation S320, for each resource index, a standard deviation corresponding to the resource index and weights corresponding to the N first physical machine nodes are obtained according to the resource information of the N first physical machine nodes.
According to the embodiment of the disclosure, for each resource index, a root mean square error may be calculated for the resource indexes included in the resource information of the N first physical machine nodes, to obtain a standard deviation corresponding to the resource index.
According to an embodiment of the present disclosure, the magnitude of the standard deviation corresponding to each of the plurality of resource indicators reflects the importance of each resource indicator. The larger the standard deviation value is, the larger the difference of the resource indexes corresponding to the N first physical machine nodes is, and the more important the resource indexes are.
According to the embodiment of the disclosure, the weights corresponding to the N first physical machine nodes can be obtained according to the size relation of the resource indexes corresponding to the N first physical machine nodes.
According to an embodiment of the present disclosure, for each resource indicator, weights corresponding to each of the N first physical machine nodes reflect: the probability that the first physical machine node is determined to be the optimal physical machine node is determined based on each resource indicator.
In operation S330, a resource indicator probability corresponding to each of the N first physical machine nodes is obtained according to the standard deviation and the weight.
According to the embodiment of the disclosure, for each first physical machine node and each resource index, the standard deviation corresponding to the first physical machine node and the weight corresponding to the first physical machine node may be multiplied to obtain the resource index probability corresponding to the first physical machine node.
In operation S340, a target physical machine node is determined from the N first physical machine nodes according to the resource indicator probability.
According to the embodiment of the disclosure, the total resource index probability corresponding to each first physical machine node can be obtained according to the resource index probabilities corresponding to the plurality of resource indexes. And screening out a maximum resource index probability value or a minimum resource index probability value according to the total resource index probability corresponding to each first physical machine node, and determining the first physical machine node corresponding to the maximum resource index probability value or the minimum resource index probability value as a target physical machine node.
In operation S350, a container is created on the target physical machine node.
According to the embodiment of the present disclosure, the type of the container may be selected according to actual service conditions, which is not limited herein. For example, the container may be a MySQL database or an Oracle database, etc.
According to the embodiment of the disclosure, the target physical machine node can be taken as an optimal physical machine node, and the container is created on the optimal physical machine node.
In accordance with an embodiment of the present disclosure, the task execution module 220 creates a container on the target physical machine node after the target physical machine node is obtained.
According to the embodiment of the disclosure, by acquiring a plurality of resource indexes corresponding to each of N first physical machine nodes, then obtaining, for each resource index, a standard deviation corresponding to the resource index and a weight corresponding to each of N first physical machine nodes according to the resource information of each of N first physical machine nodes, obtaining a standard deviation capable of more accurately reflecting the importance of each of the resource indexes and a probability capable of more accurately reflecting each of the resource indexes according to each of the resource indexes, wherein the first physical machine node is determined as an optimal physical machine node, then obtaining, according to the standard deviation and the weight, a resource index probability corresponding to each of N first physical machine nodes, obtaining a probability of available resources capable of more accurately reflecting the degree of balance of the plurality of resource indexes of each of the first physical machine nodes, and then determining, according to the resource index probability, a target physical machine node with more balanced plurality of resource indexes from the N first physical machine nodes, so as to allocate a container to the target physical machine node with more balanced resource indexes, and more reasonably allocate the container.
According to the embodiment of the disclosure, the resource allocation method provided by the embodiment of the disclosure can ensure that under a test environment and a high-density deployment container, under the condition that a plurality of resource indexes comprise indexes corresponding to a CPU, a memory, a disk and the like, the resource index probabilities corresponding to N first physical machine nodes are obtained according to standard deviations and weights corresponding to the resource indexes, respectively, then the target physical machine Node is determined from the N first physical machine nodes according to the resource index probabilities, the target physical machine Node with balanced resource distribution corresponding to the CPU, the memory, the disk and the like can be obtained, and the problem of resource hot spots caused by manual selection of Node nodes and automatic selection of k8s is solved.
According to the embodiment of the disclosure, the resource allocation method provided by the embodiment of the disclosure can realize end-to-end automatic supply of the container, and the container resource supply efficiency is greatly improved. And the whole process is automatic, no professional is needed to participate, and the cost of human resources is obviously reduced.
According to an embodiment of the present disclosure, the plurality of resource indicators includes at least one of: CPU remaining amount, memory remaining amount, disk remaining amount, number of created containers as master library and number of created containers belonging to the same application as the containers corresponding to each of the N first physical machine nodes.
According to the embodiment of the disclosure, under the condition that the plurality of resource indexes include the CPU residual quantity, the memory residual quantity and the disk residual quantity, the target physical machine nodes with balanced CPU residual quantity, memory residual quantity and disk residual quantity can be screened out from the N first physical machine nodes according to the CPU residual quantity, the memory residual quantity and the disk residual quantity.
According to the embodiment of the disclosure, since the CPU/memory used by the main library is generally much larger than that of the standby library, the container corresponding to the main library is a high-load container, and when the plurality of resource indexes include the number of created containers as the main library and the number of created containers belonging to the same application as the containers, the high-load container can be ensured to be scattered and deployed on a plurality of different first physical machine nodes according to the number of created containers as the main library, so that CPU/memory hot spots caused by the fact that the first physical machine nodes are allocated with more main libraries are avoided.
According to the embodiment of the disclosure, as the number of the created containers serving as the main library and the number of the created containers belonging to the same application as the containers are included in the plurality of resource indexes, the plurality of containers with the larger data size can be ensured to be scattered and deployed on the plurality of different physical machine nodes, and the waste of computing resources such as the CPU/memory and the like caused by the fact that the first physical machine node is allocated with more standby libraries is avoided.
Fig. 4 schematically illustrates a system architecture diagram for data acquisition and storage according to an embodiment of the present disclosure.
The index collection and storage module 230 may collect and store the index according to a system architecture diagram as shown in fig. 4.
As shown in fig. 4, the data acquisition and storage module 400 includes a system layer 410, a k8s data acquisition layer 420, and a cache database 430.
The k8s data collection layer 420 may utilize the cluster control node k8s master 421 to control the collection of data from the initial physical machine node by the k8s node 421_1, the k8s node 421_2, and the k8s containers mysql_1 (master) 421_3 and mysql_2 (master) 421_4.
According to the embodiment of the disclosure, the k8s master is a k8s cluster control node, and is used for k8s cluster management and container creation according to configuration information issued by a console, and the container can be a MySQL database. The console is used to generate container creation rules and then the configuration is issued onto the k8smaster node. The k8s node is a k8s cluster container running node, and is a physical server, namely an initial physical machine node, in the MySQL scene.
As shown in fig. 4, the system layer 410 deploys the manager Agent "411" at the k8s node (physical machine) 421_1, and deploys the manager Agent "412" at the k8s node (physical machine) 421_2. For each Agent in the agents "411" and the agents "412", the method is used for collecting data such as a disk total space, a residual space, a disk size occupied by each MySQL container of a k8s node (physical machine) corresponding to the Agent, and the like, and writing the data into the cache database 430 for storage through the system layer API component. Wherein the Agent characterizes the manager.
The k8s data acquisition layer 420 may utilize the cluster control node k8s master 421 to control the data acquisition of the initial physical machine node by the k8s node layer and the k8s container layer. The k8snode layer includes a k8s snode (physical machine) 421_1 and a k8s snode (physical machine) 421_2. The k8s container layer includes mysql_1 (master) 421_3 and mysql_2 (master) 421_4.
For each k8s node (physical machine) in the k8s node (physical machine) 421_1 and the k8s node (physical machine) 421_2, the k8s interface api/v1/nodes can be called by the data collection component corresponding to the k8s node (physical machine), so as to collect the data such as the total amount of the CPU and the total amount of the memory of the initial physical machine node, and the data is recorded in the cache database 430.
For each MySQL container in mysql_1 (master) 421_3 and mysql_2 (master) 421_4, a k8s interface api/v 1/nmespace/{ }/ports may be called by a data collection component corresponding to the MySQL container, so as to collect data such as a MySQL container name, a CPU hard limit (i.e., a CPU allocation amount), a memory hard limit (memory allocation amount) and the like that have been created on the initial physical machine node, and enter the cache database 430.
In accordance with an embodiment of the present disclosure, to obtain application and master-slave information corresponding to a created MySQL container using a system as shown in fig. 4, standardized definition of MySQL container names is required. For example, it may be defined as an AA-BB-CC format. Wherein AA is an application name, BB is MySQL group number under AA application, and CC is MySQL master/slave identification. By translating the MySQL container name, the application name, the MySQL group and the primary and secondary library identification information corresponding to the MySQL container can be obtained.
For example, a set of MySQL container names may be TEST-0001-0, TEST-0001-2, TEST-0001-3. Wherein, the TEST represents the application name, 3 containers are the main and standby libraries of the TEST-0001 group, -0 represents the MySQL containers as the main library, -2 and-3 represents the MySQL containers as the standby libraries.
From the data collected by the system shown in fig. 4, two information tables, a pod information table corresponding to table 1 and a node information table corresponding to table 2, respectively, can be generated.
TABLE 1
Column name Meaning of
pod_name MySQL container name
pod_cpu MySQL container CPU allocation
pod_mem MySQL container memory allocation
pod_size MySQL containers occupy the total amount of disk
node_ip Node IP address
app_name Applications (e.g. TEST)
pod_group MySQL Container group (e.g., TEST-0001)
mysql_number MySQL container master and slave numbers (e.g., 0,2,3, etc.)
TABLE 2
Column name Meaning of
node_ip node IP address
node_cpu node CPU total
node_mem node memory total
fs_total Total file system (total disk)
fs_free File system remaining (disk remaining)
According to an embodiment of the present disclosure, a resource information table corresponding to table 3 may be generated from a pod information table corresponding to table 1 and a node information table corresponding to table 2. Wherein table 3 shows index information corresponding to the a application.
TABLE 3 Table 3
According to an embodiment of the present disclosure, the resource allocation method includes a disk occupation amount of each created container, the attribute information includes an estimated disk application amount, and the method for resource allocation as shown in fig. 3 further includes the following steps:
obtaining the maximum disk occupation amount according to the disk occupation amount of the created container corresponding to each physical machine node;
and determining the maximum disk occupation amount as the estimated disk application amount.
For example, the total MySQL container occupation disk amount may be obtained according to the pod information table corresponding to table 1, and the total MySQL container occupation disk amount is used as the disk occupation amount of the MySQL container, so as to obtain the disk occupation amount of the created MySQL container corresponding to each physical machine node.
According to the embodiment of the disclosure, the maximum disk occupation amount is obtained according to the disk occupation amount of the created container corresponding to each physical machine node, the maximum disk occupation amount is determined to be the estimated disk application amount, the estimated disk application amount is obtained according to the historical maximum disk occupation amount, sufficient disk occupation amount can be reserved for the container, and after the container is created, the data corresponding to the container can be ensured to be successfully stored.
According to the embodiment of the present disclosure, after obtaining the index information corresponding to the application a and the predicted disk application amount shown in table 3, the Node (Node) selection module 240 may complete the selection of the optimal physical machine Node according to the collected index information. The Node (Node) selection module 240 includes two components, a Node filter and a Node filter.
According to an embodiment of the present disclosure, a Node filter is configured to filter Node nodes that do not meet a preset condition, and screen N first physical machine nodes that meet the preset condition from M initial physical machine nodes.
According to an embodiment of the present disclosure, for operation S310 shown in fig. 3, acquiring resource information of N physical machine nodes included in a server cluster node may include the following operations:
Acquiring attribute information of a container and resource information of M initial physical machine nodes included in a server cluster node, wherein M is an integer greater than or equal to N;
and screening N first physical machine nodes with the storage capacity remaining quantity larger than the preset threshold value from the M initial physical machine nodes according to the preset threshold value, the attribute information and the resource information of the M initial physical machine nodes.
According to an embodiment of the present disclosure, the preset threshold may be 10% of the total amount of storage capacity.
According to an embodiment of the present disclosure, the storage capacity remaining amount includes a CPU remaining amount, a memory remaining amount, and a disk remaining amount. For example, the preset threshold may be 10% of the total amount of CPU of the initial physical machine node for the remaining amount of CPU. For the remaining amount of memory, the preset threshold may be 10% of the total amount of memory of the initial physical machine node. For the disk remaining amount, the preset threshold may be 10% of the total amount of disk of the initial physical machine node.
According to the embodiment of the disclosure, the CPU remaining amount can be obtained according to the formula (1), and the memory remaining amount can be obtained according to the formula (2). And (3) obtaining the disk residual quantity according to the formula (3).
CPU remaining amount=node node total amount of CPU-node CPU allocation amount-CPU application amount included in attribute information (1)
Memory remaining = node total memory-node memory allocation amount-cpu occupancy included in attribute information (2)
Disk remaining amount=disk remaining amount-estimated disk application amount included in attribute information (3)
According to the embodiment of the disclosure, according to the preset threshold value, the attribute information and the resource information of the M initial physical machine nodes, N first physical machine nodes with the storage capacity remaining quantity larger than the preset threshold value are screened out of the M initial physical machine nodes, preliminary screening of the M initial physical machine nodes according to the preset threshold value is achieved, N first physical machine nodes with the storage capacity remaining quantity rechecking preset threshold value are obtained, then the target physical machine nodes are obtained from the N first physical machine nodes, and therefore the calculated quantity of obtaining the target physical machine nodes can be reduced.
According to the embodiment of the disclosure, according to the preset threshold, the attribute information and the resource information of the M initial physical machine nodes, N first physical machine nodes with the storage capacity remaining amount larger than the preset threshold are screened out from the M initial physical machine nodes, so that the preset threshold is set for the storage capacity remaining amount and used for temporarily coping with the access amount of more physical machine nodes.
Fig. 5 schematically illustrates a flowchart of screening N first physical machine nodes having a storage capacity remaining greater than a preset threshold from M initial physical machine nodes according to an embodiment of the present disclosure.
As shown in fig. 5, in step S510, resource information of M initial physical machine nodes is acquired.
In step S520, it is determined whether the CPU remaining amount of the initial physical machine node is greater than a preset threshold according to the CPU occupation amount 501 included in the attribute information and the CPU remaining amount included in the resource information of the initial physical machine node.
If not, in step S530, the initial physical machine node is filtered out.
If so, in step S540, it is determined whether the memory remaining amount of the initial physical machine node is greater than a preset threshold according to the memory occupation amount 502 included in the attribute information and the memory remaining amount included in the resource information of the initial physical machine node.
If not, the initial physical machine node is filtered out in step S550.
If so, in step S560, it is determined whether the disk remaining amount of the initial physical machine node is greater than a preset threshold according to the estimated disk application amount 503 included in the attribute information and the disk remaining amount included in the resource information of the initial physical machine node.
If not, the initial physical machine node is filtered out in step S570.
If yes, in step S580, the initial physical machine node is determined to be the first physical machine node, and the available N first physical machine node lists are obtained.
According to the embodiment of the present disclosure, after the first physical machine Node list is obtained, a Node filter included in the Node (Node) selection module 240 may be used to screen a target physical machine Node from N first physical machine nodes, and the target physical machine Node is used as an optimal physical machine Node.
According to the embodiment of the disclosure, the optimal physical machine node selected from the N first physical machine nodes needs to ensure that resources such as a CPU (central processing unit), a memory (RAM), a magnetic disk (solid state disk and a mechanical hard disk) and the like of the optimal physical machine node are balanced as much as possible, and no resource hot spot occurs. For this purpose, the optimal physical machine node can be screened according to the following principle.
1) And preferentially distributing the container to the first physical machine node with more CPU residual quantity, so as to realize CPU distribution as balanced as possible.
2) And preferentially distributing the container to the first physical machine node with more memory residual quantity, so as to realize the balance of memory distribution as much as possible.
3) And preferentially distributing the containers to the first physical machine nodes with more disk residues, so as to realize balance of disk residues as much as possible.
4) In the case where the container is the master, the container is preferentially allocated to the first physical machine node including the fewer master. Because the CPU/memory used by the main library is generally much larger than that of the standby library, the main library is scattered and distributed as much as possible, so that CPU/memory hot spots caused by excessive main libraries included in the first physical machine node can be avoided.
5) In the case that the container is a backup repository, the container is preferentially allocated to the first physical machine node including more primary repositories. Because the standby libraries use less CPU/memory, excessive standby libraries included by the first physical machine node can be prevented from wasting server resources by scattering and distributing the standby libraries as much as possible.
6) The containers are preferentially allocated to the first physical machine nodes to which fewer containers are applied. Multiple containers of the same application often have the same resource usage performance, for example, the memory usage amount is large, the database data amount is large, and the like, so that the CPU, the memory and the magnetic disk can be balanced as much as possible by scattering and placing the containers of the same application.
According to an embodiment of the present disclosure, the probability that each first physical machine node is determined to be the optimal physical machine node may be calculated according to the resource allocation method as shown in fig. 3.
According to an embodiment of the present disclosure, for operation S320 shown in fig. 3, for each resource indicator, obtaining, according to resource information of N first physical machine nodes, a standard deviation corresponding to the resource indicator and weights corresponding to the N first physical machine nodes, may include the following operations:
acquiring resource index data corresponding to each of the N first physical machine nodes according to the resource information of the N first physical machine nodes to obtain N resource index data;
normalizing the N resource index data to obtain N normalized index data;
obtaining standard deviation according to the N normalization index data;
sequencing the N resource index data to obtain serial numbers corresponding to the N first physical machine nodes respectively;
Dividing the serial numbers corresponding to the N first physical machine nodes by N to obtain weights corresponding to the N first physical machine nodes.
According to embodiments of the present disclosure, for each resource indicator, the N resource indicator data may be 0-1 normalized according to equation (4), scaling the data into the [0,1] interval.
Wherein i characterizes the i-th physical machine node. y is i And characterizing normalized index data corresponding to the ith physical machine node. X is x i And characterizing the resource index data corresponding to the ith physical machine node. X is x min Representing the minimum resource index data, x in N resource index data max And characterizing the maximum resource index data in the N resource index data.
According to the common embodiment, the operation of obtaining standard deviation according to the N normalized index data can be realized according to the formula (5).
Wherein, sigma represents the standard deviation,characterization of the mean of N normalized index data。
According to the embodiment of the disclosure, according to the resource information of the N first physical machine nodes, the resource index data corresponding to the N first physical machine nodes are obtained, N resource index data are obtained, the N resource index data are normalized, N normalized index data are obtained, and then according to the N normalized index data, standard deviation is obtained, and standard deviation capable of reflecting the importance of each resource index is obtained.
According to the embodiment of the present disclosure, the larger the value of the standard deviation, the larger the difference of the resource indexes corresponding to each of the N first physical machine nodes, the more important the resource index.
According to the embodiment of the disclosure, the N resource index data are ordered to obtain the sequence numbers corresponding to the N first physical machine nodes, the sequence numbers corresponding to the N first physical machine nodes are divided by N to obtain the weights corresponding to the N first physical machine nodes, and the probability that the first physical machine node is determined to be the optimal physical machine node for each resource index is obtained.
For example, in the case where the resource index is the CPU remaining amount, the CPU remaining amounts corresponding to the N first physical machine nodes may be sorted from 1 to N, and the sorting sequence number is larger as the CPU remaining amount is larger. The ith first physical machine node is ordered as m i ,m i E {1,2,..N }, then one can useRepresenting the probability that the i-th first physical machine node is determined to be the optimal physical machine node.
According to the embodiment of the disclosure, in the case where the container is a master library and the N resource index data are ordered, as the sequence numbers corresponding to the N first physical machine nodes increase, the CPU remaining amount increases, the memory remaining amount increases, the number of disk remaining amounts increases, the number of created containers is reduced, and the number of created containers and containers belonging to the same application decreases.
According to the embodiment of the disclosure, when the container is a master library and the N resource index data are ordered, as the sequence numbers corresponding to the N first physical machine nodes respectively increase, the CPU remaining amount increases, the memory remaining amount increases, the disk remaining amount increases, the number of created containers as the master library decreases and the number of created containers and containers belonging to the same application decreases, the N resource index data corresponding to each resource index respectively can be ordered according to the principles of 1), 2), 3), 4), 6) to obtain the sequence numbers corresponding to the N first physical machine nodes respectively in the principles of 1), 2), 3), 4), 6), and then the weights corresponding to the N first physical machine nodes in the principles of 1), 2), 3), 4), 6) can be obtained subsequently.
According to the embodiment of the disclosure, in the case that the container is a standby library and the N resource index data are ordered, as the sequence numbers corresponding to the N first physical machine nodes increase, the CPU remaining amount increases, the memory remaining amount increases, the number of disk remaining amounts increases, the created container is a master library increases, and the number of created containers and containers belonging to the same application decreases.
According to the embodiment of the disclosure, when the container is a standby library and the N resource index data are ordered, as the number of the N first physical machine nodes corresponding to each N first physical machine node increases, the CPU remaining amount increases, the memory remaining amount increases, the number of the created container as a main library increases, and the number of the created container and the container belonging to the same application decreases, the N resource index data corresponding to each resource index in the principles of 1), 2), 3), 5), 6) may be ordered to obtain the number of the N first physical machine nodes corresponding to each N first physical machine node in the principles of 1), 2), 3), 5), 6), and then the weights corresponding to each N first physical machine nodes in the principles of 1), 2), 3), 5), 6) may be obtained.
According to the embodiment of the disclosure, after the standard deviation and the weight are obtained, the resource index probabilities corresponding to the N first physical machine nodes respectively may be obtained according to the formula (6).
Wherein p is i And characterizing the resource index probability corresponding to the ith first physical machine node.
According to the embodiment of the disclosure, under the condition that the standard deviation can reflect the importance of each resource index and the weights can reflect the principles of 1), 2), 3), 5) and 6), the standard deviation and the weights are multiplied to obtain the resource index probabilities corresponding to the N first physical machine nodes respectively, so that the resource index probabilities which can reflect the importance of the resource index and also reflect the principles of 1), 2), 3), 5) and 6) can be obtained.
According to the embodiment of the disclosure, under the condition that the standard deviation can reflect the importance of each resource index and the weights can reflect the principles of 1), 2), 3), 4), and 6), the standard deviation and the weights are multiplied to obtain the resource index probabilities corresponding to the N first physical machine nodes respectively, so that the resource index probabilities which can reflect the importance of the resource index and also reflect the principles of 1), 2), 3), 4), and 6) can be obtained.
According to an embodiment of the present disclosure, for operation S340 shown in fig. 3, determining a target physical machine node from N first physical machine nodes according to a resource index probability may include the following operations:
adding the resource index probabilities corresponding to the plurality of resource indexes respectively aiming at each first physical machine node to obtain total resource index probabilities corresponding to the first physical machine node, and obtaining a plurality of total resource index probabilities;
determining a maximum resource indicator probability from the plurality of total resource indicator probabilities;
and determining a first physical machine node corresponding to the maximum resource index probability in the N first physical machine nodes as a target physical machine node.
According to the embodiment of the disclosure, under the condition that the resource index probability is capable of reflecting both the importance of the resource index and the principles of 1), 2), 3), 5) and 6), adding the resource index probabilities corresponding to the plurality of resource indexes respectively for each first physical machine node to obtain total resource index probabilities corresponding to the first physical machine nodes, obtaining a plurality of total resource index probabilities which meet the principles of 1), 2), 3), 5), and 6) and reflect the resource balance degree of the first physical machine nodes, determining the maximum resource index probability from the plurality of total resource index probabilities, determining the first physical machine node corresponding to the maximum resource index probability of the N first physical machine nodes as a target physical machine node, and realizing the more accurate screening of the target physical machine node with balanced resource from the N first physical machine nodes according to the principles of 1), 2), 3), 5), and 6).
According to the embodiment of the disclosure, under the condition that the resource index probability is capable of reflecting both the importance of the resource index and the principles of 1), 2), 3), and 4), adding the resource index probabilities corresponding to the plurality of resource indexes respectively for each first physical machine node to obtain total resource index probabilities corresponding to the first physical machine nodes, obtaining a plurality of total resource index probabilities which meet the principles of 1), 2), 3), 4), and 6) and reflect the resource balance degree of the first physical machine nodes, determining the maximum resource index probability from the plurality of total resource index probabilities, determining the first physical machine node corresponding to the maximum resource index probability of the N first physical machine nodes as a target physical machine node, and realizing the more accurate screening of the target physical machine node with balanced resource from the N first physical machine nodes according to the principles of 1), 2), 3), 4), and 6).
In accordance with an embodiment of the present disclosure, upon obtaining the target physical machine node, the task execution module 220 may implement creating a container on the target physical machine node according to the operations illustrated in FIG. 6.
Fig. 6 schematically illustrates a flow diagram for creating a container on a target physical machine node according to an embodiment of the disclosure.
As shown in fig. 6, in step S610, a task work order is queried.
In step S620, attribute information corresponding to the container is acquired according to the job ticket.
In step S630, the target physical machine node is acquired.
In step S640, the call API creates a container on the target physical machine node according to the attribute information. Wherein the API characterizes an application program interface corresponding to an application implementing container creation.
In step S650, it is determined whether the creation of the container is completed.
If yes, in step S660, the execution result is fed back.
If not, the process returns to step S650, and the process continues to step S650.
It should be noted that, unless there is an execution sequence between different operations or an execution sequence between different operations in technical implementation, the execution sequence between multiple operations may be different, and multiple operations may also be executed simultaneously in the embodiment of the disclosure.
Based on the resource allocation method, the disclosure also provides a resource allocation device. The device will be described in detail below in connection with fig. 7.
Fig. 7 schematically illustrates a block diagram of a resource allocation apparatus according to an embodiment of the present disclosure.
As shown in fig. 7, the resource configuration device 700 of this embodiment includes an acquisition module 710, a first obtaining module 720, a second obtaining module 730, a determining module 740, and a creating module 750.
The obtaining module 710 is configured to obtain resource information of N first physical machine nodes included in the server cluster node, where the resource information includes a plurality of resource indexes, and N is an integer greater than or equal to 1. In an embodiment, the obtaining module 710 may be configured to perform the operation S310 described above, which is not described herein.
The first obtaining module 720 is configured to obtain, for each resource indicator, a standard deviation corresponding to the resource indicator and weights corresponding to the N first physical machine nodes according to the resource information of the N first physical machine nodes. In an embodiment, the first obtaining module 720 may be configured to perform the operation S320 described above, which is not described herein.
And a second obtaining module 730, configured to obtain the resource indicator probabilities corresponding to the N first physical machine nodes according to the standard deviation and the weights. In an embodiment, the second obtaining module 730 may be used to perform the operation S330 described above, which is not described herein.
And the determining module 740 is configured to determine a target physical machine node from the N first physical machine nodes according to the resource indicator probability. In an embodiment, the determining module 740 may be configured to perform the operation S340 described above, which is not described herein.
A creation module 750 creates a container on the target physical machine node. In an embodiment, the creation module 750 may be configured to perform the operation S350 described above, which is not described herein.
According to an embodiment of the disclosure, the first obtaining module includes a resource index data obtaining sub-module, a remaining amount obtaining sub-module, a standard deviation obtaining sub-module, a sequence number obtaining sub-module, and a weight obtaining sub-module.
The resource index data obtaining sub-module is used for obtaining the resource index data corresponding to the N first physical machine nodes according to the resource information of the N first physical machine nodes, and obtaining N resource index data.
And the residual quantity obtaining sub-module is used for normalizing the N pieces of resource index data to obtain N pieces of normalized index data.
The standard deviation obtaining submodule is used for obtaining standard deviations according to the N normalized index data.
The sequence number obtaining sub-module is used for sequencing the N resource index data to obtain sequence numbers corresponding to the N first physical machine nodes respectively;
and the weight obtaining sub-module is used for dividing the serial numbers corresponding to the N first physical machine nodes by N to obtain the weights corresponding to the N first physical machine nodes.
According to an embodiment of the present disclosure, the plurality of resource indicators includes at least one of: CPU remaining amount, memory remaining amount, disk remaining amount, number of created containers as master library and number of created containers belonging to the same application as the containers corresponding to each of the N first physical machine nodes.
According to the embodiment of the disclosure, in the case where the container is a master library and the N resource index data are ordered, as the sequence numbers corresponding to the N first physical machine nodes increase, the CPU remaining amount increases, the memory remaining amount increases, the number of disk remaining amounts increases, the number of created containers is reduced, and the number of created containers and containers belonging to the same application decreases.
According to the embodiment of the disclosure, in the case that the container is a standby library and the N resource index data are ordered, as the sequence numbers corresponding to the N first physical machine nodes increase, the CPU remaining amount increases, the memory remaining amount increases, the number of disk remaining amounts increases, the created container is a master library increases, and the number of created containers and containers belonging to the same application decreases.
According to an embodiment of the disclosure, the determining module includes a total resource indicator probability obtaining sub-module, a maximum probability obtaining sub-module, and a determining sub-module.
The total resource index probability obtaining sub-module is used for adding the resource index probabilities corresponding to the plurality of resource indexes respectively for each first physical machine node to obtain the total resource index probability corresponding to the first physical machine node, and obtaining the plurality of total resource index probabilities.
The maximum probability obtaining sub-module is used for determining the maximum resource index probability from the plurality of total resource index probabilities.
And the determining submodule is used for determining a first physical machine node corresponding to the maximum resource index probability from the N first physical machine nodes as a target physical machine node.
According to an embodiment of the present disclosure, the acquisition module includes an acquisition sub-module and a screening sub-module.
And the acquisition sub-module is used for acquiring the attribute information of the container and the resource information of M initial physical machine nodes included in the server cluster node, wherein M is an integer greater than or equal to N.
And the screening sub-module is used for screening N first physical machine nodes with the storage capacity remaining quantity larger than the preset threshold value from the M initial physical machine nodes according to the preset threshold value, the attribute information and the resource information of the M initial physical machine nodes.
According to the embodiment of the disclosure, the resource allocation device further comprises a maximum disk occupation amount obtaining module and an estimated disk application amount determining module.
And the maximum disk occupation amount obtaining module is used for obtaining the maximum disk occupation amount according to the disk occupation amount of the created container corresponding to each physical machine node.
The estimated disk application amount determining module is used for determining the maximum disk occupation amount as the estimated disk application amount.
According to an embodiment of the present disclosure, any of the plurality of modules of the acquisition module 710, the first obtaining module 720, the second obtaining module 730, the determining module 740, and the creating module 750 may be combined in one module to be implemented, or any of the plurality of modules may be split into a plurality of modules. Alternatively, at least some of the functionality of one or more of the modules may be combined with at least some of the functionality of other modules and implemented in one module. According to embodiments of the present disclosure, at least one of the acquisition module 710, the first acquisition module 720, the second acquisition module 730, the determination module 740, and the creation module 750 may be implemented at least in part as hardware circuitry, such as a Field Programmable Gate Array (FPGA), a Programmable Logic Array (PLA), a system on a chip, a system on a substrate, a system on a package, an Application Specific Integrated Circuit (ASIC), or in hardware or firmware in any other reasonable manner of integrating or packaging the circuitry, or in any one of or a suitable combination of three of software, hardware, and firmware. Alternatively, at least one of the acquisition module 710, the first acquisition module 720, the second acquisition module 730, the determination module 740, and the creation module 750 may be at least partially implemented as computer program modules, which when executed, may perform the respective functions.
Fig. 8 schematically illustrates a block diagram of an electronic device adapted to implement a resource allocation method according to an embodiment of the disclosure.
As shown in fig. 8, an electronic device 800 according to an embodiment of the present disclosure includes a processor 801 that can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM) 802 or a program loaded from a storage section 808 into a Random Access Memory (RAM) 803. The processor 801 may include, for example, a general purpose microprocessor (e.g., a CPU), an instruction set processor and/or an associated chipset and/or a special purpose microprocessor (e.g., an Application Specific Integrated Circuit (ASIC)), or the like. The processor 801 may also include on-board memory for caching purposes. The processor 801 may include a single processing unit or multiple processing units for performing the different actions of the method flows according to embodiments of the disclosure.
In the RAM803, various programs and data required for the operation of the electronic device 800 are stored. The processor 801, the ROM 802, and the RAM803 are connected to each other by a bus 804. The processor 801 performs various operations of the method flow according to the embodiments of the present disclosure by executing programs in the ROM 802 and/or the RAM 803. Note that the program may be stored in one or more memories other than the ROM 802 and the RAM 803. The processor 801 may also perform various operations of the method flows according to embodiments of the present disclosure by executing programs stored in the one or more memories.
According to an embodiment of the present disclosure, the electronic device 800 may also include an input/output (I/O) interface 805, the input/output (I/O) interface 805 also being connected to the bus 804. The electronic device 800 may also include one or more of the following components connected to an input/output (I/O) interface 805: an input portion 806 including a keyboard, mouse, etc.; an output portion 807 including a display such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and a speaker; a storage section 808 including a hard disk or the like; and a communication section 809 including a network interface card such as a LAN card, a modem, or the like. The communication section 809 performs communication processing via a network such as the internet. The drive 810 is also connected to an input/output (I/O) interface 805 as needed. A removable medium 811 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 810 as needed so that a computer program read out therefrom is mounted into the storage section 808 as needed.
The present disclosure also provides a computer-readable storage medium that may be embodied in the apparatus/device/system described in the above embodiments; or may exist alone without being assembled into the apparatus/device/system. The computer-readable storage medium carries one or more programs which, when executed, implement methods in accordance with embodiments of the present disclosure.
According to embodiments of the present disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium, which may include, for example, but is not limited to: a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this disclosure, a computer-readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. For example, according to embodiments of the present disclosure, the computer-readable storage medium may include ROM 802 and/or RAM 803 and/or one or more memories other than ROM 802 and RAM 803 described above.
Embodiments of the present disclosure also include a computer program product comprising a computer program containing program code for performing the methods shown in the flowcharts. The program code, when executed in a computer system, causes the computer system to implement the resource allocation method provided by embodiments of the present disclosure.
The above-described functions defined in the system/apparatus of the embodiments of the present disclosure are performed when the computer program is executed by the processor 801. The systems, apparatus, modules, units, etc. described above may be implemented by computer program modules according to embodiments of the disclosure.
In one embodiment, the computer program may be based on a tangible storage medium such as an optical storage device, a magnetic storage device, or the like. In another embodiment, the computer program may also be transmitted, distributed, and downloaded and installed in the form of a signal on a network medium, and/or from a removable medium 811 via a communication portion 809. The computer program may include program code that may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the foregoing.
In such an embodiment, the computer program may be downloaded and installed from a network via the communication section 809, and/or installed from the removable media 811. The above-described functions defined in the system of the embodiments of the present disclosure are performed when the computer program is executed by the processor 801. The systems, devices, apparatus, modules, units, etc. described above may be implemented by computer program modules according to embodiments of the disclosure.
According to embodiments of the present disclosure, program code for performing computer programs provided by embodiments of the present disclosure may be written in any combination of one or more programming languages, and in particular, such computer programs may be implemented in high-level procedural and/or object-oriented programming languages, and/or assembly/machine languages. Programming languages include, but are not limited to, such as Java, c++, python, "C" or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, partly on a remote computing device, or entirely on the remote computing device or server. In the case of remote computing devices, the remote computing device may be connected to the user computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computing device (e.g., connected via the Internet using an Internet service provider).
The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
Those skilled in the art will appreciate that the features recited in the various embodiments of the disclosure and/or in the claims may be provided in a variety of combinations and/or combinations, even if such combinations or combinations are not explicitly recited in the disclosure. In particular, the features recited in the various embodiments of the present disclosure and/or the claims may be variously combined and/or combined without departing from the spirit and teachings of the present disclosure. All such combinations and/or combinations fall within the scope of the present disclosure.
The embodiments of the present disclosure are described above. However, these examples are for illustrative purposes only and are not intended to limit the scope of the present disclosure. Although the embodiments are described above separately, this does not mean that the measures in the embodiments cannot be used advantageously in combination. The scope of the disclosure is defined by the appended claims and equivalents thereof. Various alternatives and modifications can be made by those skilled in the art without departing from the scope of the disclosure, and such alternatives and modifications are intended to fall within the scope of the disclosure.

Claims (11)

1. A resource allocation method, comprising:
acquiring resource information of N first physical machine nodes included in a server cluster node, wherein the resource information comprises a plurality of resource indexes, and N is an integer greater than or equal to 1;
For each resource index, according to the resource information of the N first physical machine nodes, standard deviation corresponding to the resource index and weight corresponding to each of the N first physical machine nodes are obtained;
obtaining the resource index probabilities corresponding to the N first physical machine nodes according to the standard deviation and the weights;
determining a target physical machine node from the N first physical machine nodes according to the resource index probability; and
a container is created on the target physical machine node.
2. The method of claim 1, wherein the obtaining, for each of the resource metrics, a standard deviation corresponding to the resource metrics and weights corresponding to the N first physical machine nodes according to the resource information of the N first physical machine nodes includes:
acquiring resource index data corresponding to the N first physical machine nodes according to the resource information of the N first physical machine nodes, and obtaining N resource index data;
normalizing the N resource index data to obtain N normalized index data;
obtaining the standard deviation according to the N normalization index data;
Sequencing the N resource index data to obtain serial numbers corresponding to the N first physical machine nodes respectively;
dividing the serial numbers corresponding to the N first physical machine nodes by N to obtain weights corresponding to the N first physical machine nodes.
3. The method of claim 2, wherein the plurality of resource metrics comprise at least one of: CPU residual quantity, memory residual quantity, disk residual quantity, quantity of created containers as main library and quantity of created containers belonging to same application with said containers which are respectively correspondent to said N first physical machine nodes.
4. The method according to claim 3, wherein, in a case where the container is a master library and the N pieces of resource index data are ordered, as the sequence numbers corresponding to the N first physical machine nodes respectively increase, the CPU remaining amount increases, the memory remaining amount increases, the number of disk remaining amounts increases, the number of created containers is a master library decreases, and the number of created containers and the containers belonging to the same application decreases.
5. The method of claim 3, wherein, in a case where the container is a standby, and the N resource index data are ordered, as the sequence numbers corresponding to the N first physical machine nodes respectively increase, the CPU remaining amount increases, the memory remaining amount increases, the number of disk remaining amounts increases, the created container is a master, and the number of created containers belonging to the same application as the container decreases.
6. The method of any one of claims 1 to 5, wherein said determining a target physical machine node from the N first physical machine nodes according to the resource indicator probability comprises:
adding the resource index probabilities corresponding to the plurality of resource indexes respectively for each first physical machine node to obtain total resource index probabilities corresponding to the first physical machine node, and obtaining a plurality of total resource index probabilities;
determining a maximum resource indicator probability from the plurality of total resource indicator probabilities;
and determining a first physical machine node corresponding to the maximum resource index probability in the N first physical machine nodes as the target physical machine node.
7. The method of claim 1, wherein the obtaining resource information of N physical machine nodes included in the server cluster node includes:
acquiring attribute information of the container and resource information of M initial physical machine nodes included in the server cluster node, wherein M is an integer greater than or equal to N;
and screening the N first physical machine nodes with the storage capacity remaining quantity larger than a preset threshold value from the M initial physical machine nodes according to the preset threshold value, the attribute information and the resource information of the M initial physical machine nodes.
8. The method of claim 7, wherein the resource information includes disk occupancy of each created container, the attribute information includes a predicted disk application amount, and further comprising:
obtaining the maximum disk occupation amount according to the disk occupation amount of the created container corresponding to each physical machine node;
and determining the maximum disk occupation amount as the estimated disk application amount.
9. A resource allocation apparatus, comprising:
the acquisition module is used for acquiring resource information of N first physical machine nodes included in the server cluster node, wherein the resource information comprises a plurality of resource indexes, and N is an integer greater than or equal to 1;
the first obtaining module is used for obtaining standard deviation corresponding to the resource indexes and weights corresponding to the N first physical machine nodes according to the resource information of the N first physical machine nodes for each resource index;
the second obtaining module is used for obtaining the resource index probabilities corresponding to the N first physical machine nodes respectively according to the standard deviation and the weights;
the determining module is used for determining a target physical machine node from the N first physical machine nodes according to the resource index probability; and
And the creation module is used for creating a container on the target physical machine node.
10. An electronic device, comprising:
one or more processors;
storage means for storing one or more programs,
wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to perform the method of any of claims 1-8.
11. A computer readable storage medium having stored thereon executable instructions which, when executed by a processor, cause the processor to perform the method according to any of claims 1-8.
CN202311035476.4A 2023-08-16 2023-08-16 Resource allocation method, device, electronic equipment and storage medium Pending CN117435337A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202311035476.4A CN117435337A (en) 2023-08-16 2023-08-16 Resource allocation method, device, electronic equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202311035476.4A CN117435337A (en) 2023-08-16 2023-08-16 Resource allocation method, device, electronic equipment and storage medium

Publications (1)

Publication Number Publication Date
CN117435337A true CN117435337A (en) 2024-01-23

Family

ID=89548726

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202311035476.4A Pending CN117435337A (en) 2023-08-16 2023-08-16 Resource allocation method, device, electronic equipment and storage medium

Country Status (1)

Country Link
CN (1) CN117435337A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117971505A (en) * 2024-03-29 2024-05-03 苏州元脑智能科技有限公司 Method and device for deploying container application

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117971505A (en) * 2024-03-29 2024-05-03 苏州元脑智能科技有限公司 Method and device for deploying container application
CN117971505B (en) * 2024-03-29 2024-06-07 苏州元脑智能科技有限公司 Method and device for deploying container application

Similar Documents

Publication Publication Date Title
CN109241772B (en) Invoice block chain recording method, invoice block chain recording device, block chain gateway server and invoice block chain recording medium
CN110213128B (en) Service port detection method, electronic device and computer storage medium
CN111506404A (en) Kubernetes-based shared GPU (graphics processing Unit) scheduling method
CN106790744B (en) IP scheduling method and system
CN111917587B (en) Method for network service management by using service system and service system
CN114844791B (en) Cloud service automatic management and distribution method and system based on big data and storage medium
US20080201474A1 (en) Computer system
CN117435337A (en) Resource allocation method, device, electronic equipment and storage medium
CN109152061A (en) Channel agility method, apparatus, server and storage medium
CN105468619A (en) Resource distribution method and device used for database connection pool
CN107943423A (en) The management method and computer-readable recording medium of storage resource in cloud system
CN114817050A (en) Task execution method and device, electronic equipment and computer readable storage medium
CN113537845A (en) Task distribution method and device, electronic equipment and computer readable storage medium
CN112214321B (en) Node selection method and device for newly added micro service and micro service management platform
KR101310420B1 (en) User-definable Process-based Management System for Urban Planning and Recording Media for the Same
CN115080253B (en) GPU task allocation method and device, electronic equipment and storage medium
US10936968B2 (en) Ticket routing
CN114896073A (en) Data allocation processing method, device and system
CN110851483B (en) Method, apparatus, electronic device, and medium for screening objects
CN114493756A (en) Resource management method, device, equipment and storage medium
CN110033145B (en) Financial sharing job order separating method and device, equipment and storage medium
CN106528577B (en) Method and device for setting file to be cleaned
US20220172131A1 (en) Resource allocation device, resource allocation method, and resource allocation program
CN111338752B (en) Container adjusting method and device
US20240020168A1 (en) Resource Scheduling Method and Apparatus, Computer Device, System, and Storage Medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination