CN117971510B - Resource allocation method, device, electronic equipment and storage medium - Google Patents

Resource allocation method, device, electronic equipment and storage medium Download PDF

Info

Publication number
CN117971510B
CN117971510B CN202410383638.1A CN202410383638A CN117971510B CN 117971510 B CN117971510 B CN 117971510B CN 202410383638 A CN202410383638 A CN 202410383638A CN 117971510 B CN117971510 B CN 117971510B
Authority
CN
China
Prior art keywords
resource
server
calculated
current
request object
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202410383638.1A
Other languages
Chinese (zh)
Other versions
CN117971510A (en
Inventor
李铁刚
江勇慧
杨灵军
夏军
屈道超
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hundsun Technologies Inc
Original Assignee
Hundsun Technologies Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hundsun Technologies Inc filed Critical Hundsun Technologies Inc
Priority to CN202410383638.1A priority Critical patent/CN117971510B/en
Publication of CN117971510A publication Critical patent/CN117971510A/en
Application granted granted Critical
Publication of CN117971510B publication Critical patent/CN117971510B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Landscapes

  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)
  • Debugging And Monitoring (AREA)

Abstract

The application provides a resource allocation method, a resource allocation device, electronic equipment and a storage medium, and relates to the technical field of communication. The method comprises the following steps: acquiring resource information of each server in a server cluster and resource demand information of a plurality of resource request objects; determining an initial resource allocation result under each resource dimension of the server according to the resource information of each server and the resource demand information of each resource request object; and determining a target resource allocation result meeting the resource demand information of each resource request object according to the initial resource allocation result under each resource dimension. By forming a server cluster by the old server and the new server, collocation and utilization of the new and old server resources can be realized, and the maximum utilization of the resources can be realized. By dimension division of the resources of the server to fit the resources under each resource dimension, the accuracy of the resource allocation result under a single dimension can be improved, and the accuracy of the target resource allocation result after fusion processing is further improved.

Description

Resource allocation method, device, electronic equipment and storage medium
Technical Field
The present application relates to the field of communications technologies, and in particular, to a resource allocation method, a device, an electronic apparatus, and a storage medium.
Background
Deployment of a distributed cluster system often involves installation and deployment of applications, and needs to reasonably allocate deployment relationships between applications and servers to achieve reasonable allocation and maximum utilization of server resources.
In the prior art, a server capable of meeting deployment requirements is generally determined according to resource requirement information of a cluster system to be deployed, so that the server meeting the requirements is purchased, and each application is deployed on each server meeting the requirements according to a deployment relationship between a pre-configured server and the application.
However, the method can lead to deployment of a new distributed cluster system, and after a large number of new servers meeting the requirements are introduced, a large number of old servers are wasted resources, so that the existing server resources can not be reasonably utilized to complete deployment.
Disclosure of Invention
The application aims to provide a resource allocation method, a device, electronic equipment and a storage medium for overcoming the defects in the prior art, so that the resource fusion utilization of new and old servers is realized, and the resource utilization rate is improved.
In order to achieve the above purpose, the technical scheme adopted by the embodiment of the application is as follows:
in a first aspect, an embodiment of the present application provides a resource allocation method, including:
Acquiring resource information of each server in a server cluster and resource demand information of a plurality of resource request objects;
determining an initial resource allocation result under each resource dimension of the server according to the resource information of each server and the resource demand information of each resource request object;
And determining a target resource allocation result meeting the resource demand information of each resource request object according to the initial resource allocation result under each resource dimension, wherein the target resource allocation result is used for indicating the identification of the server to be allocated to each resource request object.
Optionally, the determining, according to the initial resource allocation result under each resource dimension, a target resource allocation result satisfying the resource requirement information of each resource request object includes:
Carrying out feasibility screening on the initial resource allocation result under each resource dimension to obtain a to-be-selected resource allocation result under each resource dimension;
And carrying out fusion processing on the to-be-selected resource allocation results in each resource dimension to obtain the target resource allocation result.
Optionally, the performing feasibility screening on the initial resource allocation result in each resource dimension to obtain a candidate resource allocation result in each resource dimension includes:
And adopting a preset screening strategy to respectively carry out feasibility screening on the initial resource allocation result in each resource dimension to obtain the resource allocation result to be selected in each resource dimension.
Optionally, the preset screening policy at least includes: the method comprises the steps that a first type of resource request object and a second type of resource request object are not allowed to be distributed on the same server, the sum of standard memory requirements or average memory requirements of the resource request objects distributed on the same server is smaller than the preset proportion of the memory of the server, the task processing peak time of the resource request objects distributed on the same server is not overlapped, the sum of data increment of the resource request objects distributed on the same server is smaller than or equal to the hard disk resource of the server, the same resource request object is not allowed to be distributed on the same server, and the network peak time of the resource request objects distributed on the same server is not overlapped.
Optionally, the fusing the resource allocation results to be selected in each resource dimension to obtain the target resource allocation result includes:
And carrying out intersection processing on the resource allocation results to be selected in each resource dimension to obtain the target resource allocation result.
Optionally, the determining the initial resource allocation result under each resource dimension of the server according to the resource information of each server and the resource requirement information of each resource request object includes:
determining a plurality of resource allocation sequences, each of the resource allocation sequences being used for indicating a resource allocation calculation order of each resource request object;
and determining an initial resource allocation result in the current resource dimension of the server according to the resource information in the current resource dimension of each server, the resource sub-demand information of each resource request object in the current resource dimension and the plurality of resource allocation sequences.
Optionally, the determining an initial resource allocation result in the current resource dimension of the server according to the resource information in the current resource dimension of each server, the resource sub-requirement information of each resource request object in the current resource dimension, and the plurality of resource allocation sequences includes:
Determining at least one intermediate resource allocation result under each resource allocation sequence according to the resource parameter of each server under the current resource dimension, the resource sub-demand information of each resource request object under the current resource dimension and the resource allocation calculation sequence indicated by each resource allocation sequence;
And carrying out fusion processing on at least one intermediate resource allocation result under each resource allocation sequence to obtain an initial resource allocation result under the current resource dimension.
Optionally, the determining at least one intermediate resource allocation result under each resource allocation sequence according to the resource parameter under the current resource dimension of each server, the resource sub-requirement information of each resource request object under the current resource dimension, and the resource allocation calculation sequence indicated by each resource allocation sequence includes:
Determining whether the current to-be-calculated server allows allocation of the current to-be-calculated resource request object according to the resource sub-demand information of the current to-be-calculated resource request object in the current resource dimension and the resource parameter of the current to-be-calculated server in the current resource dimension;
if the server to be calculated before allows the allocation of the current resource request object to be calculated, generating a resource allocation result between the current resource request object to be calculated and the current server to be calculated in the current resource dimension; updating the resource parameters of the current to-be-calculated server in the current resource dimension according to the resource sub-demand information of the current to-be-calculated resource request object in the current resource dimension, determining whether the current to-be-calculated server allows the next to-be-calculated resource request object to be allocated or not based on the updated resource parameters of the current to-be-calculated server in the current resource dimension until the resource parameters of the current to-be-calculated server in the current resource dimension are smaller than a preset threshold, calculating the next to-be-calculated server as the current new to-be-calculated server, and circularly executing until each resource request object finishes resource allocation according to the resource allocation calculation sequence indicated by the current resource allocation sequence, and obtaining at least one intermediate resource allocation result under the current resource allocation sequence; the server to be calculated corresponding to the server to be calculated currently and the resource request object to be calculated corresponding to the resource request object to be calculated currently are determined according to the current resource allocation sequence.
Optionally, the current resource dimension is a memory resource of the server; the determining whether the current to-be-calculated server allows the allocation of the current to-be-calculated resource request object according to the resource sub-demand information of the current to-be-calculated resource request object in the current resource dimension and the resource parameter of the current to-be-calculated server in the current resource dimension includes:
And determining whether the current to-be-calculated server allows the allocation of the current to-be-calculated resource request object according to the memory use requirement of the current to-be-calculated resource request object, the memory size of the current to-be-calculated server and the preset memory reserved size of the server.
Optionally, the determining whether the current to-be-calculated server allows the allocation of the current to-be-calculated resource request object according to the memory usage requirement of the current to-be-calculated resource request object, the memory size of the current to-be-calculated server, and the preset server memory reservation size includes:
determining the available memory size of the current server to be calculated according to the memory size of the current server to be calculated and the preset reserved memory size of the server;
And if the memory use requirement of the current resource request object to be calculated is smaller than or equal to the available memory size of the current server to be calculated, determining that the current server to be calculated allows the current resource request object to be allocated.
Optionally, the current resource dimension is a processor resource of a server; the determining whether the current to-be-calculated server allows the allocation of the current to-be-calculated resource request object according to the resource sub-demand information of the current to-be-calculated resource request object in the current resource dimension and the resource parameter of the current to-be-calculated server in the current resource dimension includes:
And determining whether the current to-be-calculated server allows the allocation of the current to-be-calculated resource request object according to the model of the processor supported by the current to-be-calculated resource request object, the average core number of the processor required by the current to-be-calculated resource request object, the minimum processor frequency required by the current to-be-calculated resource request object, the model of the processor of the current to-be-calculated server, the processor frequency of the current to-be-calculated server and the core number of the processor of the current to-be-calculated server.
Optionally, the determining whether the current to-be-calculated server allows allocation of the current to-be-calculated resource request object according to the model of the processor supported by the current to-be-calculated resource request object, the average core number of the processor required by the current to-be-calculated resource request object, the minimum processor frequency required by the current to-be-calculated resource request object, the model of the processor of the current to-be-calculated server, the processor frequency of the current to-be-calculated server, and the core number of the processor of the current to-be-calculated server includes:
determining whether the current to-be-calculated server allows allocation of the current to-be-calculated resource request object according to the model of a processor supported by the current to-be-calculated resource request object, the average core number of the processor required by the current to-be-calculated resource request object, the minimum processor frequency required by the current to-be-calculated resource request object, the model of the processor of the current to-be-calculated server, the converted processor frequency of the current to-be-calculated server and the processor core number of the current to-be-calculated server; or alternatively
And determining whether the current to-be-calculated server allows the allocation of the current to-be-calculated resource request object according to the model of the processor supported by the current to-be-calculated resource request object, the average core number of the processor required by the current to-be-calculated resource request object, the minimum processor frequency required by the current to-be-calculated resource request object, the model of the processor of the current to-be-calculated server, the processor frequency of the current to-be-calculated server and the converted processor core number of the current to-be-calculated server.
Optionally, the scaled processor frequency of the current server to be calculated and the scaled processor core number of the current server to be calculated are calculated in the following manner:
inquiring to obtain performance proportion parameters of the processor of the server to be calculated relative to a reference model processor according to the model of the processor of the server to be calculated;
According to the performance proportion parameters of the processor of the server to be calculated currently, equivalent conversion is carried out on the processor frequency of the server to be calculated currently, and converted processor frequency of the server to be calculated currently is obtained;
And carrying out equivalent conversion on the processor core number of the server to be calculated according to the performance proportion parameters of the processor of the server to be calculated, so as to obtain the converted processor core number of the server to be calculated.
Optionally, the determining whether the current to-be-calculated server allows allocation of the current to-be-calculated resource request object according to the model of the processor supported by the current to-be-calculated resource request object, the average core number of the processor required by the current to-be-calculated resource request object, the minimum processor frequency required by the current to-be-calculated resource request object, the model of the processor of the current to-be-calculated server, the converted processor frequency of the current to-be-calculated server, and the processor core number of the current to-be-calculated server includes:
If the model of the processor of the server to be calculated currently belongs to the model of the processor supported by the resource request to be calculated currently; the number of processor cores of the server to be calculated currently is larger than or equal to the average number of processor cores required by the resource request to be calculated currently; and the converted processor frequency of the current to-be-calculated server is greater than or equal to the minimum processor frequency required by the current to-be-calculated resource request, determining that the current to-be-calculated server allows the current to-be-calculated resource request to be allocated.
In a second aspect, an embodiment of the present application further provides a resource allocation apparatus, including: the device comprises an acquisition module and a determination module;
The acquisition module is used for acquiring the resource information of each server in the server cluster and the resource demand information of a plurality of resource request objects;
the determining module is used for determining an initial resource allocation result under each resource dimension of the server according to the resource information of each server and the resource demand information of each resource request object;
The determining module is configured to determine, according to an initial resource allocation result in each resource dimension, a target resource allocation result that satisfies resource requirement information of each resource request object, where the target resource allocation result is used to indicate an identifier of a server to be allocated to each resource request object.
Optionally, the determining module is specifically configured to perform feasibility screening on an initial resource allocation result in each resource dimension to obtain a candidate resource allocation result in each resource dimension;
And carrying out fusion processing on the to-be-selected resource allocation results in each resource dimension to obtain the target resource allocation result.
Optionally, the determining module is specifically configured to adopt a preset screening policy to perform feasibility screening on the initial resource allocation result in each resource dimension, so as to obtain a candidate resource allocation result in each resource dimension.
Optionally, the preset screening policy at least includes: the method comprises the steps that a first type of resource request object and a second type of resource request object are not allowed to be distributed on the same server, the sum of standard memory requirements or average memory requirements of the resource request objects distributed on the same server is smaller than the preset proportion of the memory of the server, the task processing peak time of the resource request objects distributed on the same server is not overlapped, the sum of data increment of the resource request objects distributed on the same server is smaller than or equal to the hard disk resource of the server, the same resource request object is not allowed to be distributed on the same server, and the network peak time of the resource request objects distributed on the same server is not overlapped.
Optionally, the determining module is specifically configured to perform intersection processing on the candidate resource allocation result in each resource dimension, so as to obtain the target resource allocation result.
Optionally, the determining module is specifically configured to determine a plurality of resource allocation sequences, where each resource allocation sequence is used to indicate a resource allocation calculation sequence of each resource request object;
and determining an initial resource allocation result in the current resource dimension of the server according to the resource information in the current resource dimension of each server, the resource sub-demand information of each resource request object in the current resource dimension and the plurality of resource allocation sequences.
Optionally, the determining module is specifically configured to determine at least one intermediate resource allocation result under each resource allocation sequence according to a resource parameter under a current resource dimension of each server, resource sub-requirement information of each resource request object under the current resource dimension, and a resource allocation calculation sequence indicated by each resource allocation sequence;
And carrying out fusion processing on at least one intermediate resource allocation result under each resource allocation sequence to obtain an initial resource allocation result under the current resource dimension.
Optionally, the determining module is specifically configured to determine, according to the resource sub-requirement information of the current resource dimension of the current resource request object to be calculated and the resource parameter of the current resource dimension of the current server to be calculated, whether the current server to be calculated allows allocation of the current resource request object to be calculated;
if the server to be calculated before allows the allocation of the current resource request object to be calculated, generating a resource allocation result between the current resource request object to be calculated and the current server to be calculated in the current resource dimension; updating the resource parameters of the current to-be-calculated server in the current resource dimension according to the resource sub-demand information of the current to-be-calculated resource request object in the current resource dimension, determining whether the current to-be-calculated server allows the next to-be-calculated resource request object to be allocated or not based on the updated resource parameters of the current to-be-calculated server in the current resource dimension until the resource parameters of the current to-be-calculated server in the current resource dimension are smaller than a preset threshold, calculating the next to-be-calculated server as the current new to-be-calculated server, and circularly executing until each resource request object finishes resource allocation according to the resource allocation calculation sequence indicated by the current resource allocation sequence, and obtaining at least one intermediate resource allocation result under the current resource allocation sequence; the server to be calculated corresponding to the server to be calculated currently and the resource request object to be calculated corresponding to the resource request object to be calculated currently are determined according to the current resource allocation sequence.
Optionally, the current resource dimension is a memory resource of the server; the determining module is specifically configured to determine whether the current server to be calculated allows allocation of the current resource request object to be calculated according to a memory usage requirement of the current resource request object to be calculated, a memory size of the current server to be calculated, and a preset memory reservation size of the server.
Optionally, the determining module is specifically configured to determine, according to a memory size of a server to be calculated currently and a preset reserved size of a server memory, an available memory size of the server to be calculated currently;
And if the memory use requirement of the current resource request object to be calculated is smaller than or equal to the available memory size of the current server to be calculated, determining that the current server to be calculated allows the current resource request object to be allocated.
Optionally, the current resource dimension is a processor resource of a server; the determining module is specifically configured to determine whether the current to-be-calculated server allows allocation of the current to-be-calculated resource request object according to a model of a processor supported by the current to-be-calculated resource request object, an average core number of the processor required by the current to-be-calculated resource request object, a minimum processor frequency required by the current to-be-calculated resource request object, a model of a processor of the current to-be-calculated server, a processor frequency of the current to-be-calculated server, and a processor core number of the current to-be-calculated server.
Optionally, the determining module is specifically configured to determine whether the current to-be-calculated server allows allocation of the current to-be-calculated resource request object according to a model of a processor supported by the current to-be-calculated resource request object, an average core number of the processor required by the current to-be-calculated resource request object, a minimum processor frequency required by the current to-be-calculated resource request object, a model of a processor of the current to-be-calculated server, a converted processor frequency of the current to-be-calculated server, and a processor core number of the current to-be-calculated server; or alternatively
And determining whether the current to-be-calculated server allows the allocation of the current to-be-calculated resource request object according to the model of the processor supported by the current to-be-calculated resource request object, the average core number of the processor required by the current to-be-calculated resource request object, the minimum processor frequency required by the current to-be-calculated resource request object, the model of the processor of the current to-be-calculated server, the processor frequency of the current to-be-calculated server and the converted processor core number of the current to-be-calculated server.
Optionally, the method further comprises: a conversion module; the conversion module is used for inquiring and obtaining performance proportion parameters of the processor of the server to be calculated relative to the reference model processor according to the model of the processor of the server to be calculated;
According to the performance proportion parameters of the processor of the server to be calculated currently, equivalent conversion is carried out on the processor frequency of the server to be calculated currently, and converted processor frequency of the server to be calculated currently is obtained;
And carrying out equivalent conversion on the processor core number of the server to be calculated according to the performance proportion parameters of the processor of the server to be calculated, so as to obtain the converted processor core number of the server to be calculated.
Optionally, the determining module is specifically configured to determine that the model of the processor of the server to be calculated currently belongs to the model of the processor supported by the resource request to be calculated currently; the number of processor cores of the server to be calculated currently is larger than or equal to the average number of processor cores required by the resource request to be calculated currently; and the converted processor frequency of the current to-be-calculated server is greater than or equal to the minimum processor frequency required by the current to-be-calculated resource request, determining that the current to-be-calculated server allows the current to-be-calculated resource request to be allocated.
In a third aspect, an embodiment of the present application provides an electronic device, including: a processor, a storage medium, and a bus, the storage medium storing machine-readable instructions executable by the processor, the processor and the storage medium in communication over the bus when the electronic device is operating, the processor executing the machine-readable instructions to implement the resource allocation method as provided in the first aspect.
In a fourth aspect, an embodiment of the present application provides a computer readable storage medium having stored thereon a computer program which, when executed by a processor, performs a resource allocation method as provided in the first aspect.
The beneficial effects of the application are as follows:
The application provides a resource allocation method, a device, electronic equipment and a storage medium, comprising the following steps: acquiring resource information of each server in a server cluster and resource demand information of a plurality of resource request objects; determining an initial resource allocation result under each resource dimension of the server according to the resource information of each server and the resource demand information of each resource request object; and determining a target resource allocation result meeting the resource demand information of each resource request object according to the initial resource allocation result under each resource dimension. By forming a server cluster by the old server and the new server, collocation and utilization of the new and old server resources can be realized, and the maximum utilization of the resources can be realized. And by carrying out dimension division on the resources of the server so as to carry out resource fitting under each resource dimension, relatively complete and comprehensive initial resource allocation results under each resource dimension can be obtained respectively. And then, based on the initial resource allocation result under each resource dimension, comprehensively analyzing, and determining a target resource allocation result meeting the resource demand information of each resource request object. On one hand, the fusion use of new and old server resources is realized, and the resource utilization rate is improved; on the other hand, the accuracy of the resource allocation result is improved, and the maximum chemical allocation of the resources is realized.
In addition, through a preset screening strategy, feasibility screening can be performed on the initial resource allocation result in each resource dimension to obtain a to-be-selected resource allocation result in each resource dimension, unreasonable resource allocation results are eliminated in time, and the accuracy of the finally obtained resource allocation result is improved.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the embodiments will be briefly described below, it being understood that the following drawings only illustrate some embodiments of the present application and therefore should not be considered as limiting the scope, and other related drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a schematic diagram of a resource allocation system according to an embodiment of the present application;
FIG. 2 is a schematic diagram of an execution flow of an automatic deployment method according to an embodiment of the present application;
Fig. 3 is a schematic flow chart of a resource allocation method according to an embodiment of the present application;
fig. 4 is a flowchart of another resource allocation method according to an embodiment of the present application;
Fig. 5 is a complete flow chart of a resource allocation method according to an embodiment of the present application;
Fig. 6 is a flowchart of another resource allocation method according to an embodiment of the present application;
fig. 7 is a flowchart of another resource allocation method according to an embodiment of the present application;
Fig. 8 is a flowchart of another resource allocation method according to an embodiment of the present application;
fig. 9 is a flowchart of another resource allocation method according to an embodiment of the present application;
Fig. 10 is a schematic diagram of a resource allocation result according to an embodiment of the present application;
fig. 11 is a flowchart of another resource allocation method according to an embodiment of the present application;
Fig. 12 is a schematic diagram of a resource allocation apparatus according to an embodiment of the present application;
fig. 13 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present application more apparent, the technical solutions of the embodiments of the present application will be clearly and completely described with reference to the accompanying drawings in the embodiments of the present application, and it should be understood that the drawings in the present application are for the purpose of illustration and description only and are not intended to limit the scope of the present application. In addition, it should be understood that the schematic drawings are not drawn to scale. A flowchart, as used in this disclosure, illustrates operations implemented according to some embodiments of the present application. It should be understood that the operations of the flow diagrams may be implemented out of order and that steps without logical context may be performed in reverse order or concurrently. Moreover, one or more other operations may be added to or removed from the flow diagrams by those skilled in the art under the direction of the present disclosure.
In addition, the described embodiments are only some, but not all, embodiments of the application. The components of the embodiments of the present application generally described and illustrated in the figures herein may be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the application, as presented in the figures, is not intended to limit the scope of the application, as claimed, but is merely representative of selected embodiments of the application. All other embodiments, which can be made by a person skilled in the art without making any inventive effort, are intended to be within the scope of the present application.
It should be noted that the term "comprising" will be used in embodiments of the application to indicate the presence of the features stated hereafter, but not to exclude the addition of other features.
In the deployment of distributed cluster systems, it is often necessary to install deployment applications or components onto servers within the cluster to deploy the complete cluster system. In the process of installing the application, how to maximize the effective utilization of the server resources becomes a key point of research.
At present, for deployment of a distributed cluster system, a required server is purchased in advance according to resource demand information of each application to be deployed in the system, so that deployment and installation of the application are realized.
Illustratively, in an actual enterprise software development project, for a privately deployed enterprise application, each time a set of systems is deployed, it is undoubtedly a wasteful and environmentally unfriendly act to re-purchase a new set of hardware. The service life of the server is long, the enterprise is likely to go on-line with a new system, and the old system is likely to go off-line, or some of the last purchased server remains. This undoubtedly results in a great waste of server resources.
In order to solve the problems, the resource allocation method provided by the scheme can properly supplement and purchase a part of new servers on the basis of the old servers, so that a server cluster is formed by the old servers and the new servers, resource fitting between the servers and the applications is performed on the basis of the server cluster formed by the old servers and the new servers, the applications are allocated to the servers in the server cluster in a reasonable mode, reasonable collocation application of the old and new resources is realized, and reasonable utilization of the server resources is realized while automatic deployment is completed.
Fig. 1 is a schematic diagram of a resource allocation system according to an embodiment of the present application; as shown in fig. 1, the resource allocation system of the present application may include: the resource request object installation package comprises a plurality of installation packages corresponding to the resource requests forming the complete distributed cluster system. The resource request object herein may refer to an object for requesting installation of a deployment on a server. The resource request object in this embodiment includes, but is not limited to, an application, a database, a file system, and the like. The server cluster may include a number of servers, which may include both the remaining old servers and the purchased new servers.
Each resource request object corresponds to a resource request object installation package, and the resource request object installation package can be composed of a description file and a configuration file template. The description file is a descriptive text document additionally preset in the installation package and is used for describing the purpose of the resource request object, other resource request objects depending on the purpose, the resource requirement of each node (including minimum and standard resource requirements), the network requirement (intranet, internet access, internet outgoing and the like) and the minimum cluster number of the resource request object. The configuration file template is used for recording information of the server allocated to the resource request object, and after the target resource allocation result is determined based on the method of the application, the template engine in the automatic deployment system can be used for replacing relevant information of the server in the configuration file template of the resource request object according to the identification of the server allocated to the resource request object so as to record the identification of the server allocated to the resource request object.
The automatic deployment system is used as a core part for managing resources of all available servers and resource request object installation packages. Comprising two main modules: a computing engine and a template engine.
The computing engine: fitting with the resources of each server in the server cluster according to the single node resource requirements, the cluster number requirements, the network segments and the dependency relations among all the resource request objects to be deployed. On the premise of keeping the system stable, how to distribute the existing resource request objects to achieve the maximum utilization of the server resources is calculated.
Template engine: and replacing preset characters in a configuration file template in the installation package of each resource request object according to the resource allocation result to form a configuration file which can really support the running of the application system program.
The resource request object installation package is imported into the automatic deployment system, and the automatic deployment system issues each resource request object installation package to a corresponding target server in the server cluster according to the information of the server recorded in the configuration file template in the resource request object installation package; and the automatic deployment system controls the target server to execute installation through a standard protocol so as to install the resource request object distributed to the server on the server.
Fig. 2 is a schematic diagram of an execution flow of an automatic deployment method according to an embodiment of the present application. The execution of the automatic deployment method is based on the resource allocation system implementation in fig. 1 described above. As shown in fig. 2, the process may include the steps of:
Step a, as a precondition of automatic deployment, IT personnel need to enter a server available for deployment into an automatic deployment system. Including but not limited to IP (Internet Protocol, network protocol) of the server, ports, usernames, passwords, etc.
And b, automatically logging in each server one by the automatic deployment system, and then acquiring various information of the current server, such as the number of cores, frequency, memory quantity, current utilization rate, hard disk residual space, network card information, network segment information and the like of a CPU (Central Processing Unit ).
And c, uploading all the resource request object installation packages to be installed to the automatic deployment system by IT personnel.
And d, the automatic deployment system reads the description file in the uploaded installation package of each resource request object, and fits the resource requirement of each resource request object with the resources of each server, network and the like existing in the server cluster to obtain a resource allocation result.
And e, the automatic deployment system distributes each resource request object installation package to a corresponding target server according to a resource allocation result, and in the process of issuing, a template engine is used for replacing server related information in a configuration file template in the resource request object installation package so as to achieve the purpose of no need of manual configuration. After the resource request object installation package is distributed, the installation script is remotely executed to install and start.
The specific implementation process of the step d is described in detail in the following embodiments as the core content of the present solution.
Fig. 3 is a flow chart of a resource allocation method according to an embodiment of the present application. As shown in fig. 3, the method may include:
s101, acquiring resource information of each server in a server cluster and resource demand information of a plurality of resource request objects.
The method supports the collocation and utilization of new and old server resources, and the servers in the server cluster can comprise the old servers remained by the last distributed cluster system and also can comprise new servers purchased corresponding to the current online distributed cluster system.
The resource request object may include, but is not limited to, an application, a database, etc., and the resource request object includes all resource request objects that request to be installed in the server cluster, and in the following embodiment of the method, the resource request object is exemplified as an application.
The resource information of each server can be obtained based on the information of each server in the server cluster uploaded by the IT personnel, and the resource demand information of each resource request object can be obtained based on the information of the installation package of each resource request object uploaded.
S102, determining an initial resource allocation result under each resource dimension of the server according to the resource information of each server and the resource demand information of each resource request object.
Optionally, resource fitting may be performed according to the resource information of each server and the resource requirement information of each resource request object, so as to determine an initial resource allocation result under each resource dimension of the server.
Wherein each resource dimension of the server includes, but is not limited to: processor resources of the server, memory resources of the server, hard disk resources of the server, network resources of the server, and the like.
According to the embodiment, the resources of the server are divided in dimensions, so that the fitting of the resources is performed under each resource dimension, and relatively complete and comprehensive initial resource allocation results under each resource dimension can be obtained.
S103, determining a target resource allocation result meeting the resource demand information of each resource request object according to the initial resource allocation result under each resource dimension, wherein the target resource allocation result is used for indicating the identification of the server to be allocated to each resource request object.
Based on the initial resource allocation results in each resource dimension, the resource allocation results can be screened and integrated to obtain the target resource allocation results meeting the resource demand information of each resource request object.
The initial resource allocation results under each resource dimension are the resource allocation results meeting the resource requirements when considering the single resource dimension, and the deployment of the resource request object does not only consider the requirements under the single resource dimension, but also needs to integrate the requirements under each resource dimension to perform unified planning and deployment.
Then, by comprehensively analyzing the initial resource allocation results in each resource dimension, the target resource allocation result satisfying the resource demand information of each resource request object can be determined therefrom. The target resource allocation result may be a resource allocation result when the resource requirements of each resource request object under each resource dimension are maximally satisfied.
The target resource allocation result may be used to indicate an identification of the server to be allocated to each resource request object. Illustratively, assume that the resource request object includes: object 1, object 2, and object 3, the servers in the server cluster include: server 1, server 2, server 3; then one possible target resource allocation result may be: object 1-server 1, object 2-server 2, object 3-server 3. That is, it indicates that the object 1 is assigned to the server 1, the object 2 is assigned to the server 2, and the object 3 is assigned to the server 3.
In summary, the resource allocation method provided in this embodiment includes: acquiring resource information of each server in a server cluster and resource demand information of a plurality of resource request objects; determining an initial resource allocation result under each resource dimension of the server according to the resource information of each server and the resource demand information of each resource request object; and determining a target resource allocation result meeting the resource demand information of each resource request object according to the initial resource allocation result under each resource dimension. By forming a server cluster by the old server and the new server, collocation and utilization of the new and old server resources can be realized, and the maximum utilization of the resources can be realized. And by carrying out dimension division on the resources of the server so as to carry out resource fitting under each resource dimension, relatively complete and comprehensive initial resource allocation results under each resource dimension can be obtained respectively. And then, based on the initial resource allocation result under each resource dimension, comprehensively analyzing, and determining a target resource allocation result meeting the resource demand information of each resource request object. On one hand, the fusion use of new and old server resources is realized, and the resource utilization rate is improved; on the other hand, the accuracy of the resource allocation result is improved, and the maximum chemical allocation of the resources is realized.
Fig. 4 is a flowchart of another resource allocation method according to an embodiment of the present application, optionally, in step S103, determining, according to an initial resource allocation result in each resource dimension, a target resource allocation result satisfying resource requirement information of each resource request object may include:
s201, carrying out feasibility screening on the initial resource allocation result in each resource dimension to obtain a candidate resource allocation result in each resource dimension.
In general, a lot of unreasonable allocation results exist in the obtained initial resource allocation results in each resource dimension, and then feasibility screening can be performed on the initial resource allocation results in each resource dimension to screen out unreasonable allocation results therein, so as to obtain the resource allocation results to be selected in each resource dimension.
S202, fusion processing is carried out on the resource allocation results to be selected under each resource dimension, and a target resource allocation result is obtained.
Since the resource allocation results to be selected in each resource dimension are the resource allocation results when the resource requirements in a single resource dimension are met, and the stable operation of the resource request object can be realized when the resources in all aspects meet the conditions, if one resource allocation result only meets the resource requirements in the first resource dimension but does not meet the resource requirements in other resource dimensions, the stable operation of the resource request object can not be supported obviously.
Based on the above, the fusion processing can be performed on the resource allocation results to be selected in each resource dimension to obtain the target resource allocation result, so as to screen out the maximized resource allocation result meeting the resource requirement in each resource dimension.
Optionally, in step S201, performing feasibility screening on the initial resource allocation result in each resource dimension to obtain a candidate resource allocation result in each resource dimension, which may include: and adopting a preset screening strategy to respectively carry out feasibility screening on the initial resource allocation result in each resource dimension to obtain the resource allocation result to be selected in each resource dimension.
In one embodiment, a preset screening policy may be adopted to perform feasibility screening on the initial resource allocation result in each resource dimension, so as to obtain a candidate resource allocation result in each resource dimension.
Alternatively, the preset screening strategy may be recorded in a strategy library in the form of text, and the predicted screening strategy may be expressed in the form of a formula.
The policy library contains a large number of screening policies, which may be based on experience summarized by long-term production operation and maintenance accidents, and these experiences may be converted into text or mathematical formulas and implemented by programs for screening the initial resource allocation results. Of course, the screening strategies in the strategy library can also be updated in real time to ensure the comprehensiveness and accuracy of screening.
Alternatively, the preset screening strategies may be broadly classified into the following categories: real-time and non-real-time separation principle, memory redundancy principle, CPU peak staggering principle, hard disk redundancy principle, risk isolation principle and network peak staggering principle.
Wherein the real-time and non-real-time separation principles may include: the first type of resource request object and the second type of resource request object are not allowed to be distributed to the same server; the memory redundancy principle may include: the sum of standard memory requirements or average memory requirements of all resource request objects distributed to the same server is smaller than the preset proportion of the memory of the server; the CPU off-peak principle may include: the peak time of task processing of each resource request object distributed to the same server is not coincident; hard disk redundancy principles may include: the sum of the data increment of each resource request object distributed to the same server is smaller than or equal to the hard disk resource of the server; risk isolation principles may include: the same resource request object is not allowed to be distributed to the same server; the network peak staggering principle may include: the network peak periods of the resource request objects allocated to the same server do not coincide.
It should be noted that, for the real-time and non-real-time separation principle, when the first type of resource request object is a resource request object for processing real-time service (such as account opening service, transaction service, query service, etc.), the second type of resource request object may be a resource request object for processing background batch service (such as transaction clearing, fund clearing, payment settlement, etc.). That is, the application program mainly processing the real-time service and the application program processing the background batch service are not distributed (deployed) to the same server as much as possible.
For the memory redundancy principle, if the Java application is used, the minimum memory requirement of the resource request objects distributed to the same server must be less than 70% of the system memory (JVM (Java Virtual Machine, java virtual machine) consumes additional physical memory, and creating threads can cause the Linux system to distribute additional memory). This principle is to avoid exiting due to an out of memory (OOM) error or forced killing by the operating system due to insufficient system memory.
For the CPU peak staggering principle, the time point of each pressure peak of each day is different due to the fact that each resource request object is different in service self characteristics. For example, transaction clearance is peak pressure at 3-4 pm each day, which requires at least 8CPU cores to support its operations. But this point in time has passed its CPU occupation is substantially 0. And fund clearing, the peak pressure of the fund clearing occurs at about 5 points, 4 CPU cores are needed to support operation in the peak, and the CPU occupies 1-2 cores in the off-peak. The CPU is calculated mainly according to the average distribution, but if the peak periods of two resource request objects coincide, the CPU needs to be judged again according to the maximum CPU requirement.
For the hard disk redundancy principle, the hard disk needs to be larger than the sum of the daily increment of each resource request object multiplied by the storage life in addition to meeting the installation requirement of each resource request object. The source of the data increment of each resource request object is different, and in the algorithm, the data increment is abstracted into a day increment and a storage life, and the data increment is not distinguished why the data increment is increased, and the possible day increment of the resource request object is 0. For example, a resource request object for generating a data file needs to be stored for a long period of time, but generally an archive period is set, and the original file can be deleted after archiving, so the storage period is the archive period. If logging is desired, different types of system logs may have shelf lives between half a year and two years, depending on regulatory requirements.
For risk isolation principles, different running instances (processes) of the same resource request object may not be deployed on the same server. This is to avoid that when the server is down, all nodes of the resource request object are not available, so that the availability of the whole system is affected. This does not meet the high availability requirements.
For the network peak staggering principle, similar to CPU peak staggering, different resource request objects in a distributed system use different bandwidth demands of a network, and the time periods of peak flow occurrence are different, when the network demands are calculated, calculation can be performed according to an average value for calculation convenience, but if the network peak time periods of two resource request objects in combination are overlapped, the network peak time periods need to be judged again according to the maximum network demands.
In one implementation manner, when the feasibility screening is performed on the initial resource allocation result under each resource dimension based on the screening policies, each screening policy needs to be satisfied at the same time, that is, the selected resource allocation result under each resource dimension obtained by screening is the resource allocation result when each listed screening policy is satisfied at the same time.
Optionally, in step S202, the fusion processing is performed on the resource allocation results to be selected in each resource dimension to obtain a target resource allocation result, which may include: and carrying out intersection processing on the to-be-selected resource allocation results in each resource dimension to obtain a target resource allocation result.
In some embodiments, based on the candidate resource allocation results in each resource dimension, the candidate resource allocation results in each resource dimension may be intersected, so as to obtain a target resource allocation result when all the resource dimensions are satisfied at the same time.
Based on the obtained target resource allocation result, the actual unconditional result can be further screened from the target resource allocation result by a one-ticket overrule mode. Here, it may be that the operation and maintenance personnel specify certain resource request objects or unique deployment rules of services according to their own experience or requirements of product operation. The system will use a "one ticket overrule" approach to screening based on these manually specified special rules.
If the calculation of all the steps is completed, the available target resource allocation result is finally obtained, step e in fig. 2 may be performed based on the available target resource allocation result, so as to complete the deployment of the resource request object in the server cluster. If the available target resource allocation results include a plurality, then the selection may be made by the operation and maintenance personnel. And if the target resource allocation result of the condition is not completely met, sequentially displaying the target resource allocation result which maximally meets the condition according to the priority for selection.
Fig. 5 is a complete flow chart of a resource allocation method according to an embodiment of the present application. As shown in FIG. 5, steps S301-1 to S301-4 are respectively a process of determining an initial resource allocation result under each resource dimension of the server. S302-1 to S302-4 are respectively used for carrying out feasibility screening on initial resource allocation results under each resource dimension based on a preset screening strategy. S303-1 to S303-4 respectively obtaining the resource allocation results to be selected in each resource dimension. S304, the fusion processing is carried out on the resource allocation results to be selected under each resource dimension, and a process of obtaining a target resource allocation result is obtained. S305 is a process of excluding unreasonable results in the target resource allocation result by a ticket overrule. S306, outputting the final available target resource allocation result.
Fig. 6 is a flowchart of another resource allocation method according to an embodiment of the present application. Optionally, in step S102, determining an initial resource allocation result under each resource dimension of the server according to the resource information of each server and the resource requirement information of each resource request object may include:
S401, determining a plurality of resource allocation sequences, wherein each resource allocation sequence is used for indicating one resource allocation calculation sequence of each resource request object.
Optionally, the process of determining the initial resource allocation result under each resource dimension of the server may be implemented by a permutation and combination algorithm according to the resource information of each server and the resource requirement information of each resource request object, so as to determine as possible all possible resource allocation combination modes meeting the resource allocation requirement under each resource dimension, and obtain a comprehensive initial resource allocation result under each resource dimension.
The permutation and combination algorithm can be realized based on the sequence of each resource request object and each server when participating in the resource allocation calculation, and when the resource allocation calculation sequence of the resource request object and the server is changed, different resource allocation results can be obtained.
Alternatively, a plurality of resource allocation sequences may be determined first, each of the resource allocation sequences indicating one resource allocation calculation order for each of the resource request objects.
Illustratively, one possible resource allocation sequence includes { resource request object 1-resource request object 2-resource request object 3; server 1-server 2-server 3}; it can be characterized that, under such a resource allocation sequence, the resource request object 1 firstly performs calculation with the server 1, if the resource request object 1 can be allocated to the server 1, an allocation result between the resource request object 1 and the server 1 is obtained, and the calculation is continuously performed by the resource request object 2 and the server 1 until the resource of the server 1 is insufficient, and the replacement server 2 participates in the calculation; if the resource request object 1 cannot be allocated to the server 1, the resource request object 1 and the server 2 perform calculation until the allocation of all the resource request objects is completed.
That is, all the resource allocation combinations can be obtained by changing the calculation order of the resource request object or the server.
S402, determining an initial resource allocation result in the current resource dimension of the server according to the resource information in the current resource dimension of each server, the resource sub-demand information of each resource request object in the current resource dimension and a plurality of resource allocation sequences.
Taking the calculation of the initial resource allocation result in a resource dimension as an example, the initial resource allocation result in the current resource dimension of the server can be determined according to the resource information in the current resource dimension of each server, the resource sub-requirement information of each resource request object in the current resource dimension, and the plurality of resource allocation sequences.
The current resource dimension may be any one of all possible resource dimensions. The plurality of resource allocation sequences may substantially encompass all possible computational orders between the resource request object and the server.
Fig. 7 is a flowchart of another resource allocation method according to an embodiment of the present application; optionally, in step S402, determining an initial resource allocation result in the current resource dimension of the server according to the resource information in the current resource dimension of each server, the resource sub-requirement information in the current resource dimension of each resource request object, and the plurality of resource allocation sequences may include:
s501, determining at least one intermediate resource allocation result under each resource allocation sequence according to the resource parameter of each server under the current resource dimension, the resource sub-demand information of each resource request object under the current resource dimension and the resource allocation calculation sequence indicated by each resource allocation sequence.
Optionally, according to the resource parameter of each server under the current resource dimension and the resource sub-requirement information of each resource request object under the current resource dimension, according to a resource allocation calculation sequence indicated by a resource allocation sequence, at least one intermediate resource allocation result under the resource allocation sequence can be calculated; so that at least one intermediate resource allocation result under each resource allocation sequence can be obtained under each resource allocation sequence.
S502, fusing at least one intermediate resource allocation result in each resource allocation sequence to obtain an initial resource allocation result in the current resource dimension.
And the obtained results are at least one intermediate resource allocation result in each resource allocation sequence in the current resource dimension, and the initial resource allocation result in the current resource dimension can be obtained by carrying out fusion processing on the intermediate resource allocation results.
The fusion process herein may refer to a union, that is, merging all intermediate resource allocation results in each resource allocation sequence in the current resource dimension to obtain an initial resource allocation result in the current resource dimension.
Fig. 8 is a flowchart of another resource allocation method according to an embodiment of the present application; optionally, in step S501, determining at least one intermediate resource allocation result under each resource allocation sequence according to the resource parameter under the current resource dimension of each server, the resource sub-requirement information of each resource request object under the current resource dimension, and the resource allocation calculation sequence indicated by each resource allocation sequence may include:
S601, determining whether the current to-be-calculated server allows allocation of the current to-be-calculated resource request object according to the resource sub-demand information of the current to-be-calculated resource request object in the current resource dimension and the resource parameter of the current to-be-calculated server in the current resource dimension.
The current resource request object to be calculated and the current server to be calculated may be determined according to the current resource allocation sequence.
This embodiment is exemplified by computational logic between a single resource request object and a single server. Whether the current to-be-calculated server allows allocation of the current to-be-calculated resource request object can be determined according to the resource sub-demand information of the current to-be-calculated resource request object in the current resource dimension and the resource parameter of the current to-be-calculated server in the current resource dimension.
The resource sub-requirement information of the current resource request object to be calculated under the current resource dimension can be read from the description file in the installation package of the current resource request object to be calculated. The resource parameters of the current resource dimension of the current server to be calculated can be read after logging in the current server to be calculated.
S602, if a server to be calculated before allows allocation of a resource request object to be calculated currently, generating a resource allocation result between the resource request object to be calculated currently and the server to be calculated currently under the current resource dimension; and updating the resource parameters of the current to-be-calculated server in the current resource dimension according to the resource sub-demand information of the current to-be-calculated resource request object in the current resource dimension, determining whether the current to-be-calculated server allows the next to-be-calculated resource request object to be allocated or not based on the updated resource parameters of the current to-be-calculated server in the current resource dimension until the resource parameters of the current to-be-calculated server in the current to-be-calculated server are smaller than a preset threshold, calculating the next to-be-calculated server as the current new to-be-calculated server, and performing circularly until each resource request object finishes resource allocation according to the resource allocation calculation sequence indicated by the current resource allocation sequence, and obtaining at least one intermediate resource allocation result under the current resource allocation sequence.
If it is determined that the server to be calculated allows allocation of the current resource request object to be calculated according to the calculation result, a resource allocation result between the server to be calculated and the current resource request object to be calculated may be generated.
And then, updating the resource parameters of the server to be calculated at present in the current resource dimension according to the resource sub-demand information of the resource request object to be calculated at present in the current resource dimension so as to ensure the accuracy of the next calculation result. That is, the resource sub-requirement information of the current resource request object to be calculated in the current resource dimension is subtracted from the resource parameter of the current resource dimension of the current server to be calculated, so as to obtain the remaining resource parameter of the current resource dimension of the current server to be calculated.
And then, determining whether the current server to be calculated allows the next resource request object to be allocated according to the resource sub-demand information of the next resource request object to be calculated in the current resource dimension and the updated resource parameter of the current server to be calculated in the current resource dimension until the resource parameter of the current server to be calculated in the current resource dimension is smaller than a preset threshold value, stopping the resource allocation calculation of the current server to be calculated and replacing the next server to be calculated to participate in the calculation, and circularly executing the steps until each resource request object finishes the resource allocation according to the resource allocation calculation sequence indicated by the current resource allocation sequence and at least one intermediate resource allocation result under the current resource allocation sequence is obtained.
The server to be calculated corresponding to the server to be calculated currently and the resource request object to be calculated corresponding to the resource request object to be calculated currently are determined according to the current resource allocation sequence.
In some alternative embodiments, the resource dimensions of the server include, but are not limited to: processor resources of the server, memory resources of the server, hard disk resources of the server, and network resources of the server.
Processor resources of the server include, but are not limited to: the frequency of the processor, the number of cores of the processor, the model (type) of the processor. The memory resources of the server include: memory size. The hard disk resources of the server include: hard disk space size. Network resources of the server include, but are not limited to: network speed, network segment, etc.
Optionally, the current resource dimension is a memory resource of the server; in step S601, determining whether the current to-be-calculated server allows allocation of the current to-be-calculated resource request object according to the resource sub-requirement information of the current to-be-calculated resource request object in the current resource dimension and the resource parameter of the current to-be-calculated server in the current resource dimension may include: and determining whether the current to-be-calculated server allows the allocation of the current to-be-calculated resource request object according to the memory use requirement of the current to-be-calculated resource request object, the memory size of the current to-be-calculated server and the preset memory reserved size of the server.
Taking the current resource dimension as the memory resource of the server as an example, the memory use requirement size of the current resource request object to be calculated and the memory size of the current server to be calculated can be obtained, and whether the current server to be calculated allows the current resource request object to be allocated or not is judged under the condition that the reserved server memory with a certain allowance is considered.
Fig. 9 is a flowchart of another resource allocation method according to an embodiment of the present application; optionally, in the foregoing step, determining whether the current to-be-calculated server allows allocation of the current to-be-calculated resource request object according to a memory usage requirement of the current to-be-calculated resource request object, a memory size of the current to-be-calculated server, and a preset server memory reservation size may include:
S701, determining the available memory size of the current server to be calculated according to the memory size of the current server to be calculated and the preset reserved memory size of the server.
Optionally, the memory size of the server to be calculated currently may be subtracted from the preset reserved memory size of the server to obtain the available memory size of the server to be calculated currently. The purpose of memory reservation is to avoid that when other deployment demands exist, the server cannot provide resources, and downtime is caused.
The resource reservation size of the server is different according to the difference of the distributed cluster system to be deployed and the difference of the current resource dimension, and the data can be preconfigured and directly read for use in calculation.
S702, if the memory usage requirement of the current resource request object to be calculated is smaller than or equal to the available memory size of the current server to be calculated, determining that the current server to be calculated allows the current resource request object to be allocated.
When the memory use requirement of the current to-be-calculated resource request object is less than or equal to the available memory size of the current to-be-calculated server, it can be determined that the current to-be-calculated server allows the current to-be-calculated resource request object to be allocated.
Fig. 10 is a schematic diagram of a resource allocation result according to an embodiment of the present application. Taking a server with 10GB of memory as an example, assume that the resource request objects include 5, where the memory usage requirement of the resource request object a is 2.5GB, the memory usage requirement of the resource request object B is 1.5GB, the memory usage requirement of the resource request object C is 3GB, the memory usage requirement of the resource request object D is 3.5GB, and the memory usage requirement of the resource request object E is 1GB.
As can be seen from fig. 10, ABC may be allocated to the same server or ADE may be allocated to the same server from among the 5 resource request objects, considering only that the memory satisfies the requirement. In a practical scenario, the number of servers and the number of resource request objects are far greater than those of the above examples, so that there are many possibilities of practical permutation and combination. Thus, the initial resource allocation result under the memory resource of the server can be obtained.
The initial resource allocation result may include a plurality of entries, each entry representing one possible resource allocation result. Each item may be represented by a two-dimensional array, the x-coordinate of which may represent the number of servers, and the y-coordinate of which may represent the number of resource request objects, a two-dimensional array may be represented by the following array.
That is, in one possible resource allocation result, the server 1 may allocate the resource request object 1, the resource request object 2, and the resource request object 3; a resource request object 1, a resource request object 2 and a resource request object 3 can be allocated on the server 2; the server 3 can be allocated with a resource request object 1, a resource request object 4 and a resource request object 5; the server 4 may be allocated a resource request object 1, a resource request object 4, and a resource request object 5.
Where the same resource request object may appear on different servers, depending on the number of deployments the resource request object itself requires to fulfill the high availability requirement. The final output of the permutation and combination is more likely than if each resource request object only occurred once.
Thus, the initial resource allocation result under the memory resource of the server can be calculated.
Optionally, the current resource dimension is a processor resource of the server; in step S601, determining whether the current to-be-calculated server allows allocation of the current to-be-calculated resource request object according to the resource sub-requirement information of the current to-be-calculated resource request object in the current resource dimension and the resource parameter of the current to-be-calculated server in the current resource dimension may include:
And determining whether the current to-be-calculated server allows the allocation of the current to-be-calculated resource request object according to the model of the processor supported by the current to-be-calculated resource request object, the average core number of the processor required by the current to-be-calculated resource request object, the minimum processor frequency required by the current to-be-calculated resource request object, the model of the processor of the current to-be-calculated server, the processor frequency of the current to-be-calculated server and the core number of the processor of the current to-be-calculated server.
Taking the current resource dimension as the processor resource of the server as an example, the embodiment can obtain the model number of the processor supported by the current resource request object to be calculated, the average core number of the processor required by the current resource request object to be calculated, and the minimum processor frequency required by the current resource request object to be calculated. Meanwhile, the model of a processor of the server to be calculated currently, the processor frequency of the server to be calculated currently and the number of processor cores of the server to be calculated currently are obtained, and whether the server to be calculated currently allows allocation of the resource request object to be calculated currently is calculated.
Optionally, determining whether the current to-be-calculated server allows allocation of the current to-be-calculated resource request object according to a model of a processor supported by the current to-be-calculated resource request object, an average core number of processors required by the current to-be-calculated resource request object, a minimum processor frequency required by the current to-be-calculated resource request object, and a model of a processor of the current to-be-calculated server, a processor frequency of the current to-be-calculated server, and a processor core number of the current to-be-calculated server may include:
Determining whether the current to-be-calculated server allows allocation of the current to-be-calculated resource request object according to the model of a processor supported by the current to-be-calculated resource request object, the average core number of the processor required by the current to-be-calculated resource request object, the minimum processor frequency required by the current to-be-calculated resource request object, the model of the processor of the current to-be-calculated server, the converted processor frequency of the current to-be-calculated server and the processor core number of the current to-be-calculated server; or determining whether the current to-be-calculated server allows the allocation of the current to-be-calculated resource request object according to the model of the processor supported by the current to-be-calculated resource request object, the average core number of the processor required by the current to-be-calculated resource request object, the minimum processor frequency required by the current to-be-calculated resource request object, the model of the processor of the current to-be-calculated server, the processor frequency of the current to-be-calculated server and the converted processor core number of the current to-be-calculated server.
When the resource dimension is the processor resource of the server, the calculation is relatively complex compared with the memory resource of the server, and the processor resource of the server to be considered includes, but is not limited to, the model number of the processor, the core number of the processor and the frequency of the processor.
Since the server cluster includes both old servers and new servers, an additional factor, i.e., equivalent performance of the CPU, needs to be considered in the context of collocation of the old and new servers. Because of the new and old collocation scenes, the CPU single-core performances of different servers are different, so that the calculation of the core number cannot represent the exact multi-core performance proportion. Therefore, a reference value can be set to perform single core performance conversion according to the CPU model of the server. Which may include scaling of CPU frequency and scaling of CPU core number.
In the above-mentioned method, when determining whether the current to-be-calculated server allows the allocation of the current to-be-calculated resource request object under the processor resource of the server, there may be two realizable modes, one is to determine whether the current to-be-calculated server allows the allocation of the current to-be-calculated resource request object according to the model of the processor supported by the current to-be-calculated resource request object, the average core number of the processors required by the current to-be-calculated resource request object, the minimum processor frequency required by the current to-be-calculated resource request object, and the model of the processor of the current to-be-calculated server, the converted processor frequency of the current to-be-calculated server, and the processor core number of the current to-be-calculated server. Namely, the CPU frequency of the server adopts the equivalent converted frequency. And determining whether the current to-be-calculated server allows the allocation of the current to-be-calculated resource request object according to the model of the processor supported by the current to-be-calculated resource request object, the average core number of the processor required by the current to-be-calculated resource request object, the minimum processor frequency required by the current to-be-calculated resource request object, the model of the processor of the current to-be-calculated server, the processor frequency of the current to-be-calculated server and the converted processor core number of the current to-be-calculated server. Wherein, the CPU core number of the server adopts the equivalent converted core number.
Fig. 11 is a flowchart of another resource allocation method according to an embodiment of the present application; optionally, the scaled processor frequency of the current server to be calculated and the scaled processor core number of the current server to be calculated are calculated in the following manner:
s801, according to the model of the processor of the server to be calculated currently, inquiring to obtain the performance proportion parameter of the processor of the server to be calculated currently relative to the reference model processor.
The reference model processor may be preset, for example, 100% may be set based on the single core performance of a certain model of the commonly used intel-to-goldfinger processors, and then the single core performance of other processors of different models may be calculated to obtain performance proportion parameters according to the proportion of the processor of different models higher or lower than the reference model processor. For example, after conversion, the single core performance of a CPU of a certain domestic model may be set to 60% and the single core performance of a CPU of a certain model to a strong platinum processor may be set to 120% with respect to a reference model processor.
In one implementation, the performance scaling parameters of the processor of the server relative to the reference model processor may be implemented by way of a look-up table. The CPU equivalent performance configuration table may be generated in advance, and performance proportion parameters of CPUs of different models with respect to the reference model CPU may be recorded in the CPU equivalent performance configuration table. The table can be maintained once at preset intervals, and newly released CPU model and performance proportion parameters are supplemented.
Then, in the application process, the performance proportion parameter corresponding to the model of the processor of the server to be calculated currently can be queried from the CPU equivalent performance configuration table according to the model of the processor of the server to be calculated currently.
S802, equivalent conversion is carried out on the processor frequency of the server to be calculated according to the performance proportion parameters of the processor of the server to be calculated, and converted processor frequency of the server to be calculated is obtained.
And obtaining the converted processor frequency of the server to be calculated based on the performance proportion parameter of the processor of the server to be calculated, the processor frequency of the server to be calculated and the frequency of the reference model processor.
Exemplary: assuming that the performance proportion parameter of the processor of the server to be calculated is 60%, and the processor frequency of the server to be calculated is 5GHz, the formula may be adopted: 60 percent ofAnd 5, calculating to obtain the converted processor frequency of the server to be calculated currently.
S803, equivalent conversion is carried out on the processor core number of the server to be calculated according to the performance proportion parameters of the processor of the server to be calculated, and the converted processor core number of the server to be calculated is obtained.
Similarly, the equivalent conversion can be performed on the processor core number of the current server to be calculated according to the performance proportion parameter of the processor of the current server to be calculated and the processor core number of the current server to be calculated, so as to obtain the converted processor core number of the current server to be calculated.
Optionally, in the above step, determining whether the current to-be-calculated server allows allocation of the current to-be-calculated resource request object according to the model of the processor supported by the current to-be-calculated resource request object, the average number of cores of the processor required by the current to-be-calculated resource request object, the minimum processor frequency required by the current to-be-calculated resource request object, the model of the processor of the current to-be-calculated server, the converted processor frequency of the current to-be-calculated server, and the number of cores of the processor of the current to-be-calculated server may include: if the model of the processor of the server to be calculated currently belongs to the model of the processor supported by the resource request to be calculated currently; the number of processor cores of the server to be calculated currently is larger than or equal to the average number of processor cores required by the resource request to be calculated currently; and the converted processor frequency of the current to-be-calculated server is greater than or equal to the minimum processor frequency required by the current to-be-calculated resource request, determining that the current to-be-calculated server allows the current to-be-calculated resource request to be allocated.
The above judgment manner can be abstracted into the following formula: (t)hardwareConfig.supportCpuType)&&(n>= hardwareConfig.avgCpuCoreNum)&&(F(f)>= hardwareConfig. minCpuFrequency); Wherein t represents the CPU model of the server to be calculated currently; hardwireconfig.suppltctutype represents the model number of the CPU supported by the resource request object to be currently calculated; n represents the CPU core number of the server to be calculated currently; hardwareConfig.avgCpuCoreNum represents the average number of CPU cores required by the resource request object to be currently calculated; f represents the CPU frequency of the server currently to be calculated, and hardtoweconfig. MinCpuFrequency represents the minimum CPU frequency required by the resource request object currently to be calculated. And F (F) represents the scaled CPU frequency of the server currently to be calculated.
Then, when the output result of the above formula is true, it indicates that the server to be currently calculated allows the allocation of the resource request object to be currently calculated.
That is, the model of the processor of the server currently to be calculated belongs to the model of the processor supported by the resource request currently to be calculated; the number of processor cores of the server to be calculated currently is larger than or equal to the average number of processor cores required by the resource request to be calculated currently; and the converted processor frequency of the current to-be-calculated server is greater than or equal to the minimum processor frequency required by the current to-be-calculated resource request, determining that the current to-be-calculated server allows the current to-be-calculated resource request to be allocated.
Or in another implementation, may also be based on the formula (thardwareConfig.supportCpuType)&&(F(n)>= hardwareConfig.avgCpuCoreNum)&&(f>= hardwareConfig. minCpuFrequency), And judging, if the judgment result is true, indicating that the server to be calculated currently allows the allocation of the resource request object to be calculated currently.
It should be noted that, if a resource request object is already allocated on a server, in the subsequent calculation, the number of CPU cores of the server needs to be subtracted from the number of CPU cores required by the already allocated resource request object, that is, n1=n-hardwareconfig.avgcpu core num; n in the above formula may be changed to n1 and then it is calculated whether the next resource request object is allowed to be allocated to the server, and this process is repeated until all the resource request objects complete the calculation.
Similar to the calculation process of the memory resource of the server and the processor resource of the server, when the resource dimension is the hard disk resource of the server, the initial resource allocation result under the hard disk resource can be calculated according to the hard disk requirement, and the factors to be considered in calculation include but are not limited to deployment size, daily increment, temporary file, preservation time and the like; when the resource dimension is the network resource of the server, the factors to be considered include the network speed, the network segment, and the like, and the initial resource allocation result under the network resource can be calculated according to the network resource requirements of the network speed, the network segment, and the like.
Illustratively, the embodiment includes, with each resource dimension of the server: examples of memory resources of the server, processor resources of the server, hard disk resources of the server, and network resources of the server. The initial resource allocation result under the memory resource of the server is assumed to be Y (memory), the initial resource allocation result under the processor resource of the server is assumed to be Y (CPU), the initial resource allocation result under the hard disk resource of the server is assumed to be Y (hard disk), and the initial resource allocation result under the network resource of the server is assumed to be Y (network), for a total of 4 sets.
The feasibility screening can be performed on the initial resource allocation results under each resource dimension based on the preset screening policy, so that the candidate resource allocation result under the memory resource of the server is X (memory), the candidate resource allocation result under the processor resource of the server is X (CPU), the candidate resource allocation result under the hard disk resource of the server is X (hard disk), and the candidate resource allocation result under the network resource of the server is X (network).
The resource allocation results to be selected actually represent possible resource allocation results which can meet the deployment requirement of the distributed cluster system under the condition that only one resource dimension is considered. Then, the 4 candidate resource allocation results may be intersected, and the target resource allocation result may be obtained by X (memory) Σx (CPU) Σx (hard disk) Σx (network). Only the resource allocation results that simultaneously satisfy the resource allocation requirements in the 4 resource dimensions will be reserved as target resource allocation results.
Next, specific deployment rules of certain resource request objects can be specified based on experience of operation staff or requirements of product operation, and resource allocation results which are not in fact qualified are screened from target resource allocation results in a one-ticket overrule mode.
If the calculation of the steps is completed, the available target resource allocation result is finally obtained, and the available target resource allocation result is finally used for executing the deployment of the distributed cluster system. And if a plurality of available target resource allocation results are available, displaying and allowing operation and maintenance personnel to select. And if the available target resource allocation result completely meeting the condition is not available, displaying the available target resource allocation result which does not meet the minimum item for the operation and maintenance personnel to reference, and determining whether to select by the operation and maintenance personnel.
In summary, the resource allocation method provided in this embodiment includes: acquiring resource information of each server in a server cluster and resource demand information of a plurality of resource request objects; determining an initial resource allocation result under each resource dimension of the server according to the resource information of each server and the resource demand information of each resource request object; and determining a target resource allocation result meeting the resource demand information of each resource request object according to the initial resource allocation result under each resource dimension. By forming a server cluster by the old server and the new server, collocation and utilization of the new and old server resources can be realized, and the maximum utilization of the resources can be realized. And by carrying out dimension division on the resources of the server so as to carry out resource fitting under each resource dimension, relatively complete and comprehensive initial resource allocation results under each resource dimension can be obtained respectively. And then, based on the initial resource allocation result under each resource dimension, comprehensively analyzing, and determining a target resource allocation result meeting the resource demand information of each resource request object. On one hand, the fusion use of new and old server resources is realized, and the resource utilization rate is improved; on the other hand, the accuracy of the resource allocation result is improved, and the maximum chemical allocation of the resources is realized.
In addition, through a preset screening strategy, feasibility screening can be performed on the initial resource allocation result in each resource dimension to obtain a to-be-selected resource allocation result in each resource dimension, unreasonable resource allocation results are eliminated in time, and the accuracy of the finally obtained resource allocation result is improved.
The following describes a device, equipment, storage medium, etc. for executing the resource allocation method provided by the present application, and specific implementation processes and technical effects thereof are referred to above, and are not described in detail below.
Fig. 12 is a schematic diagram of a resource allocation device according to an embodiment of the present application, where functions implemented by the resource allocation device correspond to steps executed by the above method. The apparatus may be understood as an automatic deployment system in the above-described resource allocation system, and as shown in fig. 12, the apparatus may include: an acquisition module 110 and a determination module 120;
An obtaining module 110, configured to obtain resource information of each server in the server cluster and resource requirement information of a plurality of resource request objects;
A determining module 120, configured to determine an initial resource allocation result under each resource dimension of the server according to the resource information of each server and the resource requirement information of each resource request object;
The determining module 120 is configured to determine, according to the initial resource allocation result in each resource dimension, a target resource allocation result that satisfies the resource requirement information of each resource request object, where the target resource allocation result is used to indicate an identifier of a server to be allocated to each resource request object.
Optionally, the determining module 120 is specifically configured to perform feasibility screening on the initial resource allocation result in each resource dimension, so as to obtain a candidate resource allocation result in each resource dimension;
and carrying out fusion processing on the to-be-selected resource allocation results in each resource dimension to obtain a target resource allocation result.
Optionally, the determining module 120 is specifically configured to perform feasibility screening on the initial resource allocation result in each resource dimension by adopting a preset screening policy, so as to obtain a candidate resource allocation result in each resource dimension.
Optionally, the preset screening policy at least includes: the sum of standard memory requirements or average memory requirements of the resource request objects of the first type and the resource request objects of the second type which are not allowed to be distributed on the same server are smaller than the preset proportion of the memory of the server, the peak time of task processing of the resource request objects distributed on the same server is not coincident, the sum of data increment of the resource request objects distributed on the same server is smaller than or equal to the hard disk resource of the server, the same resource request object is not allowed to be distributed on the same server, and the peak time of network of the resource request objects distributed on the same server is not coincident.
Optionally, the determining module 120 is specifically configured to perform intersection processing on the candidate resource allocation result in each resource dimension, so as to obtain a target resource allocation result.
Optionally, the determining module 120 is specifically configured to determine a plurality of resource allocation sequences, where each resource allocation sequence is used to indicate a resource allocation calculation sequence of each resource request object;
and determining an initial resource allocation result in the current resource dimension of the server according to the resource information of each server in the current resource dimension, the resource sub-demand information of each resource request object in the current resource dimension and the plurality of resource allocation sequences.
Optionally, the determining module 120 is specifically configured to determine at least one intermediate resource allocation result under each resource allocation sequence according to the resource parameter under the current resource dimension of each server, the resource sub-requirement information of each resource request object under the current resource dimension, and the resource allocation calculation sequence indicated by each resource allocation sequence;
and carrying out fusion processing on at least one intermediate resource allocation result under each resource allocation sequence to obtain an initial resource allocation result under the current resource dimension.
Optionally, the determining module 120 is specifically configured to determine whether the current server to be calculated allows allocation of the current resource request object to be calculated according to the resource sub-requirement information of the current resource request object to be calculated in the current resource dimension and the resource parameter of the current server to be calculated in the current resource dimension;
If the server to be calculated before allows the allocation of the current resource request object to be calculated, generating a resource allocation result between the current resource request object to be calculated and the current server to be calculated under the current resource dimension; updating the resource parameters of the current to-be-calculated server in the current resource dimension according to the resource sub-demand information of the current to-be-calculated resource request object in the current resource dimension, determining whether the current to-be-calculated server allows the next to-be-calculated resource request object to be allocated or not based on the updated resource parameters of the current to-be-calculated server in the current resource dimension until the resource parameters of the current to-be-calculated server in the current to-be-calculated server are smaller than a preset threshold, calculating the next to-be-calculated server as the current new to-be-calculated server, and performing circularly until each resource request object finishes resource allocation according to the resource allocation calculation sequence indicated by the current resource allocation sequence, and obtaining at least one intermediate resource allocation result under the current resource allocation sequence; the server to be calculated corresponding to the server to be calculated currently and the resource request object to be calculated corresponding to the resource request object to be calculated currently are determined according to the current resource allocation sequence.
Optionally, the current resource dimension is a memory resource of the server; the determining module 120 is specifically configured to determine whether the current server to be calculated allows allocation of the current resource request object to be calculated according to the memory usage requirement of the current resource request object to be calculated, the memory size of the current server to be calculated, and the preset memory reservation size of the server.
Optionally, the determining module 120 is specifically configured to determine, according to a memory size of a server to be currently calculated and a preset reserved size of a server memory, an available memory size of the server to be currently calculated;
and if the memory use requirement of the current to-be-calculated resource request object is smaller than or equal to the available memory size of the current to-be-calculated server, determining that the current to-be-calculated server allows the current to-be-calculated resource request object to be allocated.
Optionally, the current resource dimension is a processor resource of the server; the determining module 120 is specifically configured to determine whether the current to-be-calculated server allows allocation of the current to-be-calculated resource request object according to a model of a processor supported by the current to-be-calculated resource request object, an average core number of processors required by the current to-be-calculated resource request object, a minimum processor frequency required by the current to-be-calculated resource request object, and a model of a processor of the current to-be-calculated server, a processor frequency of the current to-be-calculated server, and a processor core number of the current to-be-calculated server.
Optionally, the determining module 120 is specifically configured to determine whether the current to-be-calculated server allows allocation of the current to-be-calculated resource request object according to a model of a processor supported by the current to-be-calculated resource request object, an average core number of a processor required by the current to-be-calculated resource request object, a minimum processor frequency required by the current to-be-calculated resource request object, a model of a processor of the current to-be-calculated server, a converted processor frequency of the current to-be-calculated server, and a processor core number of the current to-be-calculated server; or determining whether the current to-be-calculated server allows the allocation of the current to-be-calculated resource request object according to the model of the processor supported by the current to-be-calculated resource request object, the average core number of the processor required by the current to-be-calculated resource request object, the minimum processor frequency required by the current to-be-calculated resource request object, the model of the processor of the current to-be-calculated server, the processor frequency of the current to-be-calculated server and the converted processor core number of the current to-be-calculated server.
Optionally, the method further comprises: a conversion module; the conversion module is used for inquiring and obtaining the performance proportion parameter of the processor of the server to be calculated relative to the reference model processor according to the model of the processor of the server to be calculated;
According to the performance proportion parameters of the processor of the server to be calculated currently, equivalent conversion is carried out on the processor frequency of the server to be calculated currently, and converted processor frequency of the server to be calculated currently is obtained;
And carrying out equivalent conversion on the processor core number of the server to be calculated according to the performance proportion parameters of the processor of the server to be calculated, so as to obtain the converted processor core number of the server to be calculated.
Optionally, the determining module 120 is specifically configured to determine that if the model of the processor of the server to be currently calculated belongs to the model of the processor supported by the resource request to be currently calculated; the number of processor cores of the server to be calculated currently is larger than or equal to the average number of processor cores required by the resource request to be calculated currently; and the converted processor frequency of the current to-be-calculated server is greater than or equal to the minimum processor frequency required by the current to-be-calculated resource request, determining that the current to-be-calculated server allows the current to-be-calculated resource request to be allocated.
The foregoing apparatus is used for executing the method provided in the foregoing embodiment, and its implementation principle and technical effects are similar, and are not described herein again.
The above modules may be one or more integrated circuits configured to implement the above methods, for example: one or more Application SPECIFIC INTEGRATED Circuits (ASIC), or one or more microprocessors (DIGITAL SINGNAL processor, DSP), or one or more field programmable gate arrays (Field Programmable GATE ARRAY, FPGA), etc. For another example, when a module above is implemented in the form of a processing element scheduler code, the processing element may be a general-purpose processor, such as a central processing unit (Central Processing Unit, CPU) or other processor that may invoke the program code. For another example, the modules may be integrated together and implemented in the form of a system-on-a-chip (SOC).
The modules may be connected or communicate with each other via wired or wireless connections. The wired connection may include a metal cable, optical cable, hybrid cable, or the like, or any combination thereof. The wireless connection may include a connection through a LAN, WAN, bluetooth, zigBee, or NFC, or any combination thereof. Two or more modules may be combined into a single module, and any one module may be divided into two or more units. It will be clear to those skilled in the art that, for convenience and brevity of description, specific working procedures of the above-described system and apparatus may refer to corresponding procedures in the method embodiments, and are not repeated in the present disclosure.
Fig. 13 is a schematic structural diagram of an electronic device according to an embodiment of the present application, where the device may be a computing device with a data processing function.
The apparatus includes: a processor 801, and a storage medium 802.
The storage medium 802 is used to store a program, and the processor 801 calls the program stored in the storage medium 802 to execute the above-described method embodiment. The specific implementation manner and the technical effect are similar, and are not repeated here.
In which a storage medium 802 stores program code that, when executed by the processor 801, causes the processor 801 to perform various steps in the resource allocation method according to various exemplary embodiments of the application described in the "exemplary methods" section of this specification.
The Processor 801 may be a general purpose Processor such as a Central Processing Unit (CPU), digital signal Processor (DIGITAL SIGNAL Processor, DSP), application SPECIFIC INTEGRATED Circuit (ASIC), field programmable gate array (Field Programmable GATE ARRAY, FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or may implement or perform the methods, steps, and logic blocks disclosed in embodiments of the application. The general purpose processor may be a microprocessor or any conventional processor or the like. The steps of a method disclosed in connection with the embodiments of the present application may be embodied directly in a hardware processor for execution, or in a combination of hardware and software modules in the processor for execution.
The storage medium 802 is a non-volatile computer-readable storage medium that can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules. The storage medium may include at least one type of storage medium, and may include, for example, flash Memory, a hard disk, a multimedia card, a card-type storage medium, a random access storage medium (Random Access Memory, RAM), a static random access storage medium (Static Random Access Memory, SRAM), a programmable Read-Only storage medium (Programmable Read Only Memory, PROM), a Read-Only storage medium (ROM), a charged erasable programmable Read-Only storage medium (ELECTRICALLY ERASABLE PROGRAMMABLE READ-Only storage, EEPROM), a magnetic storage medium, a magnetic disk, an optical disk, and the like. A storage medium is any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer, but is not limited to such. The storage medium 802 of the present application may also be circuitry or any other device capable of implementing a storage function for storing program instructions and/or data.
Optionally, the present application also provides a program product, such as a computer readable storage medium, comprising a program for performing the above-described method embodiments when being executed by a processor.
In the several embodiments provided by the present application, it should be understood that the disclosed apparatus and method may be implemented in other manners. For example, the apparatus embodiments described above are merely illustrative, e.g., the division of the units is merely a logical function division, and there may be additional divisions when actually implemented, e.g., multiple units or components may be combined or integrated into another system, or some features may be omitted or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or units, which may be in electrical, mechanical or other form.
The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, each functional unit in the embodiments of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in hardware plus software functional units.
The integrated units implemented in the form of software functional units described above may be stored in a computer readable storage medium. The software functional unit is stored in a storage medium, and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor (english: processor) to perform some of the steps of the methods according to the embodiments of the application. And the aforementioned storage medium includes: u disk, mobile hard disk, read-Only Memory (ROM), random access Memory (Random Access Memory, RAM), magnetic disk or optical disk, etc.

Claims (16)

1. A method for resource allocation, comprising:
acquiring resource information of each server in a server cluster and resource demand information of a plurality of resource request objects; the server cluster comprises old servers remained by the old distributed cluster system and new servers purchased by the new distributed cluster system correspondingly;
determining an initial resource allocation result under each resource dimension of the server according to the resource information of each server and the resource demand information of each resource request object;
determining a target resource allocation result meeting the resource demand information of each resource request object according to the initial resource allocation result under each resource dimension, wherein the target resource allocation result is used for indicating the identification of a server to be allocated to each resource request object;
The determining an initial resource allocation result under each resource dimension of the server according to the resource information of each server and the resource demand information of each resource request object comprises:
determining a plurality of resource allocation sequences, each of the resource allocation sequences being used for indicating a resource allocation calculation order of each resource request object;
and determining an initial resource allocation result in the current resource dimension of the server according to the resource information in the current resource dimension of each server, the resource sub-demand information of each resource request object in the current resource dimension and the plurality of resource allocation sequences.
2. The method for allocating resources according to claim 1, wherein determining a target resource allocation result satisfying the resource requirement information of each resource request object according to the initial resource allocation result in each resource dimension comprises:
Carrying out feasibility screening on the initial resource allocation result under each resource dimension to obtain a to-be-selected resource allocation result under each resource dimension;
And carrying out fusion processing on the to-be-selected resource allocation results in each resource dimension to obtain the target resource allocation result.
3. The method for allocating resources according to claim 2, wherein the performing feasibility screening on the initial resource allocation result in each resource dimension to obtain the candidate resource allocation result in each resource dimension comprises:
And adopting a preset screening strategy to respectively carry out feasibility screening on the initial resource allocation result in each resource dimension to obtain the resource allocation result to be selected in each resource dimension.
4. The method for allocating resources according to claim 3, wherein the preset screening policy at least includes: the method comprises the steps that a first type of resource request object and a second type of resource request object are not allowed to be distributed on the same server, the sum of standard memory requirements or average memory requirements of the resource request objects distributed on the same server is smaller than the preset proportion of the memory of the server, the task processing peak time of the resource request objects distributed on the same server is not overlapped, the sum of data increment of the resource request objects distributed on the same server is smaller than or equal to the hard disk resource of the server, the same resource request object is not allowed to be distributed on the same server, and the network peak time of the resource request objects distributed on the same server is not overlapped.
5. The method for allocating resources according to claim 2, wherein the fusing the resource allocation results to be selected in each resource dimension to obtain the target resource allocation result includes:
And carrying out intersection processing on the resource allocation results to be selected in each resource dimension to obtain the target resource allocation result.
6. The method for allocating resources according to claim 1, wherein determining the initial resource allocation result in the current resource dimension of the server according to the resource information in the current resource dimension of each server, the resource sub-requirement information in the current resource dimension of each resource request object, and the plurality of resource allocation sequences comprises:
Determining at least one intermediate resource allocation result under each resource allocation sequence according to the resource parameter of each server under the current resource dimension, the resource sub-demand information of each resource request object under the current resource dimension and the resource allocation calculation sequence indicated by each resource allocation sequence;
And carrying out fusion processing on at least one intermediate resource allocation result under each resource allocation sequence to obtain an initial resource allocation result under the current resource dimension.
7. The method of claim 6, wherein determining at least one intermediate resource allocation result in each resource allocation sequence according to the resource parameter in the current resource dimension of each server, the resource sub-requirement information of each resource request object in the current resource dimension, and the resource allocation calculation sequence indicated by each resource allocation sequence, comprises:
Determining whether the current to-be-calculated server allows allocation of the current to-be-calculated resource request object according to the resource sub-demand information of the current to-be-calculated resource request object in the current resource dimension and the resource parameter of the current to-be-calculated server in the current resource dimension;
if the server to be calculated before allows the allocation of the current resource request object to be calculated, generating a resource allocation result between the current resource request object to be calculated and the current server to be calculated in the current resource dimension; updating the resource parameters of the current to-be-calculated server in the current resource dimension according to the resource sub-demand information of the current to-be-calculated resource request object in the current resource dimension, determining whether the current to-be-calculated server allows the next to-be-calculated resource request object to be allocated or not based on the updated resource parameters of the current to-be-calculated server in the current resource dimension until the resource parameters of the current to-be-calculated server in the current resource dimension are smaller than a preset threshold, calculating the next to-be-calculated server as the current new to-be-calculated server, and circularly executing until each resource request object finishes resource allocation according to the resource allocation calculation sequence indicated by the current resource allocation sequence, and obtaining at least one intermediate resource allocation result under the current resource allocation sequence; the server to be calculated corresponding to the server to be calculated currently and the resource request object to be calculated corresponding to the resource request object to be calculated currently are determined according to the current resource allocation sequence.
8. The method for allocating resources according to claim 7, wherein the current resource dimension is a memory resource of a server; the determining whether the current to-be-calculated server allows the allocation of the current to-be-calculated resource request object according to the resource sub-demand information of the current to-be-calculated resource request object in the current resource dimension and the resource parameter of the current to-be-calculated server in the current resource dimension includes:
And determining whether the current to-be-calculated server allows the allocation of the current to-be-calculated resource request object according to the memory use requirement of the current to-be-calculated resource request object, the memory size of the current to-be-calculated server and the preset memory reserved size of the server.
9. The method for allocating resources according to claim 8, wherein determining whether the current to-be-calculated server allows allocation of the current to-be-calculated resource request object according to the memory usage requirement of the current to-be-calculated resource request object, the memory size of the current to-be-calculated server, and the preset server memory reservation size comprises:
determining the available memory size of the current server to be calculated according to the memory size of the current server to be calculated and the preset reserved memory size of the server;
And if the memory use requirement of the current resource request object to be calculated is smaller than or equal to the available memory size of the current server to be calculated, determining that the current server to be calculated allows the current resource request object to be allocated.
10. The resource allocation method of claim 7, wherein the current resource dimension is a processor resource of a server; the determining whether the current to-be-calculated server allows the allocation of the current to-be-calculated resource request object according to the resource sub-demand information of the current to-be-calculated resource request object in the current resource dimension and the resource parameter of the current to-be-calculated server in the current resource dimension includes:
And determining whether the current to-be-calculated server allows the allocation of the current to-be-calculated resource request object according to the model of the processor supported by the current to-be-calculated resource request object, the average core number of the processor required by the current to-be-calculated resource request object, the minimum processor frequency required by the current to-be-calculated resource request object, the model of the processor of the current to-be-calculated server, the processor frequency of the current to-be-calculated server and the core number of the processor of the current to-be-calculated server.
11. The method according to claim 10, wherein determining whether the currently-to-be-calculated server allows allocation of the currently-to-be-calculated resource request object according to a model number of processors supported by the currently-to-be-calculated resource request object, an average core number of processors required by the currently-to-be-calculated resource request object, a minimum processor frequency required by the currently-to-be-calculated resource request object, and a model number of processors of the currently-to-be-calculated server, a processor frequency of the currently-to-be-calculated server, and a processor core number of the currently-to-be-calculated server comprises:
determining whether the current to-be-calculated server allows allocation of the current to-be-calculated resource request object according to the model of a processor supported by the current to-be-calculated resource request object, the average core number of the processor required by the current to-be-calculated resource request object, the minimum processor frequency required by the current to-be-calculated resource request object, the model of the processor of the current to-be-calculated server, the converted processor frequency of the current to-be-calculated server and the processor core number of the current to-be-calculated server; or alternatively
And determining whether the current to-be-calculated server allows the allocation of the current to-be-calculated resource request object according to the model of the processor supported by the current to-be-calculated resource request object, the average core number of the processor required by the current to-be-calculated resource request object, the minimum processor frequency required by the current to-be-calculated resource request object, the model of the processor of the current to-be-calculated server, the processor frequency of the current to-be-calculated server and the converted processor core number of the current to-be-calculated server.
12. The resource allocation method according to claim 11, wherein the scaled processor frequency of the currently-to-be-calculated server and the scaled processor core number of the currently-to-be-calculated server are calculated as follows:
inquiring to obtain performance proportion parameters of the processor of the server to be calculated relative to a reference model processor according to the model of the processor of the server to be calculated;
According to the performance proportion parameters of the processor of the server to be calculated currently, equivalent conversion is carried out on the processor frequency of the server to be calculated currently, and converted processor frequency of the server to be calculated currently is obtained;
And carrying out equivalent conversion on the processor core number of the server to be calculated according to the performance proportion parameters of the processor of the server to be calculated, so as to obtain the converted processor core number of the server to be calculated.
13. The method for allocating resources according to claim 11, wherein the determining whether the current to-be-calculated server allows allocation of the current to-be-calculated resource request object according to the model of the processor supported by the current to-be-calculated resource request object, the average number of cores of the processor required by the current to-be-calculated resource request object, the minimum processor frequency required by the current to-be-calculated resource request object, and the model of the processor of the current to-be-calculated server, the scaled processor frequency of the current to-be-calculated server, and the number of cores of the processor of the current to-be-calculated server comprises:
If the model of the processor of the server to be calculated currently belongs to the model of the processor supported by the resource request to be calculated currently; the number of processor cores of the server to be calculated currently is larger than or equal to the average number of processor cores required by the resource request to be calculated currently; and the converted processor frequency of the current to-be-calculated server is greater than or equal to the minimum processor frequency required by the current to-be-calculated resource request, determining that the current to-be-calculated server allows the current to-be-calculated resource request to be allocated.
14. A resource allocation apparatus, comprising: the device comprises an acquisition module and a determination module;
the acquisition module is used for acquiring the resource information of each server in the server cluster and the resource demand information of a plurality of resource request objects; the server cluster comprises old servers remained by the old distributed cluster system and new servers purchased by the new distributed cluster system correspondingly;
the determining module is used for determining an initial resource allocation result under each resource dimension of the server according to the resource information of each server and the resource demand information of each resource request object;
The determining module is configured to determine, according to an initial resource allocation result in each resource dimension, a target resource allocation result that satisfies resource demand information of each resource request object, where the target resource allocation result is used to indicate an identifier of a server to be allocated to each resource request object;
The determining module is specifically configured to determine a plurality of resource allocation sequences, where each resource allocation sequence is used to indicate a resource allocation calculation sequence of each resource request object;
and determining an initial resource allocation result in the current resource dimension of the server according to the resource information in the current resource dimension of each server, the resource sub-demand information of each resource request object in the current resource dimension and the plurality of resource allocation sequences.
15. An electronic device, comprising: a processor, a storage medium and a bus, the storage medium storing program instructions executable by the processor, the processor and the storage medium communicating over the bus when the electronic device is running, the processor executing the program instructions to implement the resource allocation method according to any one of claims 1 to 13.
16. A computer readable storage medium, characterized in that the storage medium has stored thereon a computer program which, when executed by a processor, implements the resource allocation method according to any of claims 1 to 13.
CN202410383638.1A 2024-04-01 2024-04-01 Resource allocation method, device, electronic equipment and storage medium Active CN117971510B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202410383638.1A CN117971510B (en) 2024-04-01 2024-04-01 Resource allocation method, device, electronic equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202410383638.1A CN117971510B (en) 2024-04-01 2024-04-01 Resource allocation method, device, electronic equipment and storage medium

Publications (2)

Publication Number Publication Date
CN117971510A CN117971510A (en) 2024-05-03
CN117971510B true CN117971510B (en) 2024-06-25

Family

ID=90864929

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202410383638.1A Active CN117971510B (en) 2024-04-01 2024-04-01 Resource allocation method, device, electronic equipment and storage medium

Country Status (1)

Country Link
CN (1) CN117971510B (en)

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115373828A (en) * 2021-05-19 2022-11-22 中移(苏州)软件技术有限公司 Resource scheduling method, device, equipment and computer storage medium
CN115687184A (en) * 2021-07-23 2023-02-03 华为技术有限公司 Resource allocation method and device
CN115766336A (en) * 2022-09-27 2023-03-07 中国联合网络通信集团有限公司 Resource allocation method, device, equipment and storage medium

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
Newell, A等.《28th ACM Symposium on Operating Systems Principles》.2021,全文. *
服务器虚拟化在医院信息系统中的设计与实现;洪怀江;苏晓广;;中国医疗设备;20170325(03);全文 *

Also Published As

Publication number Publication date
CN117971510A (en) 2024-05-03

Similar Documents

Publication Publication Date Title
Xu et al. Dynamic resource allocation for load balancing in fog environment
CN108632365B (en) Service resource adjusting method, related device and equipment
CN108090225B (en) Database instance running method, device and system and computer readable storage medium
CN108776934B (en) Distributed data calculation method and device, computer equipment and readable storage medium
US9280390B2 (en) Dynamic scaling of a cluster of computing nodes
CN111966500B (en) Resource scheduling method and device, electronic equipment and storage medium
US9479382B1 (en) Execution plan generation and scheduling for network-accessible resources
CN105049268A (en) Distributed computing resource allocation system and task processing method
WO2021227999A1 (en) Cloud computing service system and method
CN105556499A (en) Intelligent auto-scaling
CN107968810A (en) A kind of resource regulating method of server cluster, device and system
CN113742033B (en) Kubernetes cluster federation system and implementation method thereof
CN111309440B (en) Method and equipment for managing and scheduling multiple types of GPUs
CN104639594A (en) System and method for allocating physical resources and virtual resources
CN110138688A (en) Dynamic adjusts method, apparatus, equipment and the readable storage medium storing program for executing of business interface
US12026536B2 (en) Rightsizing virtual machine deployments in a cloud computing environment
WO2020148342A1 (en) A method and a system for managing the computing resources of a cloud computing platform
CN115033340A (en) Host selection method and related device
US20220382603A1 (en) Generating predictions for host machine deployments
CN110096339B (en) System load-based capacity expansion and contraction configuration recommendation system and method
CN102511041B (en) Computer implementation and calculating system
CN114844791A (en) Cloud service automatic management and distribution method and system based on big data and storage medium
CN114443294A (en) Big data service component deployment method, system, terminal and storage medium
CN114070847B (en) Method, device, equipment and storage medium for limiting current of server
CN117971510B (en) Resource allocation method, device, electronic equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant