CN116010450A - Method, apparatus, electronic device and computer readable medium for resource allocation - Google Patents
Method, apparatus, electronic device and computer readable medium for resource allocation Download PDFInfo
- Publication number
- CN116010450A CN116010450A CN202310118053.2A CN202310118053A CN116010450A CN 116010450 A CN116010450 A CN 116010450A CN 202310118053 A CN202310118053 A CN 202310118053A CN 116010450 A CN116010450 A CN 116010450A
- Authority
- CN
- China
- Prior art keywords
- user
- query
- resource allocation
- target
- representing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Abstract
Embodiments of the present application provide a method, an apparatus, an electronic device, and a computer readable medium for resource allocation. The method comprises the following steps: responding to a resource allocation instruction, determining priority data and query data corresponding to each user in a target cluster, wherein the priority data is used for representing the importance degree of the service corresponding to the user, and the query data is used for representing the influence of the user query on the resources in the target cluster; determining a target planning relation of resource allocation based on priority data and query data corresponding to each user, wherein the target planning relation is used for determining a resource allocation scheme meeting constraint conditions, and the constraint conditions are used for representing conditions which should be met when resource allocation is carried out on each user in a target cluster; according to the target planning relation, determining an optimal resource allocation scheme corresponding to a target query quantity, wherein the target query quantity is used for representing the maximum query quantity which can be realized by the target cluster under the condition that constraint conditions are met; and according to the optimal resource allocation scheme, performing resource allocation on each user in the target cluster.
Description
Technical Field
The present disclosure relates to the field of data analysis technologies, and in particular, to a method, an apparatus, an electronic device, and a computer readable medium for resource allocation.
Background
As more and more business data migrate to the clickHouse cluster, the types of data stored in the clickHouse cluster and the query execution are increasingly complex, the resource use conditions are different, and the resource imbalance between the user level and the time window level exists.
At present, unbalanced resources are mainly allocated by adopting a service isolation and manual analysis method, however, the methods have high time and labor cost and inaccurate resource allocation results because a great deal of time is required to arrange service data or to recalculate the service data when the service data change.
Disclosure of Invention
An object of the embodiments of the present application is to provide a method, an apparatus, an electronic device, and a computer readable medium for resource allocation, which can implement fast, automatic, and accurate resource allocation.
In order to solve the technical problems, embodiments of the present application are realized by the following aspects.
In a first aspect, an embodiment of the present application provides a method for allocating resources, including: responding to a resource allocation instruction, determining priority data and query data corresponding to each user in a target cluster, wherein the priority data is used for representing the importance degree of a service corresponding to the user, and the query data is used for representing the influence of user query on resources in the target cluster; determining a target planning relation of resource allocation based on priority data and query data corresponding to each user, wherein the target planning relation is used for determining a resource allocation scheme meeting constraint conditions, and the constraint conditions are used for representing conditions which should be met when resource allocation is carried out on each user in the target cluster; determining an optimal resource allocation scheme corresponding to a target query volume according to the target planning relationship, wherein the target query volume is used for representing the maximum query volume which can be realized by the target cluster under the condition that the constraint condition is met; and carrying out resource allocation on each user in the target cluster according to the optimal resource allocation scheme.
In a second aspect, an embodiment of the present application provides an apparatus for resource allocation, including: the first determining module is used for responding to the resource allocation instruction, determining priority data and query data corresponding to each user in the target cluster, wherein the priority data is used for representing the importance degree of the service corresponding to the user, and the query data is used for representing the influence of the user query on the resources in the target cluster; the second determining module is used for determining a target planning relation of resource allocation based on priority data and query data corresponding to each user, wherein the target planning relation is used for determining a resource allocation scheme meeting constraint conditions, and the constraint conditions are used for representing conditions which should be met when the resources are allocated to each user in the target cluster; a third determining module, configured to determine, according to the target planning relationship, an optimal resource allocation scheme corresponding to a target query volume, where the target query volume is used to characterize a maximum query volume that can be achieved by the target cluster when the constraint condition is satisfied; and the resource allocation module is used for allocating resources to each user in the target cluster according to the optimal resource allocation scheme.
In a third aspect, an embodiment of the present application provides an electronic device, including: a memory, a processor, and computer-executable instructions stored on the memory and executable on the processor, which when executed by the processor, implement the method of resource allocation as described in the first aspect above.
In a fourth aspect, embodiments of the present application provide a computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the method for allocating resources according to the first aspect.
In the embodiment of the application, priority data and query data corresponding to each user in a target cluster are determined, wherein the priority data is used for representing the importance degree of the service corresponding to the user, and the query data is used for representing the influence of user query on resources in the target cluster; determining a target planning relation of resource allocation based on priority data and query data corresponding to each user, wherein the target planning relation is used for determining a resource allocation scheme meeting constraint conditions, and the constraint conditions are used for representing conditions which should be met when resource allocation is carried out on each user in the target cluster; determining an optimal resource allocation scheme corresponding to a target query volume according to the target planning relationship, wherein the target query volume is used for representing the maximum query volume which can be realized by the target cluster under the condition that the constraint condition is met; and carrying out resource allocation on each user in the target cluster according to the optimal resource allocation scheme. The resource allocation process does not need to be manually involved in carrying out complex calculation, the time and labor cost can be reduced, and a scheme with the maximum resource utilization is found out through a target planning relationship to carry out resource allocation based on priority data and query data corresponding to each user, so that the accuracy of an allocation result can be ensured, and therefore, the rapid, automatic and accurate resource allocation can be realized.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings that are required to be used in the embodiments or the description of the prior art will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments described in the present application, and that other drawings may be obtained according to these drawings without inventive effort to a person skilled in the art.
Fig. 1 is a schematic flow chart of a method for allocating resources according to an embodiment of the present application;
FIG. 2 is another flow chart of a method for resource allocation according to an embodiment of the present application;
fig. 3 is a schematic structural diagram of an apparatus for resource allocation according to an embodiment of the present application;
fig. 4 is a schematic hardware structure of an electronic device for executing a method for resource allocation according to an embodiment of the present application.
Detailed Description
In order to better understand the technical solutions in the present application, the following description will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only some embodiments of the present application, not all embodiments. All other embodiments, which can be made by one of ordinary skill in the art based on the embodiments herein without making any inventive effort, shall fall within the scope of the present application.
As described above, the existing resource allocation method requires a lot of time to sort the service data or to recalculate when the service data is changed, and thus has high time and labor costs, and the resource allocation result may be inaccurate.
As more and more business data migrate to the clickHouse cluster, the types of data and queries stored in the clickHouse cluster are increasingly complex, and the resource usage is also different. From the aspect of reading and writing, more computing resources are needed for reading more and less services; less computing resources are required to write more read less traffic. From the query frequency, part of service query frequency is stable, and the resource usage amount is stable; the frequency of part of service inquiry fluctuates with the service data, the use amount of resources in the peak period is extremely high, and the use amount of resources in the low peak period is almost zero. From the importance of data, the monitoring type query needs real-time statistics, is important, and relates to intolerance failure of the query, the analysis type data has low validity requirement, and forcible cancellation is allowed when resources are insufficient. In addition, some non-standard query conditions such as oversized service query, over-frequency query and the like exist.
The main treatment methods aiming at the problems at present comprise a business isolation method and a traditional manual analysis method. The service isolation method selects machines with different resource configurations to build clusters according to different service data types, stores the data of the re-computing service in the computing clusters, and stores the re-storing data in the storage clusters. Multiple clusters are needed, and the operation and maintenance cost is high. And by the traditional manual analysis method, different Profile and Quote limits are set for different users according to the characteristics of various business data by counting the use condition of various businesses so as to achieve the effect of balancing the use of resources. The traditional manual mode generally sets the resource utilization ratio of each service by comparing the dimensions of query frequency, query time length, resource consumption and the like of each service and combining the importance weights of each service, but various factors cannot be comprehensively considered due to the fact that the service utilization ratio is always considered.
In this case, it is possible to consider using factors to be considered as influence factors, setting an objective function by a relationship between the influence factors and decision variables, and finding out a scheme of maximizing resource utilization within a range allowed by constraint conditions, that is, solving an optimal solution of resource allocation by using a linear programming algorithm.
In view of this, the embodiments of the present application aim to provide a method for allocating resources, and the main technical concept is as follows: by replacing manual analysis, the time and labor cost are reduced; and acquiring a resource allocation scheme based on the data corresponding to the user, so that the accuracy of resource allocation is improved.
Fig. 1 is a schematic flow chart of a method for allocating resources provided in an embodiment of the present application, where the method may be performed by an electronic device, for example, a terminal device or a server device. In other words, the method may be performed by software or hardware installed at a terminal device or a server device. The service end includes but is not limited to: a single server, a server cluster, a cloud server or a cloud server cluster, and the like. The method is suitable for single-cluster and multi-service scenes. As shown, the method may include the following steps.
Step S110: and determining priority data and query data corresponding to each user in the target cluster in response to the resource allocation instruction.
The resource allocation instruction is used for triggering the resource allocation of the target cluster and can be realized through an independent script program. The target cluster is a database cluster, such as a Clickhouse cluster, that can set an upper limit on the resources used by each user. The priority data is used to characterize the importance of the service corresponding to the user, and is usually set by an administrator or the user. The higher the importance of the user service, the higher the priority corresponding to the user. The query data is used to characterize the impact of a user query on resources in the target cluster. The query data may be historical query data or historical statistics corresponding to historical query conditions of the user. Priority data and query data correspond to considerations in resource allocation statistics.
Clickhouse clusters have resource imbalances at the user level, e.g., a portion of users require more computing resources for traffic and other users require less computing resources for traffic. By determining priority data and query data corresponding to each user, resource benefits and resource effects corresponding to the users can be determined.
Step S120: and determining a target planning relation of resource allocation based on the priority data and the query data corresponding to each user.
The target planning relationship is used for determining a resource allocation scheme meeting constraint conditions, and the resource allocation scheme comprises decision variables and the constraint conditions. The decision variable may be an unknown quantity to be determined, which may be decided and controlled by a decision maker, e.g. an administrator. The constraint condition is used for representing the condition which is required to be met when the resource allocation is carried out on each user in the target cluster or the limitation of various resource conditions when the decision variable takes value.
Step S130: and determining an optimal resource allocation scheme corresponding to the target query volume according to the target planning relation.
The target query volume is used for representing the maximum query volume which can be realized by the target cluster under the condition that the constraint condition is met.
Step S140: and carrying out resource allocation on each user in the target cluster according to the optimal resource allocation scheme.
Through the optimal resource allocation scheme, under the condition of limited resources, the resources of each user can be limited, so that the target cluster can realize the maximum query quantity.
In the embodiment of the application, priority data and query data corresponding to each user in a target cluster are determined by responding to a resource allocation instruction, wherein the priority data is used for representing the importance degree of a service corresponding to the user, and the query data is used for representing the influence of user query on resources in the target cluster; determining a target planning relation of resource allocation based on priority data and query data corresponding to each user, wherein the target planning relation is used for determining a resource allocation scheme meeting constraint conditions, and the constraint conditions are used for representing conditions which should be met when resource allocation is carried out on each user in the target cluster; determining an optimal resource allocation scheme corresponding to a target query volume according to the target planning relationship, wherein the target query volume is used for representing the maximum query volume which can be realized by the target cluster under the condition that the constraint condition is met; according to the optimal resource allocation scheme, each user in the target cluster is allocated, the complex calculation is not needed in the resource allocation process, the time and labor cost can be reduced, the scheme with the maximum resource utilization is found out through the target planning relationship based on the priority data and the query data corresponding to each user to allocate the resources, and the accuracy of an allocation result can be ensured, so that the rapid, automatic and accurate resource allocation can be realized.
In one possible implementation, the target planning relationship comprises a linear function; the determining the target planning relation of the resource allocation based on the priority data and the query data corresponding to each user comprises the following steps: determining a linear function of resource allocation based on the priority data and the query data corresponding to each user, wherein the linear function comprises the constraint condition and is used for representing a functional relation between the query quantity realized by the target cluster and the resource quantity allocated to each user under the constraint condition; and determining an optimal resource allocation scheme corresponding to the target query volume according to the target planning relationship comprises: and determining a feasible solution corresponding to the maximum value of the linear function as an optimal solution among a plurality of feasible solutions which are corresponding to the linear function and meet the constraint condition, and determining an optimal resource allocation scheme based on the optimal solution, wherein the maximum value of the linear function represents the target query quantity.
The linear function, i.e. the objective function, is composed of decision variables reflecting the requirements of the decision maker, e.g. an administrator or a user, for the linear programming. The set of constraints is a feasible region of the linear function. When the feasible region is a limited region and is not empty, the linear function has limited feasible solutions comprising an optimal solution, and the optimal objective function value is in the feasible region; when the feasible region is an infinite region, the linear function value is unbounded; when the feasible region is an empty set, the linear function has no feasible solution. One feasible solution corresponds to one resource allocation scheme, which may include the amount of resources allocated to each user.
Specifically, the amount of queries implemented by the target cluster is affected by the amount of resources allocated to the user, the priority data, and the query data, and the corresponding constraint conditions need to be satisfied.
In one possible implementation, the priority data is positively correlated with the optimal solution and the query data is negatively correlated with the optimal solution if the constraint condition is satisfied.
Specifically, under the condition that other data are the same, the higher the user priority, the larger the target query quantity corresponding to the optimal solution, and the larger the realized resource benefit. Similarly, under the condition that other data are the same, the smaller the value of the indicated query data is, the larger the target query quantity corresponding to the optimal solution is, and the larger the realized resource benefit is.
In one possible implementation, the constraint includes: the sum of the resources allocated to all users is less than or equal to the maximum available resources of the target cluster; the sum of the maximum query amounts which can be born by the resources distributed to all users in unit time is smaller than or equal to the maximum query amount which can be born by the target cluster in unit time; for each user, the resource amount allocated to the user is greater than or equal to the minimum resource amount required by the user, and the minimum resource amount is determined according to the priority data and the query data corresponding to the user.
Optionally, the amount of available resources and the amount of sustainable queries of the target cluster are considered based on the user priority data and the user history query data. Specifically, the sum of the resources allocated to all the users of the target cluster is not greater than the maximum available resource of the target cluster, where the resources allocated to the users may be the CPU size, the memory size, and the like, for example, the maximum memory of the target cluster is 128G, and then the sum of the memories occupied by all the users is less than or equal to 128G. The sum of the query amounts of all users of the target cluster in unit time is not more than the maximum query amount which can be born by the target cluster in unit time, for example, the maximum query amount which can be born by the target cluster in 1 minute is 10 ten thousand times, and then the sum of the query amounts of all users in 1 minute is less than or equal to 10 ten thousand times.
In addition, according to the user priority data and the user history query data, the condition that each user meets the resources required by normal service, namely the minimum resource quantity of the user service is met, and the resources allocated to each service in the constraint condition are not smaller than the minimum resource quantity. The minimum resource amount is predetermined experience value data, specifically, experience value data determined according to priority data and query data corresponding to each user.
In one possible implementation, the query data includes normal query time-consuming data and an abnormal query factor, the normal query time-consuming data being used to characterize an average resource consumption of a user completing a single query without an abnormal query; the abnormal query factor is used to characterize the degree of negative impact of an abnormal query on a normal query.
The average time consumed by the normal inquiry of the user corresponds to the consumption of resources of the single inquiry of the user. Resource considerations are manifold and, optionally, use time to unify the consumption of individual queries on behalf of a user. The less a single query resource consumes, the more queries a fixed resource can complete.
The abnormal query factor is related data of abnormal queries of users, and the abnormal queries have negative effects on the normal queries, for example, abnormal queries which occupy a large amount of resources can cause the normal queries to be unable to be executed normally, and are considered as an abnormal factor. Optionally, the abnormal query factor is a ratio of total abnormal query occupancy time to total abnormal query time, i.e. abnormal query factor = total abnormal query occupancy time/total abnormal query time. According to the scheme, the abnormal query of the user history is taken as an abnormal factor in the processing, so that the method has higher accuracy.
It can be understood that, under the condition that the constraint condition is satisfied and the resource amount is fixed, the shorter the average query time is, the fewer abnormal queries occur, the larger the target query amount can be realized, and the generated resource utilization benefit is larger.
In one possible implementation, the linear function is:
the constraint conditions are as follows:
wherein z is used for representing the query quantity realized by the target cluster under the constraint condition, i is used for representing any user, n is used for representing the number of users, and x i For indicating the amount of resources allocated to user i, p i For representing the priority data corresponding to user i, t i For representing the normal query time-consuming data corresponding to user i, a i For representing the abnormal query factor corresponding to user i, l i The method comprises the steps of representing the minimum resource quantity required by a user i, and N represents the maximum query quantity which can be born by the target cluster in unit time; and the maximum available resource of the target cluster takes a value of 1.
The minimum resource amount required by the user and the maximum query amount which can be born by the target group in unit time are historical experience values.
In resource allocation, we need to normalize all available resources of the clickHouse cluster, i.e. the total available resources are processed in units of "1". Independent of specific resource data, such as disk size, CPU size, memory size, etc. Correspondingly, the resources allocated to each user are expressed in percentage form. Thus, the method can be universally applied to ClickHouse clusters with different specifications, namely ClickHouse clusters with different disk sizes, CPU sizes, memory sizes and the like.
In the embodiment of the application, priority data and query data corresponding to each user in a target cluster are determined by responding to a resource allocation instruction, wherein the priority data is used for representing the importance degree of a service corresponding to the user, and the query data is used for representing the influence of user query on resources in the target cluster; determining a target planning relation of resource allocation based on priority data and query data corresponding to each user, wherein the target planning relation is used for determining a resource allocation scheme meeting constraint conditions, and the constraint conditions are used for representing conditions which should be met when resource allocation is carried out on each user in the target cluster; determining an optimal resource allocation scheme corresponding to a target query volume according to the target planning relationship, wherein the target query volume is used for representing the maximum query volume which can be realized by the target cluster under the condition that the constraint condition is met; and carrying out resource allocation on each user in the target cluster according to the optimal resource allocation scheme. The resource allocation process does not need to be manually involved in carrying out complex calculation, the time and labor cost can be reduced, and a scheme with the maximum resource utilization is found out through a target planning relationship to carry out resource allocation based on priority data and query data corresponding to each user, so that the accuracy of an allocation result can be ensured, and therefore, the rapid, automatic and accurate resource allocation can be realized.
Fig. 2 is another flow chart of a method for allocating resources according to an embodiment of the present application. As shown, the method may include the following steps.
Step S200:
generating the resource allocation instruction according to the user request; and/or generating the resource allocation instruction under the condition that the user and/or the user service in the target cluster are detected to be changed, wherein each user corresponds to at least one user service, and the resources allocated to the user are determined by the user service corresponding to the user.
Specifically, due to the complexity and uncertainty of the clickHouse service data, an adjustment needs to be made to the resource allocation scheme of each user at random. When the program detects that the user service is changed or a new user is added, a resource allocation instruction is generated to trigger a resource allocation flow. Or when the user actively requests to trigger the resource allocation flow, for example, when the resource allocation is initially performed, generating a resource allocation instruction according to the user request so as to trigger the resource allocation flow.
Each user corresponds to at least one user service, and the resources allocated to the user are affected by the user service corresponding to the user. In particular, when the user corresponds to a user service, the importance level of the user, i.e., the user priority, depends on the importance level of the user service. When a user corresponds to a plurality of user services, the importance level of the user depends on the highest importance level among the plurality of user services.
Specifically, for example, after the resource allocation is performed on each user in the target cluster according to the optimal resource allocation scheme, a resource allocation instruction may be generated to restart the resource allocation flow under the condition that the user or the user service changes. Namely, responding to the resource allocation instruction, and determining updated priority data and query data corresponding to each user; determining a target planning relation of updated resource allocation based on updated priority data and query data corresponding to each user; according to the updated target planning relation, adjusting the optimal resource allocation scheme, namely determining the updated optimal resource allocation scheme corresponding to the target query quantity; and performing resource allocation on each user in the target cluster according to the updated optimal resource allocation scheme.
Alternatively, only the priority data and the query data corresponding to the changed user may be updated. The updated priority data and query data corresponding to the changed first user and the updated priority data and query data corresponding to the unchanged second user are used as updated priority data and query data corresponding to each user in the target cluster for the subsequent resource allocation flow according to the change of the user and/or the change of the user service in the target cluster.
And step S210, responding to the resource allocation instruction, and determining priority data, normal query time-consuming data and abnormal query factors corresponding to each user in the target cluster.
Step S220, determining a linear function based on priority data, normal query time-consuming data and abnormal query factors corresponding to each user.
And S230, determining a feasible solution corresponding to the maximum value of the linear function as an optimal solution among a plurality of feasible solutions corresponding to the linear function and meeting the constraint condition, and determining an optimal resource allocation scheme based on the optimal solution.
And step 240, performing resource allocation on each user in the target cluster according to the optimal resource allocation scheme.
Steps S210-S240 may employ descriptions of the corresponding steps of the previous embodiment, and the description of the repeatable portions is omitted here.
The linear function is solved to obtain the resource allocation scheme with the maximum profitability, but before the resource allocation scheme is executed, historical query data of each user needs to be counted in advance. In the application, the historical query data of each user is statistically regulated according to the statistical rules through the special script. The whole process of resource allocation to the clickHouse using a linear programming algorithm can be regarded as a dynamic loop process. The whole process is as follows: program detection finds that the user service changes or a new user is added, and triggers a resource allocation flow; confirming each influence factor according to the user priority and the user history inquiry condition; confirming a linear function and obtaining an optimal solution according to the linear function; and adjusting a user resource allocation scheme by using the optimal solution.
Therefore, by the method for resource allocation in the embodiment of the application, the linear programming thought is introduced into the problem of resource allocation in the field of databases, and the optimal scheme for the resource allocation of the databases is comprehensively prepared by using the linear programming algorithm, so that the dynamic adjustment of resources is realized while the automation of the dynamic allocation of ClickHouse resources is realized, and the availability of the resource allocation scheme is improved.
Fig. 3 shows a schematic structural diagram of an apparatus for resource allocation according to an embodiment of the present application, where the apparatus 300 includes: a first determination module 310, a second determination module 320, a third determination module 330, and a resource allocation module 340.
The first determining module 310 is configured to determine, in response to a resource allocation instruction, priority data and query data corresponding to each user in a target cluster, where the priority data is used to characterize an importance level of a service corresponding to the user, and the query data is used to characterize an influence of a user query on resources in the target cluster. The second determining module 320 is configured to determine, based on priority data and query data corresponding to each user, a target planning relationship for resource allocation, where the target planning relationship is used to determine a resource allocation scheme that meets constraint conditions, and the constraint conditions are used to characterize conditions that should be met when resource allocation is performed on each user in the target cluster. The third determining module 330 is configured to determine, according to the target planning relationship, an optimal resource allocation scheme corresponding to a target query volume, where the target query volume is used to characterize a maximum query volume that the target cluster can implement if the constraint condition is satisfied. The resource allocation module 340 is configured to allocate resources to each user in the target cluster according to the optimal resource allocation scheme.
In one possible implementation, the target planning relationship comprises a linear function. The second determining module 320 is specifically configured to determine, based on the priority data and the query data corresponding to each user, a linear function for resource allocation, where the linear function includes the constraint condition, and the linear function is used to characterize a functional relationship between a query quantity implemented by the target cluster and a resource quantity allocated to the user under the constraint condition. The third determining module 330 is further configured to determine, from a plurality of feasible solutions corresponding to the linear function and satisfying the constraint condition, a feasible solution corresponding to a maximum value of the linear function as an optimal solution, and determine an optimal resource allocation scheme based on the optimal solution, where the maximum value of the linear function characterizes the target query volume.
In one possible implementation, the priority data is positively correlated with the optimal solution and the query data is negatively correlated with the optimal solution if the constraint condition is satisfied.
In one possible implementation, the constraint includes: the sum of the resources allocated to all users is less than or equal to the maximum available resources of the target cluster; the sum of the maximum query amounts which can be born by the resources distributed to all users in unit time is smaller than or equal to the maximum query amount which can be born by the target cluster in unit time; the resource quantity allocated to the user is larger than or equal to the minimum resource quantity required by the user, and the minimum resource quantity is determined according to the priority data and the query data corresponding to the user.
In one possible implementation, the query data includes normal query time-consuming data and an abnormal query factor, the normal query time-consuming data being used to characterize an average resource consumption of a user completing a single query without an abnormal query; the abnormal query factor is used to characterize the degree of negative impact of an abnormal query on a normal query.
In one possible implementation, the linear function is:
the constraint conditions are as follows:
wherein z is used for representing the query quantity realized by the target cluster, i is used for representing any user, n is used for representing the number of users, and x i For indicating the amount of resources allocated to user i, p i For representing the priority data corresponding to user i, t i For representing the normal query time-consuming data corresponding to user i, a i For representing the abnormal query factor corresponding to user i, l i The method comprises the steps of representing the minimum resource quantity required by a user i, and N represents the maximum query quantity which can be born by the target cluster in unit time; and the maximum available resource of the target cluster takes a value of 1.
In a possible implementation manner, the device for allocating resources further includes a generating module, configured to generate the resource allocation instruction according to a user request, where the user request is used to request the target cluster to allocate resources; and/or generating the resource allocation instruction under the condition that the user and/or the user service in the target cluster are detected to be changed, wherein each user corresponds to at least one user service, and the resources allocated to the user are influenced by the user service corresponding to the user.
The apparatus 300 provided in this embodiment of the present application may perform the methods described in the foregoing method embodiments, and implement the functions and beneficial effects of the methods described in the foregoing method embodiments, which are not described herein again.
Fig. 4 shows a schematic diagram of a hardware structure of an electronic device 400 for performing the method of resource allocation provided in the embodiments of the present application, and referring to this figure, at a hardware level, the electronic device includes a processor 410, optionally including an internal bus 420, a network interface 430, and a memory. The Memory may include a Memory 440, such as a Random-Access Memory (RAM), and may further include a non-volatile Memory 450, such as at least 1 disk Memory. Of course, the electronic device may also include hardware required for other services.
The processor 410, network interface 430, and memory may be interconnected by an internal bus 420, which may be an industry standard architecture (Industry Standard Architecture, ISA) bus, a peripheral component interconnect (Peripheral Component Interconnect, PCI) bus, or an extended industry standard architecture (Extended Industry Standard Architecture, EISA) bus, among others. The buses may be classified as address buses, data buses, control buses, etc. For ease of illustration, only one bi-directional arrow is shown in the figure, but not only one bus or one type of bus.
And the memory is used for storing programs. In particular, the program may include program code including computer-operating instructions. The storage may include memory 440 and nonvolatile storage 450 and provide instructions and data to the processor 410.
The processor 410 reads the corresponding computer program from the non-volatile memory 450 into the memory 440 and then runs to form a means of locating the target user at a logical level. The processor 410 executes the programs stored in the memory and is specifically configured to perform the methods described in the embodiments of fig. 1 and 2, and achieve the same or corresponding technical effects.
The methods disclosed above in the embodiments of fig. 1 and 2 of the present application may be applied to a processor or implemented by the processor 410. The processor 410 may be an integrated circuit chip having signal processing capabilities. In implementation, the steps of the above method may be performed by integrated logic circuitry in hardware or instructions in software in the processor 410. The processor 410 may be a general-purpose processor, including a central processing unit (Central Processing Unit, CPU), a network processor (Network Processor, NP), etc.; but also digital signal processors (Digital Signal Processor, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC), field programmable gate arrays (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The disclosed methods, steps, and logic blocks in the embodiments of the present application may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of a method disclosed in connection with the embodiments of the present application may be embodied directly in hardware, in a decoded processor, or in a combination of hardware and software modules in a decoded processor. The software modules may be located in a random access memory, flash memory, read only memory, programmable read only memory, or electrically erasable programmable memory, registers, etc. as well known in the art. The storage medium is located in a memory, and the processor 410 reads information in the memory and performs the steps of the method described above in connection with its hardware.
The electronic device may also execute the methods described in the foregoing method embodiments, and implement the functions and beneficial effects of the methods described in the foregoing method embodiments, which are not described herein.
Of course, other implementations, such as a logic device or a combination of hardware and software, are not excluded from the electronic device of the present application, that is, the execution subject of the following processing flow is not limited to each logic unit, but may be hardware or a logic device.
The embodiments of the present application also provide a computer-readable storage medium storing one or more programs that, when executed by an electronic device including a plurality of application programs, cause the electronic device to perform the methods described in the embodiments of fig. 1 and 2, and achieve the same or corresponding technical effects.
The computer readable storage medium includes Read-Only Memory (ROM), random access Memory (Random Access Memory RAM), magnetic disk or optical disk, etc.
Further, embodiments of the present application also provide a computer program product comprising a computer program stored on a non-transitory computer readable storage medium, the computer program comprising program instructions which, when executed by a computer, implement the methods described in the embodiments of fig. 1 and 2 and achieve the same or corresponding technical effects.
In summary, the foregoing description is only of the preferred embodiments of the present application, and is not intended to limit the scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application.
The system, apparatus, module or unit set forth in the above embodiments may be implemented in particular by a computer chip or entity, or by a product having a certain function. One typical implementation is a computer. In particular, the computer may be, for example, a personal computer, a laptop computer, a cellular telephone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.
Computer readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of storage media for a computer include, but are not limited to, phase change memory (PRAM), static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium, which can be used to store information that can be accessed by a computing device. Computer-readable media, as defined herein, does not include transitory computer-readable media (transmission media), such as modulated data signals and carrier waves.
It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article or apparatus that comprises the element.
In this specification, each embodiment is described in a progressive manner, and identical and similar parts of each embodiment are all referred to each other, and each embodiment mainly describes differences from other embodiments. In particular, for system embodiments, since they are substantially similar to method embodiments, the description is relatively simple, as relevant to see a section of the description of method embodiments.
Claims (10)
1. A method of resource allocation, comprising:
responding to a resource allocation instruction, determining priority data and query data corresponding to each user in a target cluster, wherein the priority data is used for representing the importance degree of a service corresponding to the user, and the query data is used for representing the influence of user query on resources in the target cluster;
Determining a target planning relation of resource allocation based on priority data and query data corresponding to each user, wherein the target planning relation is used for determining a resource allocation scheme meeting constraint conditions, and the constraint conditions are used for representing conditions which should be met when resource allocation is carried out on each user in the target cluster;
determining an optimal resource allocation scheme corresponding to a target query volume according to the target planning relationship, wherein the target query volume is used for representing the maximum query volume which can be realized by the target cluster under the condition that the constraint condition is met;
and carrying out resource allocation on each user in the target cluster according to the optimal resource allocation scheme.
2. The method of claim 1, wherein the target planning relationship comprises a linear function;
the determining the target planning relation of the resource allocation based on the priority data and the query data corresponding to each user comprises the following steps:
determining a linear function of resource allocation based on the priority data and the query data corresponding to each user, wherein the linear function comprises the constraint condition and is used for representing a functional relation between the query quantity realized by the target cluster and the resource quantity allocated to each user under the constraint condition; and
The determining an optimal resource allocation scheme corresponding to the target query volume according to the target planning relation comprises the following steps:
and determining a feasible solution corresponding to the maximum value of the linear function as an optimal solution among a plurality of feasible solutions which are corresponding to the linear function and meet the constraint condition, and determining an optimal resource allocation scheme based on the optimal solution, wherein the maximum value of the linear function represents the target query quantity.
3. The method of claim 2, wherein the priority data is positively correlated with the optimal solution and the query data is negatively correlated with the optimal solution if the constraint is satisfied.
4. The method of claim 1, the constraint comprising:
the sum of the resources allocated to all users is less than or equal to the maximum available resources of the target cluster;
the sum of the maximum query amounts which can be born by the resources distributed to all users in unit time is smaller than or equal to the maximum query amount which can be born by the target cluster in unit time;
for each user, the resource amount allocated to the user is greater than or equal to the minimum resource amount required by the user, and the minimum resource amount is determined according to the priority data and the query data corresponding to the user.
5. The method of claim 1, wherein the query data includes normal query time-consuming data and an abnormal query factor, the normal query time-consuming data being used to characterize average resource consumption of a user completing a single query without an abnormal query; the abnormal query factor is used to characterize the degree of negative impact of an abnormal query on a normal query.
6. The method of claim 2, wherein,
the linear function is:
the constraint conditions are as follows:
wherein z is used for representing the query quantity realized by the target cluster under the constraint condition, i is used for representing any user, n is used for representing the number of users, and x i For indicating the amount of resources allocated to user i, p i For representing the priority data corresponding to user i, t i For representing the normal query time-consuming data corresponding to user i, a i For representing the abnormal query factor corresponding to user i, l i The method comprises the steps of representing the minimum resource quantity required by a user i, and N represents the maximum query quantity which can be born by the target cluster in unit time; and the maximum available resource of the target cluster takes a value of 1.
7. The method of claim 1, further comprising:
Generating the resource allocation instruction according to a user request, wherein the user request is used for requesting the target cluster to allocate resources; and/or
And under the condition that the user and/or user service in the target cluster are detected to be changed, generating the resource allocation instruction, wherein each user corresponds to at least one user service, and the resources allocated to the user are influenced by the user service corresponding to the user.
8. An apparatus of resource allocation, comprising:
the first determining module is used for responding to the resource allocation instruction, determining priority data and query data corresponding to each user in the target cluster, wherein the priority data is used for representing the importance degree of the service corresponding to the user, and the query data is used for representing the influence of the user query on the resources in the target cluster;
the second determining module is used for determining a target planning relation of resource allocation based on priority data and query data corresponding to each user, wherein the target planning relation is used for determining a resource allocation scheme meeting constraint conditions, and the constraint conditions are used for representing conditions which should be met when the resources are allocated to each user in the target cluster;
A third determining module, configured to determine, according to the target planning relationship, an optimal resource allocation scheme corresponding to a target query volume, where the target query volume is used to characterize a maximum query volume that can be achieved by the target cluster when the constraint condition is satisfied;
and the resource allocation module is used for allocating resources to each user in the target cluster according to the optimal resource allocation scheme.
9. An electronic device, comprising:
a processor; and
a memory arranged to store computer executable instructions which when executed perform the method of resource allocation of any of claims 1 to 7 using the processor.
10. A computer readable medium storing one or more programs, which when executed by an electronic device comprising a plurality of application programs, cause the electronic device to perform the method of resource allocation of any of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310118053.2A CN116010450A (en) | 2023-02-02 | 2023-02-02 | Method, apparatus, electronic device and computer readable medium for resource allocation |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310118053.2A CN116010450A (en) | 2023-02-02 | 2023-02-02 | Method, apparatus, electronic device and computer readable medium for resource allocation |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116010450A true CN116010450A (en) | 2023-04-25 |
Family
ID=86033627
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310118053.2A Pending CN116010450A (en) | 2023-02-02 | 2023-02-02 | Method, apparatus, electronic device and computer readable medium for resource allocation |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116010450A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117217644A (en) * | 2023-11-06 | 2023-12-12 | 华清科盛(北京)信息技术有限公司 | Resource allocation method and device for logistics operation task |
-
2023
- 2023-02-02 CN CN202310118053.2A patent/CN116010450A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117217644A (en) * | 2023-11-06 | 2023-12-12 | 华清科盛(北京)信息技术有限公司 | Resource allocation method and device for logistics operation task |
| CN117217644B (en) * | 2023-11-06 | 2024-01-12 | 华清科盛(北京)信息技术有限公司 | Resource allocation method and device for logistics operation task |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107450979B (en) | Block chain consensus method and device | |
| US11734271B2 (en) | Data query method, apparatus and device | |
| WO2017016423A1 (en) | Real-time new data update method and device | |
| CN114157666A (en) | Load balancing system, method, medium and device based on intelligent cloud platform | |
| CN108243032B (en) | Method, device and equipment for acquiring service level information | |
| CN110659137B (en) | Processing resource allocation method and system for offline tasks | |
| CN111562884B (en) | Data storage method and device and electronic equipment | |
| CN116010450A (en) | Method, apparatus, electronic device and computer readable medium for resource allocation | |
| CN109118361B (en) | Method, device and system for managing limit | |
| US20190362005A1 (en) | Tune resource setting levels for query execution | |
| CN116820729A (en) | Offline task scheduling method and device and electronic equipment | |
| CN109245943B (en) | Fault positioning method and device | |
| CN115237960A (en) | Information pushing method and device, storage medium and electronic equipment | |
| CN114691175A (en) | Application updating method, device and system | |
| CN109522118B (en) | Service processing method, device, server and readable storage medium | |
| CN107369093B (en) | Service determination method and device | |
| CN113360565B (en) | Method and apparatus for flow control | |
| CN115801787B (en) | Road end data transmission method and device, electronic equipment and storage medium | |
| CN112346849A (en) | Method and device for configuring CPU | |
| CN116431327B (en) | Task current limiting processing method and fort machine | |
| CN110308996B (en) | Matching method for user URL request | |
| CN112016915B (en) | Method and device for providing service for user | |
| CN116186145A (en) | Method, device, system, equipment and medium for realizing database service isolation | |
| CN117827383A (en) | Database transaction processing method and device | |
| CN112882825A (en) | Method, device and equipment for allocating storage resources |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |