CN116795521A - Method for optimizing resource allocation based on reinforcement learning predictions - Google Patents

Method for optimizing resource allocation based on reinforcement learning predictions Download PDF

Info

Publication number
CN116795521A
CN116795521A CN202210254510.6A CN202210254510A CN116795521A CN 116795521 A CN116795521 A CN 116795521A CN 202210254510 A CN202210254510 A CN 202210254510A CN 116795521 A CN116795521 A CN 116795521A
Authority
CN
China
Prior art keywords
ppn
poc
time
point
processor
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210254510.6A
Other languages
Chinese (zh)
Inventor
陈文贤
许铭杰
曾弘毅
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Xianzhi Yunduan Data Co ltd
Original Assignee
Xianzhi Yunduan Data Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Xianzhi Yunduan Data Co ltd filed Critical Xianzhi Yunduan Data Co ltd
Priority to CN202210254510.6A priority Critical patent/CN116795521A/en
Publication of CN116795521A publication Critical patent/CN116795521A/en
Pending legal-status Critical Current

Links

Landscapes

  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

The invention discloses a method for optimizing resource allocation according to reinforcement learning prediction, which comprises the following steps: a) Providing a prediction of the number of resource units of the workload exceeding N time points after the 0 th time point to the processor; b) Calculate at least one 0 th possible computation cost (POC) 0 ) It is based on the time point (PPN 1 ) At least one possible offered number (PPN); c) Repeating the following sub-steps by the processor for the i-th point in time: c1 Calculating at least one ith possible computation cost (POC) i ) The method comprises the steps of carrying out a first treatment on the surface of the c2 Finding the smallest and second small POC i The method comprises the steps of carrying out a first treatment on the surface of the And c 3) will be used to calculate the minimum POC i PPN of (c) i Set to the i-th specified number; and d) providing, by the processor, 1 unit of resources at the 0 th point in time and the i-th specified unit of resources to the workload at the i-th point in time.

Description

Method for optimizing resource allocation based on reinforcement learning predictions
Technical Field
The invention relates to a method for optimizing configuration. In particular, the present invention relates to a method for optimizing resource allocation based on reinforcement learning predictions.
Background
In a computer cluster, hardware resources are dynamically allocated to a workload. Since the resource requirements, such as RAM modules or CPUs, to be utilized by the workload can vary over time, there are many prediction methods that underestimate the amount of future. Although the cost of building a computer cluster is generally fixed, the operational costs including lifetime amortization and power consumption increase as the number of hardware resources changes. Significant changes in hardware resources from any predicted workload may not be economical from a total cost perspective. Without prediction, the adjustment of the amount of hardware resources is passive, resulting in a large response delay.
Therefore, to reduce the impact of the above problems, a more accurate prediction of workload is required. The Reinforcement Learning (RL) approach introduced in the past is to exploit the learning process to explore the action possibilities at any point in the future, making optimization decisions to meet the goals of certain tasks or games. When the RL method is used to predict workload, it leads to all possible path problems. That is, if the work load has the option of predicting m time points in the future using n resources at a time, the RL method must consider n based on a long-term learning itinerary m Resource deployment of the path. Obviously, the computational complexity of the decision process required to calculate the optimization results is exponential. The computational cost is another nuisance for the manager, as it requires more software development effort.
From a balance perspective, the total cost of the workload may be minimized if the amount of hardware resources is adjusted as little as possible, while the computational cost of the RL method is limited. However, the prior art does not provide a method for this function of computer clustering.
Disclosure of Invention
In order to solve the above problems, the present invention provides a method of optimizing resource allocation according to reinforcement learning predictions.
To achieve the above object, the present invention provides a method of optimizing resource allocation based on reinforcement learning predictions. The processor determines the number of resource units in the computer cluster to be utilized. The method comprises the following steps: a) Providing a prediction to the processor of the number of resource units required by the workload beyond N time points after the 0 th time point, wherein a maximum of M units of source, U, can be provided i N, M, i is a positive integer based on the number of units predicted to be required at the i-th time point; b) Calculating, by the processor, at least one 0 th possibilityCalculation cost (POC) 0 ) It is based on the fact that in U 1 To the 1 st time point in M (PPN 1 ) In (3) at least one possible offered amount (PPN) of POC 0 Obtained from POC 0 ＝K+RF×|PPN 1 –K|+PPN 1 Wherein RF is a rebalancing factor between 0 and 1, and K is a real number; c) The following sub-steps are repeated sequentially for the i-th time point by the processor, i being from 1 to N, wherein: c1 Calculating at least one ith possible computation cost (POC) i ) Wherein POC i Obtained from POC i ＝POC (i-1) +RF×|PPN (i+1) -PPN i |+PPN (i+1) Wherein POC (i-1) Is the possible calculation cost calculated for the (i-1) th point in time, PPN (i+1) Is at U (i+1) PPN to the (i+1) th point in time in M, PPN i Is at U i PPN to the ith point in M for calculating POC i And POC (physical layer of) (i-1) PPN of (c) i Have the same value; c2 Finding the smallest and second small POC i The method comprises the steps of carrying out a first treatment on the surface of the And c 3) if minimum and second small POCs i Is calculated from the same PPN i Will then be used to calculate the minimum POC i PPN of (c) i Set as the i-th specified quantity, remove the uncomputed POC from the calculated i-th specified quantity at the next point in time i The method comprises the steps of carrying out a first treatment on the surface of the And d) providing, by the processor, 1 unit of resources at the 0 th point in time and the i-th specified unit of resources to the workload at the i-th point in time.
In an embodiment of the invention, the method may further comprise the sub-steps of: c4 If not from the same PPN i Calculating minimum and second Small POCs i Then PPN is respectively used i Calculating POC (i+1) Will be used to calculate the minimum POC (i+1) PPN of (c) i Set as the i-th specified quantity, remove the uncomputed POC from the calculated i-th specified quantity at the next point in time i Wherein POC (i+1) Is the possible computational cost calculated for the (i+1) th point in time.
Preferably, the resources include memory modules, CPUs, I/O throughput, response time, requests per second, or latency.
The invention also providesAnother method of optimizing resource allocation based on reinforcement-learned predictions. The processor determines the number of resource units in the computer cluster to be utilized. The method comprises the following steps: a) Providing a prediction to the processor of the number of resource units required by the workload beyond N time points after the 0 th time point, wherein a maximum of M units of source, U, can be provided i N, M, i is a positive integer based on the number of units predicted to be required at the i-th time point; b) Calculating, by the processor, at least one 0 th possible computational cost (POC) 0 ) It is based on the fact that in U 1 To U (U) 1 Time point 1 in the smallest of +A and M (PPN 1 ) In (3) at least one possible offered amount (PPN) of POC 0 Obtained from POC 0 ＝K+RF×|PPN 1 –K|+PPN 1 Wherein RF is a rebalancing factor between 0 and 1, a is an integer, and K is a real number; c) The following sub-steps are repeated sequentially for the i-th time point by the processor, i being from 1 to N, wherein: c1 Calculating at least one ith possible computation cost (POC) i ) Wherein POC i Obtained from POC i ＝POC (i-1) +RF×|PPN (i+1) -PPN i |+PPN (i+1) Wherein POC (i-1) Is the possible calculation cost calculated for the (i-1) th point in time, PPN (i+1) Is at U (i+1) To U (U) (i+1) PPN, PPN at the (i+1) th point in time in the smallest of +a and M i Is at U i To U (U) i PPN at the ith time point in the smallest of +A and M for calculating POC i And POC (physical layer of) (i-1) PPN of (c) i Have the same value; c2 Finding the smallest and second small POC i The method comprises the steps of carrying out a first treatment on the surface of the And c 3) if minimum and second small POCs i Is calculated from the same PPN i Will then be used to calculate the minimum POC i PPN of (c) i Set as the i-th specified quantity, remove the uncomputed POC from the calculated i-th specified quantity at the next point in time i The method comprises the steps of carrying out a first treatment on the surface of the And d) providing, by the processor, 1 unit of resources at the 0 th point in time and the i-th specified unit of resources to the workload at the i-th point in time.
In one embodiment of the invention, the method may further comprise the sub-steps ofThe steps are as follows: c4 If not from the same PPN i Calculating minimum and second Small POCs i Then PPN is respectively used i Calculating POC (i+1) Will be used to calculate the minimum POC (i+1) PPN of (c) i Set as the i-th specified quantity, remove the uncomputed POC from the calculated i-th specified quantity at the next point in time i Wherein POC (i+1) Is the possible computational cost calculated for the (i+1) th point in time.
Preferably, the resources include memory modules, CPUs, I/O throughput, response time, requests per second, or latency.
Another method for optimizing resource allocation based on reinforcement learning predictions is implemented by a processor determining the number of units of resources in a cluster of computers to be used, comprising the steps of: a) Providing a prediction to the processor of the number of resource units required by the workload beyond N time points after the 0 th time point, wherein a maximum of M units of source, U, can be provided i N, M, i is a positive integer based on the number of units predicted to be required at the i-th time point; b) Calculating, by the processor, at least one 1 st possible computational cost (POC) 1 ) It is based on the fact that in U 1 To the 1 st time point in M (PPN 1 ) At least one possible offered quantity (PPN) and at U 2 To the 2 nd time point in M (PPN 2 ) Wherein POC is at least one PPN of 1 Obtained from POC 1 ＝K+RF×|PPN 1 -K|+PPN 1 +RF×|PPN 2 –PPN 1 |+PPN 2 Wherein RF is a rebalancing factor between 0 and 1, and K is a real number; c) Will be used to calculate the minimum POC 1 PPN of (c) 1 Set to the 1 st specified number; d) The following sub-steps are repeated sequentially for time points by the processor, i being from 2 to 2 x [ N/2]]Wherein: d1 Calculating at least one (i+1) th possible computation cost POC (i+1) Wherein POC (i+1) Obtained from POC (i+1) ＝POC (i-1) +RF×|PPN (i+1) -PPN i |+PPN (i+1) +wi, where Wi is RF×|PPN (i+2) –PPN( i+1) |+PPN (i+2) ，POC (i-1) Is the possible computational cost calculated for the (i-1) th point in time,PPN (i+2) is at U (i+2) PPN, PPN to the (i+2) th point in time in M (i+1) Is at U (i+1) PPN to the (i+1) th point in time in M, PPN i Is the PPN at the ith point in time in Ui through M, used to calculate POC (i+1) And POC (physical layer of) (i-1) PPN of (c) i Have the same value; wherein Wi is omitted from the calculation if (i+2) is greater than N; d2 Finding the smallest and second small POC i The method comprises the steps of carrying out a first treatment on the surface of the And d 3) if minimum and second small POCs i Calculated from the same PPN i Will then be used to calculate the minimum POC (i+1) PPN of (c) i Set to the i-th specified number and used to calculate the minimum POC (i+1) PPN of (c) (i+1) Set to the (i+1) th specified number, remove the uncomputed POC from the calculated i-th specified number at the next point in time (i+1) The method comprises the steps of carrying out a first treatment on the surface of the And e) providing, by the processor, 1 unit of resources at a 0 th point in time and a j-th specified unit of resources to the workload at a j-th point in time, where j is in 1 to N.
Preferably, the resources include memory modules, CPUs, I/O throughput, response time, requests per second, or latency.
Drawings
Fig. 1 shows a hardware architecture to which the present invention is applied.
Fig. 2 is a flow chart of a method of a first embodiment of the invention.
Fig. 3 shows the prediction results.
Fig. 4 is a look-up table.
Fig. 5 to 7 list the calculations for all time points.
Fig. 8 is a flow chart of an improved method of a second embodiment of the present invention.
Fig. 9 shows the same prediction results as fig. 3, with a different number of possible offerings.
Fig. 10 and 11 list the calculations for all time points.
Fig. 12 is a flow chart of a method of a third embodiment of the invention.
Fig. 13 shows the prediction result.
Fig. 14 is a look-up table.
Fig. 15 to 19 list the calculations for all time points.
Reference numerals illustrate: 10-computer clusters; 20-the internet; 30-client; 100-a calculation unit; 101-CPU;101 a-a processor; 101b-CPU; 102-a memory module; 103-a storage device; 120-network communication interface.
Detailed Description
The invention will now be described in more detail with reference to the following examples.
Please refer to fig. 1. It shows the hardware architecture of the application of the invention. The computer cluster 10 (e.g., a data center) has a plurality of computing units 100, each having a plurality of CPUs 101 and memory modules 102 (e.g., SDRAM), sharing data stored in storage devices 103 (e.g., hard disk). The computing units 100 may cooperatively support the operation of workloads via the internal data network 110. The computer cluster 10 further connects the client 30 to the internet 20 via the network communication interface 120. For example, the computing unit 100 provides streaming services to the client 30 as a workload. The number of CPUs 101 and memory modules 102 provided to the workload determines how quickly requests for the workload can be reacted. The CPU and memory modules are referred to in embodiments as resources. While more resources are used for the workload, which increases the performance of the workload, the overall system is also loaded. The lifetime of a resource is consumed and other workloads cannot share the same resource. Thus, the present invention is a method of optimizing resource allocation. The configuration is based on predictions of resource requirements, but modified by reinforcement learning. The method is implemented by a processor 101a (ASIC or one of the CPUs 100) in one of the computing units 100. The processor 101a determines the number of units of resources in the computer cluster 10 to be utilized as a result of the method. The predictions may be calculated and provided by another CPU 101b or input from an external system outside of the computer cluster 10.
Please refer to fig. 2. Which is a flow chart of a method of a first embodiment of the invention. The first step of the method is to provide a prediction to the processor of the number of resource units required for the workload beyond N time points after the 0 th time point, where a maximum of M units of origin, U, can be provided i Based on the number of units predicted to be required at the ith time point, N, M, i is all positiveNumber (S01). The prediction may be any method based on algorithms, formulas, experience, or even artificial intelligence, as long as there is a unit number of resources required for future workloads. Not all results of the prediction are used for the calculation according to the invention. Only a limited number N is used. N is a number defined by the user of the present invention. In this embodiment, N is set to 9. The prediction may provide resource requirements for more than N points in time. The additional data may be used for another stage of resource configuration computation. In this embodiment, M is exemplified by 6. In view of the values of M and N, the prediction results are shown in fig. 3 for the next operation. The solid dots represent the predicted unit numbers of resources (e.g., CPUs) at different points in time. For example, according to the prediction, the 3 rd time point would require 6 units of resources. In a broad sense, all controllable operation results of a system that can improve workload performance are considered a resource such as, but not limited to, I/O throughput, response time, requests per second, latency.
The second step of the method is to calculate at least one 0 th possible computation cost (POC) by the processor 0 ) It is based on the fact that in U 1 To the 1 st time point in M (PPN 1 ) In (3) at least one possible offered amount (PPN) of POC 0 Obtained from POC 0 ＝K+RF×|PPN 1 -K|+PPN 1 Where RF is a rebalancing factor between 0 and 1, and K is a real number (S02). This step is preliminary calculated by the processor 101 a. For clarity, please refer to fig. 5. It lists the calculations for a portion of the time points. Each calculation is divided into three matrices. The left matrix lists all PPNs at the previous point in time. The middle matrix is compared to fig. 4 and shows a portion of the calculation on the transition cost. The right matrix adds POC to each transition cost of the same column, showing the result of finding a specified number for the current point in time. The basic idea of the invention is to determine an appropriate unit number of resources for a point in time by taking into account the accumulated operational costs of all kinds of combinations of resources calculated from the previous point in time for providing to the workload, finding the minimum accumulated cost, using the possible number of offerings as a specified number of resources for utilization to the workload. The calculation of the next point in time is considered if necessary. The whole process is prolongedLogic extending from reinforcement learning (dynamic programming) uses similar concepts such as cost functions, maximum rewards (minimum cost), iterations. However, to reduce lengthy computation procedures, reasonable and unnecessary computations are omitted. PPN represents all the number of candidates at a point in time. PPN (PPN) i Is from the predicted quantity U i To any of the maximum numbers M that the system can provide. For example, at time 2, the predicted unit number of resources required is 3, and up to 6 units of resources can be utilized for the calculated PPN at time 1 and 2 2 Is 3, 4, 5, 6. If U i As with M, only one number is used. At any point in time, when a unit of resource is utilized, the lifetime of the resource is reduced and power consumption is reduced. To simplify and quantify the results, all subsequent calculations use a minimum cost, 1, as K. For ease of explanation, in all embodiments, K is set to 1. Of course, the number of resource changes between two subsequent points in time is also expensive, but not just one. Thus, the change is reduced by the rebalancing factor. RF varies with different kinds of resources. The RF is selected based on the characteristics of the resource. In this embodiment, 0.6 is taken as an example.
This is an exception to the calculation of time 0. Since there is no previous point in time, the start value should be specified as the accumulated cost. According to the invention, the number is "1". In the calculation formula, "|PPN 1 -1| "gets the amount of resource change between time 0 and 1, rf×|ppn 1 -1| "will|PPN 1 -a 40% decrease in 1| (rf=0.6), calculation of the 1 st time point yields two POC 0 :8.4 and 10. Note that in other examples, POC 0 The number of (2) may be greater than 2 or 1.
The third step of the method is to repeat the following sub-steps in sequence for the ith time point by the processor, i being from 1 to N (S03). This step is a calculation for all N time points. This substep is described in detail below.
The first sub-step is to calculate at least one ith possible computation cost (POC) i ) Wherein POC i Obtained from POC i ＝POC (i-1) +RF×|PPN (i+1) -PPN i |+PPN (i+1) Wherein POC (i-1) Is the possible calculation cost calculated for the (i-1) th point in time, PPN (i+1) Is at U (i+1) PPN to the (i+1) th point in time in M, PPN i Is at U i PPN to the ith point in M for calculating POC i And POC (physical layer of) (i-1) PPN of (c) i Has the same value (S03-1). For example, the 1 st time point is calculated. In FIG. 5, two POCs 0 "POC" as the first substep of the above formula (i-1) ". Here, PPN (i+1) Is PPN 2 Comprising 3, 4, 5, 6.PPN (PPN) i Is PPN 1 And 5 and 6 are as described above. To simplify "RF×|PPN (i+1) -PPN i The look-up table of FIG. 4 shows U considered from 1 to 6, operated on by I' i And U (i+1) M versus PPN (i+1) And PPN i Are maintained at 6 and rf is equal to 0.6. Since the right matrix only adds PPN to each transition cost of the same column, 8.4 is added to 4.2, 4.6, 5, 6.6, but not to 4.8, 5.2, 5.6, 6. Thus, the obtained list of POCs 1 (POC i ) 12.6, 13, 13.4, 15. Likewise, 10 is added to 4.8, 5.2, 5.6, 6, but not to 4.2, 4.6, 5, 6.6. Another column of POCs obtained 1 14.8, 15.2, 15.6, 16.PPN (PPN) 1 In "5" is used to calculate POC 0 "8.4" in (C) and POC 1 12.6, 13, 13.4, 15.
The second sub-step is to find the minimum and second small POCs i (S03-2). In FIG. 5, the minimum and second small POCs 1 12.6 and 13 respectively. All from the same PPN 1 But PPN 2 Different.
The third sub-step is if the minimum and second small POCs i Is calculated from the same PPN i Will then be used to calculate the minimum POC i PPN of (c) i Set as the i-th specified quantity, remove the uncomputed POC from the calculated i-th specified quantity at the next point in time i (S03-3). As indicated above, the minimum and second small POC 1 From the same PPN 1 Thus meeting the precondition for calculating the minimum POC i (12.6) PPN 1 5 (connected with 12.6 by arrow) and designated as the first1 specified number. Calculation of the next time point removes POCs not calculated from the 1 st specified quantity 5 (i.e., 14.8, 15.2, 15.6, 16) 1 . Only 12.6, 13, 13.4, 15 are left to time point 2. The same calculation is seen in fig. 5 to 7.
It should be noted that the calculations from steps S01 to S03 are only used for the resource configuration temporarily at the 1 st point in time, and nothing is utilized at this time. Calculations were prepared before time 1 and even before time 0.
The final step of the method is to provide 1 unit of resources at the 0 th point in time and the i-th specified unit of resources to the workload at the i-th point in time by the processor (S04). This step is a resource allocation step. The unit number of resources is 5, 6, 5, 6 for the 1 st time point to the 9 th time point according to the related data calculated in the previous step. The results differ from predictions that require 5, 2, 6, 1, 5, 6, 5, 2, 9 for time 1 to 9, respectively. The number of units of resource provision minimizes the cost of the operation.
Although the same calculation was performed for the 2 nd time point to the 9 th time point, there are two exceptions in different cases.
The first example is shown in fig. 6. The precondition for the third substep requires a minimum and a second small POC i Should be calculated from the same PPN i . However, when processing the calculation for the 4 th point in time, the minimum and second small POCs 4 Calculated from different PPNs 4 . That is, the minimum POC 4 (30.6) calculated from 5, and the second Small POC 4 (32) Calculated from 6. This means that it is difficult to know which PPN 4 There is a minimum computational cost for the 5 th time point. According to the invention, the solution for this is two PPNs calculated separately 4 For the 5 th time point, the 4 th specified number is determined based on the results of the two calculations. Thus, a fourth sub-step is required if not from the same PPN i Calculating minimum and second Small POCs i Then PPN is respectively used i Calculating POC (i+1) Will be used to calculate the minimum POC (i+1) PPN of (c) i Set to the i-th designated number from the next time pointThe ith specified number of computations of (c) removes the uncomputed POC i Wherein POC (i+1) Is the possible computational cost calculated for the (i+1) th point in time. In the example of FIG. 6, PPN 4 5 of (2) and related POC 4 (30.6 and 32.2) for calculation of time point 5, version 1; PPN (PPN) 4 6 of (2) and related POC 4 (32.2 and 32) for calculation of the 5 th time point, version 2. In this case, the two version display 5 is the calculated minimum POC 5 (34.4 and 35.4) PPN 4 . Then 5 rounds are set to the 4 th specified number. The remainder is the same as the third sub-step and is not repeated.
The second exception occurred at the calculation at time 9. Since the present method accepts only predictions for 9 time points, even the 10 th time point can be predicted, it is not used. Thus, PPN 10 Set to "0". The calculations are shown in fig. 7.
From the conventional reinforcement learning point of view, all data of the right matrix in fig. 5 to 7 should be calculated if any change to find the minimum cost is not lost. For example, the right matrix calculated at time 1 contains only 8 data, and 36 calculations should be made to obtain 36 data. For this time point, 28 computations are saved; from time 0 to time 9, the whole process saves 257 calculations; the actual situation has thousands of resource units and hundreds of time points; millions of computations can be saved. By means of the method, the exponential complexity can be reduced. Near optimal and acceptable results can be achieved in a limited time, greatly increasing availability and reducing computational costs.
In another modification, the calculation amount can be further reduced according to the spirit of the present invention. The improved method is shown in the second embodiment.
Please refer to fig. 8. Which is a flow chart of an improved method of a second embodiment of the present invention. To simplify the description of the second embodiment, the resource demand prediction of the first embodiment is still applied here. Furthermore, the second embodiment also applies to the architecture of fig. 1. The available source maximum unit M remains 6. The only difference between the method of the first embodiment and the modified method is the definition of PPN. PPN amount for each time point of this embodimentFurther reducing. PPN contains the number of units U required at the ith point in time i And other larger numbers. The larger number may be M-free. The modified method is shown in fig. 10 and 11, listing the calculations at all time points.
The first step of the improved method is to provide a prediction to the processor of the number of resource units required for the workload beyond N time points after the 0 th time point, where a maximum of M units of origin, U, can be provided i Is based on predicting the number of units required at the ith time point, N, M, i are all positive integers (S11). This step is the same as step S01. As described above, N and M remain the same to illustrate the second embodiment.
The second step of the improved method is to calculate at least one 0 th possible computation cost (POC) by the processor 0 ) It is based on the fact that in U 1 To U (U) 1 Time point 1 in the smallest of +A and M (PPN 1 ) In (3) at least one possible offered amount (PPN) of POC 0 Obtained from POC 0 ＝K+RF×|PPN 1 -K|+PPN 1 Where RF is a rebalancing factor between 0 and 1, A is an integer, and K can be any real number, where K is set to 1 (S12). RF was also set to 0.6. The difference between step S02 and step S12 is PPN 1 Limited by its upper limit. For better understanding of the differences, please refer to fig. 9. As in fig. 3, there are the same predictors, but different numbers of possible offerings. In this embodiment, A is 2. For each time point calculation, the bottom line is U i The upper limit is U 1 The smaller of +A or M. Calculation of time 0, U 1 +A is equal to 7 and M is 6. Thus, PPN because the most available resources have 6 units 1 Is 6. And cannot be 7. POC at time 0 since 1 was the smallest cost to be given initially 0 The calculated results of (2) are also 8.4 and 10. Please refer back to fig. 7. If for every ith point in time PPN i The upper limit of (2) is connected by a dotted line, PPN i The lower limit of (2) is connected by a solid line, a "tunnel" is formed. The specified number of resource units at all points in time falls into the tunnel.
The third step of the improvement method is to repeat the following sub-steps in sequence for the ith time point by the processor, i being from 1 to N (S13). It seems to be the same as step S03. However, the sub-steps are different. The following is a description of these sub-steps.
The first sub-step is to calculate at least one ith possible computation cost (POC) i ) Wherein POC i Obtained from POC i ＝POC (i-1) +RF×|PPN (i+1) -PPN i |+PPN (i+1) Wherein POC (i-1) Is the possible calculation cost calculated for the (i-1) th point in time, PPN (i+1) Is at U (i+1) To U (U) (i+1) PPN, PPN at the (i+1) th point in time in the smallest of +a and M i Is at U i To U (U) i PPN at the ith time point in the smallest of +A and M for calculating POC i And POC (physical layer of) (i-1) PPN of (c) i Has the same value (S13-1). Obviously, PPN (i+1) And PPN i The upper limit of (c) has changed. Calculation of the 1 st time Point, PPN 2 Comprising 3, 4 and 5.U (U) 2 The basis prediction is 3. Due to U 2 The smallest of +2 and 6 is U 2 +2 (5), so 6 is no longer used. Likewise, PPN 3 Only 6 are included. PPN (PPN) 4 Comprises 1, 2 and 3.PPN (PPN) 5 Comprising 5 and 6.PPN (PPN) 6 Comprising 2, 3 and 4.PPN (PPN) 7 Comprising 5 and 6.PPN (PPN) 8 Comprising 2, 3 and 4.PPN (PPN) 9 Containing 6. POC because PPN has different definitions i The calculation results of (2) vary. For example, in the calculation of time 1, 8.4 is added to 4.2, 4.6, 5, but not to 4.8, 5.2, 5.6. Thus, a list of POCs 1 (POC i ) 12.6, 13, 13.4. Likewise, 10 would be added to 4.8, 5.2, 5.6, but not to 4.2, 4.6, 5. Another column of POCs obtained 1 14.8, 15.2, 15.6.PPN (PPN) 1 In "5" is used to calculate POC 0 "8.4" in (C) and POC 1 13.4, 13, 13.6.
The second sub-step is to find the minimum and second small POCs i (S13-2). This step is identical to sub-step S03-2. In fig. 10, the minimum and second small POC 1 12.6 and 13 respectively. They are from the same PPN 1 But PPN 2 Different.
Third stepSubstep is if the minimum and second small POCs i Is calculated from the same PPN i Will then be used to calculate the minimum POC i PPN of (c) i Set as the i-th specified quantity, remove the uncomputed POC from the calculated i-th specified quantity at the next point in time i (S13-3). This step is identical to substep S03-3. Since the second and first sub-steps of the two methods are different, the subsequent results are also different. For example, using the same prediction, the 4 th specified number of the second embodiment is 3, and the 4 th specified number of the first embodiment is 5.
Likewise, the final step of the improved method is to provide 1 unit of resources at the 0 th point in time and the i-th specified unit of resources to the workload at the i-th point in time by the processor (S14). The same as in step S04, will not be described in detail.
As with the first embodiment, two exceptions are encountered with the improved method. In the second embodiment, no first exception is seen. However, other examples may occur. In this case, the following sub-steps are performed: if not from the same PPN i Calculating minimum and second Small POCs i Then PPN is respectively used i Calculating POC (i+1) Will be used to calculate the minimum POC (i+1) PPN of (c) i Set as the i-th specified quantity, remove the uncomputed POC from the calculated i-th specified quantity at the next point in time i Wherein POC (i+1) Is the possible computational cost calculated for the (i+1) th point in time.
The second exception also occurred in the calculation of the 9 th time point of the second embodiment. The same approach of the first embodiment is used to deal with the PPN here 10 。PPN 10 Set to "0". The calculations are shown in fig. 11.
The total number of calculations for the method of the first embodiment is 67 (version 2 does not occur in the second embodiment regardless of the calculation at the 5 th time point), while the modified method of the second embodiment is 41. The modified method of the second embodiment is less computationally intensive than the first embodiment. 26 calculations are saved. Although the number of resources provided at each point in time is not the same, there is no significant difference between them, and the same prediction is valid.
In the above embodiment, a specified number is obtained from the calculation of two time points. According to the invention, more than a specified number can be obtained from the calculation of two points in time. From another point of view, the previous embodiments collect data to determine the resources provided by a window "between two points in time. More empty windows are available for collecting data. The number of computations increases, but time is saved. Another method of optimizing resource allocation based on reinforcement-learned predictions is shown in the third embodiment.
Please refer to fig. 12. Which is a flow chart of a method of a third embodiment of the invention. In this embodiment, the resource demand prediction of the previous embodiment is still applied. Likewise, the third embodiment may also be used in the architecture of fig. 1. The available source maximum unit M remains 6. The method is illustrated in fig. 15 to 19, listing the calculations at all time points.
The first step of the method is to provide a prediction to the processor of the number of resource units required for the workload beyond N time points after the 0 th time point, where a maximum of M units of origin, U, can be provided i Is based on predicting the number of units required at the ith time point, N, M, i are all positive integers (S21). This step is the same as steps S01 and S11. As described above, N and M remain the same to illustrate the present embodiment.
The second step of the method is to calculate at least one 1 st possible computation cost (POC) by the processor 1 ) It is based on the fact that in U 1 To the 1 st time point in M (PPN 1 ) At least one possible offered quantity (PPN) and at U 2 To the 2 nd time point in M (PPN 2 ) Wherein POC is at least one PPN of 1 Obtained from POC 1 ＝K+RF×|PPN 1 -K|+PPN 1 +RF×|PPN 2 -PPN 1 |+PPN 2 Where RF is a rebalancing factor between 0 and 1, and K is a real number (S22). RF is also set to 0.6 and K is 1. For better understanding of the differences, please refer to fig. 13 and 15. The same prediction results are shown in fig. 13 and fig. 3. Fig. 14 contains two calculation tables. The table above lists RF×|PPN 1 -1|+PPN 1 +RF×|PPN 2 -PPN 1 |+PPN 2 While the following table shows POCs 1 ＝1+RF×|PPN 1 -1|+PPN 1 +RF×|PPN 2 -PPN 1 |+PPN 2 Is a function of the calculation of (1). "1" (not in PPN) 1 And POC (physical layer of) 1 Of (c) are the specified numbers in the table below. The initial cost is set to provide only one resource. Computing POC 1 Is similar to the formula of POC i But calculates the equation spanning three time points (two empty windows).
The third step of the method is to calculate the minimum POC 1 PPN of (c) 1 The 1 st designated number is set (S23). The specified number of functions is the same as the first embodiment. Due to POC 1 Minimum, PPN 1 The result may be booted and selected as the 1 st specified number.
The fourth step of the method is to repeat the following sub-steps sequentially for time points by the processor, i being an even number from 2 to 2 x [ N/2] (S24). The definition of "i" differs from the previous embodiments. First, "i" is an even number, such as 2, 4, 6 …, etc. Calculation under Gaussian notation [ N/2 ]. In this embodiment, N is 9, so [ N/2] is 8. That is, 2, 4, 6, 8 are taken as "i" of the calculations for the different iterations. These sub-steps are described below.
The first sub-step is to calculate at least one (i+1) th possible computation cost POC (i+1) Wherein POC (i+1) Obtained from POC (i+1) ＝POC (i-1) +RF×|PPN (i+1) -PPN i |+PPN (i+1) +wi, where Wi is RF×|PPN (i+2) -PPN (i+1) |+PPN (i+2) ，POC (i-1) Is the possible calculation cost calculated for the (i-1) th point in time, PPN (i+2) Is at U (i+2) PPN, PPN to the (i+2) th point in time in M (i+1) Is at U (i+1) PPN to the (i+1) th point in time in M, PPN i Is at U i PPN to the ith point in M for calculating POC (i+1) And POC (physical layer of) (i-1) PPN of (c) i Have the same value; wherein Wi is omitted from the calculation if (i+2) is greater than N (S24-1). For convenience, POC (i+1) ＝POC (i-1) +RF×|PPN (i+1) -PPN i |+PPN (i+1) +RF×|PPN (i+2) -PPN (i+1) |+PPN (i+2) See fig. 14, while rf=0.6. For example, let i=2. See fig. 16. Figure 16 lists the calculations at all time points. Here, POC 3 Calculated from POC 1 . In FIG. 15, POC 1 12.6, 13, 13.4, 15. Formula becomes POC 3 ＝POC 1 +RF×|PPN 3 -PPN 2 |+PPN 3 +RF×|PPN 4 -PPN 3 |+PPN 4 . The table above shows RF×|PPN 3 -PPN 2 |+PPN 3 +RF×|PPN 4 -PPN 3 |+PPN 4 As a result of (a). In FIG. 16, PPN 2 Is 3 to 6, PPN 3 Only 6, PPN 4 Is 1 to 6. Therefore, only 24 numbers were counted and displayed in the above table. POC (Point of care) 3 The results of (2) are in the bottom right of the table below.
The second sub-step is to find the minimum and second small POCs i (S24-2). This step is identical to sub-steps S03-2 and S13-2. In fig. 16, the minimum and second small POC 3 24 and 24.2, respectively. They are from different PPNs 2 。
The third sub-step is if the minimum and second small POCs i Calculated from the same PPN i Will then be used to calculate the minimum POC (i+1) PPN of (c) i Set to the i-th specified number and used to calculate the minimum POC (i+1) PPN of (c) (i+1) Set to the (i+1) th specified number, remove the uncomputed POC from the calculated i-th specified number at the next point in time (i+1) (S24-3). In this embodiment, although the smallest and second smallest POCs 3 Calculated from the same PPN 2 But the PPN used 2 Can be found by the same procedure of the second embodiment. Omitted here is not repeated. 24 is PPN 2 (dot background). Thus, the 2 nd specified number is 5, and the 3 rd specified number is 6 (dot background). Calculated POC for next iteration 3 24, 24.4, 24.8, 25.2, 25.6, 26. The calculation results for i=4 and i=6 are shown in fig. 17 and 18.
The different calculations occur when i is 8. Without PPN 10 . It matches (i+2) greater than N (10)>9) Is the case in (a). The Wi portion is omitted from the calculation. The calculation result is shown in fig. 19. Simplified formula becomes POC 9 ＝POC 7 +RF×|PPN 9 -PPN 8 |+PPN 9 . This formula is the same as in the previous embodiment. However, the 8 th and 9 th specified numbers are selected according to the same method. The 8 th specified number is 5 and the 9 th specified number is 6.
The final step of the method is to provide 1 unit of resources by the processor at the 0 th point in time and a j specified unit of resources to the workload at the j th point in time, where j is in 1 to N (S25). It is identical to steps S04 and S14, but the variable symbols are different.
The calculated total number of the method of the third embodiment is 137. The method of the third embodiment uses more computation than the first and second embodiments. Although the number of resources provided at each point in time may not be the same, but not much worse, the time to result may be reduced.
It should be noted that all of the above mathematical formulas are for illustration only and are not limiting the application of the present invention. Any other mathematical formula that can express the same calculus logic is also within the scope of the present invention.
While this invention has been described in terms of what is presently considered to be the most practical and preferred embodiment, it is to be understood that the invention is not necessarily limited to the disclosed embodiment. On the contrary, it is intended to cover various modifications and similar arrangements included within the spirit and scope of the claims, which should be accorded the broadest interpretation so as to encompass all such modifications and similar structures.

Claims (8)

1. A method for optimizing resource allocation based on reinforcement-learning predictions, implemented by a processor determining a unit number of resources in a cluster of computers to be used, comprising the steps of:
a) Providing a prediction to the processor of the number of resource units required by the workload beyond N time points after the 0 th time point, wherein a maximum of M units of source can be provided, U i N, M, i is a positive integer based on the number of units predicted to be required at the i-th time point;
b) Calculating, by the processor, at least one 0 th possible computation cost POC 0 It isAccording to U 1 To the 1 st time point PPN in M 1 In the amount PPN, wherein POC 0 Obtained from
POC 0 ＝K+RF×|PPN 1 –K|+PPN 1 ，
Wherein RF is a rebalancing factor between 0 and 1, and K is a real number;
c) The following sub-steps are repeated sequentially for the ith time point by the processor, i being from 1 to N:
c1 Calculating at least one ith possible computation cost (POC) i ) Wherein POC i Obtained from
POC i ＝POC (i-1) +RF×|PPN (i+1) -PPN i |+PPN (i+1) ，
Wherein POC (i-1) Is the possible calculation cost calculated for the (i-1) th point in time, PPN (i+1) Is at U (i+1) PPN to the (i+1) th point in time in M, PPN i Is at U i PPN to the ith point in M for calculating POC i And POC (physical layer of) (i-1) PPN of (c) i Have the same value;
c2 Finding the smallest and second small POC i The method comprises the steps of carrying out a first treatment on the surface of the A kind of electronic device with high-pressure air-conditioning system
c3 If minimum and second Small POC i Is calculated from the same PPN i Will then be used to calculate the minimum POC i PPN of (c) i Set as the i-th specified quantity, remove the uncomputed POC from the calculated i-th specified quantity at the next point in time i The method comprises the steps of carrying out a first treatment on the surface of the A kind of electronic device with high-pressure air-conditioning system
d) Providing, by the processor, 1 unit of resources at the 0 th point in time and the i-th specified number of units of resources to the workload at the i-th point in time.
2. The method of claim 1, further comprising the sub-steps of: c4 If not from the same PPN i Calculating minimum and second Small POCs i Then PPN is respectively used i Calculating POC (i+1) Will be used to calculate the minimum POC (i+1) PPN of (c) i Set as the i-th specified number, divide the calculated i-th specified number from the next time pointDe-calculated POC i Wherein POC (i+1) Is the possible computational cost calculated for the (i+1) th point in time.
3. The method of claim 1, wherein resources comprise memory modules, CPUs, I/O throughput, response time, requests per second, or latency.
4. A method for optimizing resource allocation based on reinforcement-learning predictions, implemented by a processor determining a unit number of resources in a cluster of computers to be used, comprising the steps of:
a) Providing a prediction to the processor of the number of resource units required by the workload beyond N time points after the 0 th time point, wherein a maximum of M units of source can be provided, U i N, M, i is a positive integer based on the number of units predicted to be required at the i-th time point;
b) Calculating, by the processor, at least one 0 th possible computation cost POC 0 It is based on the fact that in U 1 To U (U) 1 Point in time PPN 1 in the smallest of +a and M 1 In the amount PPN, wherein POC 0 Obtained from
POC 0 ＝K+RF|PPN 1 –K|+PPN 1 ，
Wherein RF is a rebalancing factor between 0 and 1, a is an integer, and K is a real number;
c) The following sub-steps are repeated sequentially for the ith time point by the processor, i being from 1 to N:
c1 At least one ith possible computation cost POC i Wherein POC i Obtained from
POC i ＝POC (i-1) +RF×|PPN (i+1) -PPN i |+PPN (i+1) ，
Wherein POC (i-1) Is the possible calculation cost calculated for the (i-1) th point in time, PPN (i+1) Is at U (i+1) To U (U) (i+1) PPN, PPN at the (i+1) th point in time in the smallest of +a and M i Is at U i To U (U) i PPN at the ith time point in the smallest of +A and M for calculating POC i And POC (physical layer of) (i-1) PPN of (c) i Have the same value;
c2 Finding the smallest and second small POC i The method comprises the steps of carrying out a first treatment on the surface of the A kind of electronic device with high-pressure air-conditioning system
c3 If minimum and second Small POC i Is calculated from the same PPN i Will then be used to calculate the minimum POC i PPN of (c) i Set as the i-th specified quantity, remove the uncomputed POC from the calculated i-th specified quantity at the next point in time i The method comprises the steps of carrying out a first treatment on the surface of the A kind of electronic device with high-pressure air-conditioning system
d) Providing, by the processor, 1 unit of resources at the 0 th point in time and the i-th specified number of units of resources to the workload at the i-th point in time.
5. The method of claim 4, further comprising the sub-steps of: c4 If not from the same PPN i Calculating minimum and second Small POCs i Then PPN is respectively used i Calculating POC (i+1) Will be used to calculate the minimum POC (i+1) PPN of (c) i Set as the i-th specified quantity, remove the uncomputed POC from the calculated i-th specified quantity at the next point in time i Wherein POC (i+1) Is the possible computational cost calculated for the (i+1) th point in time.
6. The method of claim 4, wherein resources comprise memory modules, CPUs, I/O throughput, response times, requests per second, or delays.
7. A method for optimizing resource allocation based on reinforcement-learning predictions, implemented by a processor determining a unit number of resources in a cluster of computers to be used, comprising the steps of:
a) Providing a prediction to the processor of the number of resource units required by the workload beyond N time points after the 0 th time point, wherein a maximum of M units of source can be provided, U i Is based on the prediction at the ith time pointThe required unit numbers, N, M, i, are positive integers;
b) Calculating, by the processor, at least one 1 st possible computation cost POC 1 It is based on the fact that in U 1 To the 1 st time point PPN in M 1 In the order of at least one possible offered quantity PPN and at U 2 To the 2 nd time point PPN in M 2 Wherein POC is at least one PPN of 1 Obtained from
POC 1 ＝K+RF×|PPN 1 -K|+PPN 1 +RF×|PPN 2 –PPN 1 |+PPN 2 ，
Wherein RF is a rebalancing factor between 0 and 1, and K is a real number;
c) Will be used to calculate the minimum POC 1 PPN of (c) 1 Set to the 1 st specified number;
d) The following sub-steps are repeated sequentially for time points by the processor, i being an even number from 2 to 2 x [ N/2 ]:
d1 Calculating at least one (i+1) th possible computation cost POC (i+1) Wherein POC (i+1) Obtained from
POC (i+1) ＝POC (i-1) +RF×|PPN (i+1) -PPN i |+PPN (i+1) +Wi，
Wherein Wi is RF×|PPN (i+2) –PPN (i+1) |+PPN (i+2) ，POC (i-1) Is the possible calculation cost calculated for the (i-1) th point in time, PPN (i+2) Is at U (i+2) PPN, PPN to the (i+2) th point in time in M (i+1) Is at U (i+1) PPN to the (i+1) th point in time in M, PPN i Is at U i PPN to the ith point in M for calculating POC (i+1) And POC (physical layer of) (i-1) PPN of (c) i Have the same value; wherein Wi is omitted from the calculation if (i+2) is greater than N;
d2 Finding the smallest and second small POC i The method comprises the steps of carrying out a first treatment on the surface of the A kind of electronic device with high-pressure air-conditioning system
d3 If minimum and second Small POC i Calculated from the same PPN i Will then be used to calculate the minimum POC (i+1) PPN of (c) i Set to the i-th specified number and used to calculate the minimum POC (i+1) PPN of (c) (i+1) Set to the (i+1) th specified number, remove the uncomputed POC from the calculated i-th specified number at the next point in time (i+1) The method comprises the steps of carrying out a first treatment on the surface of the A kind of electronic device with high-pressure air-conditioning system
e) Providing, by the processor, 1 unit of resources at a 0 th point in time and a j-th specified number of units of resources to the workload at a j-th point in time, where j is in 1 to N.
8. The method of claim 7, wherein resources comprise memory modules, CPUs, I/O throughput, response times, requests per second, or delays.
CN202210254510.6A 2022-03-15 2022-03-15 Method for optimizing resource allocation based on reinforcement learning predictions Pending CN116795521A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210254510.6A CN116795521A (en) 2022-03-15 2022-03-15 Method for optimizing resource allocation based on reinforcement learning predictions

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210254510.6A CN116795521A (en) 2022-03-15 2022-03-15 Method for optimizing resource allocation based on reinforcement learning predictions

Publications (1)

Publication Number Publication Date
CN116795521A true CN116795521A (en) 2023-09-22

Family

ID=88038252

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210254510.6A Pending CN116795521A (en) 2022-03-15 2022-03-15 Method for optimizing resource allocation based on reinforcement learning predictions

Country Status (1)

Country Link
CN (1) CN116795521A (en)

Similar Documents

Publication Publication Date Title
US20230267330A1 (en) Parameter-Efficient Multi-Task and Transfer Learning
Vengerov A reinforcement learning approach to dynamic resource allocation
Verbelen et al. Graph partitioning algorithms for optimizing software deployment in mobile cloud computing
EP3616134A1 (en) Systems and methods for improved optimization of machine-learned models
JP5121936B2 (en) RESOURCE ALLOCATION DEVICE, RESOURCE ALLOCATION PROGRAM, RECORDING MEDIUM, AND RESOURCE ALLOCATION METHOD
CN109144719B (en) Collaborative unloading method based on Markov decision process in mobile cloud computing system
US9959248B1 (en) Iterative simple linear regression coefficient calculation for big data using components
Kim et al. Prediction based sub-task offloading in mobile edge computing
CN112181664B (en) Load balancing method and device, computer readable storage medium and electronic equipment
Głąbowski Modelling of state-dependent multirate systems carrying BPP traffic
Jalalian et al. A hierarchical multi-objective task scheduling approach for fast big data processing
Efrosinin et al. An algorithmic approach to analysing the reliability of a controllable unreliable queue with two heterogeneous servers
US20220116823A1 (en) Apparatus and method for dynamic resource allocation in cloud radio access networks
CN116795521A (en) Method for optimizing resource allocation based on reinforcement learning predictions
Stepanov et al. Approximate Method for Estimating Characteristics of Joint Service of Real-Time Traffic and Elastic Data Traffic in Multiservice Access Nodes
TWI805257B (en) Method for optimizing resource allocation based on prediction with reinforcement learning
CN116541106A (en) Computing task unloading method, computing device and storage medium
US20230267006A1 (en) Method for optimizing resource allocation basedon prediction with reinforcement learning
CN116302507A (en) Application service dynamic deployment and update method based on vacation queuing
CN114020469A (en) Edge node-based multi-task learning method, device, medium and equipment
Alyatama et al. Continuous memory allocation model for cloud services
Ferreira et al. Moments of losses during busy-periods of regular and nonpreemptive oscillating M^ X/G/1/n MX/G/1/n systems
Alyatama Pricing and quantization of memory for cloud services with uniform request distribution
CN112118135A (en) Minimum resource configuration method and device for cloud edge cooperative architecture industrial internet platform
Litoiu et al. Object allocation for distributed applications with complex workloads

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
REG Reference to a national code

Ref country code: HK

Ref legal event code: DE

Ref document number: 40091819

Country of ref document: HK