US7433856B2  Optimization with unknown objective function  Google Patents
Optimization with unknown objective function Download PDFInfo
 Publication number
 US7433856B2 US7433856B2 US10669722 US66972203A US7433856B2 US 7433856 B2 US7433856 B2 US 7433856B2 US 10669722 US10669722 US 10669722 US 66972203 A US66972203 A US 66972203A US 7433856 B2 US7433856 B2 US 7433856B2
 Authority
 US
 Grant status
 Grant
 Patent type
 Prior art keywords
 function
 ƒ
 utility
 method
 resources
 Prior art date
 Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
 Expired  Fee Related, expires
Links
Images
Classifications

 G—PHYSICS
 G06—COMPUTING; CALCULATING; COUNTING
 G06F—ELECTRICAL DIGITAL DATA PROCESSING
 G06F9/00—Arrangements for programme control, e.g. control unit
 G06F9/06—Arrangements for programme control, e.g. control unit using stored programme, i.e. using internal store of processing equipment to receive and retain programme
 G06F9/46—Multiprogramming arrangements
 G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
 G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
 G06F9/5011—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resources being hardware resources other than CPUs, Servers and Terminals
 G06F9/5016—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resources being hardware resources other than CPUs, Servers and Terminals the resource being the memory
Abstract
Description
1. Field of the Invention
The present invention generally relates to the field of nonlinear optimization and, more particularly, to resource allocation, as for example, buffer pool optimization in computer database software where only the marginal utility is known.
2. Background Description
Conventionally, resource allocation maximizing some utility function using nonlinear optimization requires knowledge of the utility as a function of the allocated resources. For example, the resources could be the memory allocated to a series of computer processors and the utility is the aggregate performance of these processors or the reduction in power consumption of these processors. In prior art methods for solving this problem, knowledge of this utility function is required. Almost all prior art algorithms for solving this problem can be classified into one of the following three classes:

 1. Only the utility function ƒ is known, i.e., at each point, there is a procedure to evaluate ƒ (x). Examples of such algorithms include the simplex method and genetic algorithms.
 2. Only the utility function ƒ and the gradient ∇ƒ are known. Examples include steepest descent, conjugate gradient and QuasiNewton methods.
 3. The utility function ƒ, its gradient ∇ƒ and its Hessian ∇^{2}ƒ are known. Examples include Newton's method.
Thus, in prior art algorithms, knowledge of the utility function is required in order to solve the optimization problem. In some applications, knowledge of this utility function is absent, and prior art iterative optimization algorithms cannot be used to solve this problem.
As a specific example, buffer pool optimization in International Business Machines' (IBM's) DB2 computer database software is a problem where only the gradient of the utility function is known, i.e., at each point we know the benefit in terms of saved disk access time due to allocating more memory, but the utility, which in this case is the total saved disk access time due to current buffer pool setting, is unknown.
Therefore, there is a need in the art for a method to allocate resources maximizing such utility using algorithms which do not need to know the utility as a function of the resources.
It is therefore an object of the present invention to provide a way to maximize the utility of an allocation of resources without the need to know the utility as a function of the allocated resources.
In a general form, the problem can be formulated mathematically as follows. The object is to maximize the utility function ƒ (x) subject to constraints on the vector x. Each vector x denotes a particular allocation of resources and the constraints generally describe properties of the resources which must be satisfied, for instance, the total number of resources must be less than a certain number, etc.
The present invention provides a method for allocating resources by specifying an allocation x which maximizes ƒ (x) while satisfying the constraints on x where the steps of the method do not require knowledge of ƒ. Sometimes the goal is to minimize some function ƒ in which case we refer to ƒ as a cost function rather than a utility function. In the nonlinear programming literature, ƒ is usually called an objective function.
In one embodiment of the present invention, a method for allocating resources (i.e., determining x) may comprise the steps of starting from an initial allocation, calculating the marginal utility of said allocation, calculating the constraint functions of said allocation, and applying this information to obtain a next allocation and these steps are repeated until a stopping criteria is satisfied, in which case a locally optimal allocation is returned.
According to the invention, we provide algorithms which are used to solve the above nonlinear programming problem where

 1. only the gradient ∇ƒ is known, or
 2. only the gradient ∇ƒ and the Hessian ∇^{2}ƒ are known, i.e., there is no procedure to evaluate the objective function ƒ. These two cases are optimization problems which have not been considered before. A simple example of this problem is when ƒ=∫g, where g is an explicit function whose integral cannot be written in closed form.
Our algorithms are derived from prior art optimization algorithms, but modified to not need evaluation of the objective function ƒ.
The foregoing and other objects, aspects and advantages will be better understood from the following detailed description of a preferred embodiment of the invention with reference to the drawings, in which:
Referring now to the drawings, and more particularly to
The present invention comprises a method, computer program product and system for resource allocation when the utility function is unknown and the marginal utility is known. The marginal utility with respect to a particular resource is the additional incremental utility obtained by incremental increase of that resource and is equivalent to the gradient of the utility function ƒ (x). Examples of the utility are total time saved, number of transactions processed, power consumption saved, processors utilization, memory utilization, etc.
All prior art iterative nonlinear optimization algorithms assumes knowledge of the utility function ƒ and if available, the gradient ∇ƒ, and the Hessian ∇^{2}ƒ.
In general, the problem is given as:
By replacing ƒ with −ƒ, the problem is written as a minimization problem which is the standard form in nonlinear programming:
In this case we will call ƒ the cost function rather than the utility function. Some examples of cost are total disk I/O (input/output) time, power consumption and overall system response time. The present invention provides a method for solving the minimization problem above without the need to evaluate ƒ. This is useful in applications where evaluating ƒ is impossible, very expensive, timeconsuming and/or inaccurate. An example of such ƒ is when ƒ=∫g, where g is a function whose integral cannot be written in closed form, e.g., g=e^{−x} ^{ 2 }.
Good text on nonlinear programming and numerical optimization include: Practical Optimization by P. Gill et al., Academic Press, 1981, Practical Methods of Optimization by R. Fletcher, Wiley & Sons, 1987, and Numerical Optimization by J. Nocedal and S. Wright, Springer, 1999.
We employ modification of line search based algorithms such as steepest descent, conjugate gradient, quasiNewton or Newton's method. In these line search based algorithms, almost all components of the algorithm do not require evaluation of ƒ. The stopping criteria based on first or second order optimaltality conditions also do not require evaluation of ƒ. The only place where evaluation of ƒ is required is in the scalar line search component where a onedimensional optimization is performed along a particular direction p. Our invention is to use a scalar line search method that does not require evaluation of ƒ. For instance, in prior art backtracking line search along a search direction p, the step size α is decreased until the sufficient decrease or Armijo condition is satisfied:
ƒ(x+αp)≦ƒ(x)+c _{1}α∇ƒ(x)^{T} p
where A^{T }denotes the transpose of the matrix (or vector) A. This requires evaluation of the function ƒ. The disclosed invention uses a backtracking line search that decreases the step a until the following condition is satisfied:
(∇ƒ(x+αp)−c _{1}∇ƒ(x))^{T} p≦0 (1)
This results in a line search which uses only gradient information. Using this line search with the rest of the components in a prior art line search type nonlinear programming method results in a nonlinear programming method which uses only gradient information, and if available, Hessian information, to minimize ƒ and does not require evaluation of ƒ itself.
This is illustrated in
In a preferred embodiment, the resources are a series of memory buffers in a computing system running the database software DB2 and the goal is to minimize access time by allocating memory to these buffers appropriately. This is illustrated in
The utility function ƒ (x) is the total disk time avoided by using the allocation vector x, whereas the partial derivative
is the time saved by allocation more resources to x_{i}. This is illustrated in
The gradient ∇ƒ consists of the partial derivatives of ƒ, and corresponds to the benefit in access time saved due to increased memory buffer allocation. The system configuration is such that ∇ƒ is readily available, whereas ƒ is not available (or requires too much effort to compute accurately).
By the KarushKuhnTucker optimaltality condition, at the optimal point x*=(x_{1}*, . . . , x_{n}*) where ƒ is maximized, the gradient ∇ƒ satisfies
∇ƒ(x*)_{i}=λ, if x_{i}*>d_{i }
∇ƒ(x*)_{i}≦λ, if x_{i}*=d_{i }
for some real number λ. To solve this problem where the inequality constraints are linear, which is a special case of the general problem, an active set line search algorithm can be used.
A flowchart of such an algorithm is shown in
Most line search based nonlinear optimization algorithms can be modified in the same manner by those skilled in the art to obtain a corresponding algorithm that does not require evaluation of ƒ. For example, for more general constraints, line search based methods such as penalty function methods, augmented Lagrangian methods, and sequential quadratic programming methods can be used, where the scalar line search component in these methods is replaced by the modified line search described above. Furthermore, since the utility function ƒ in
While the invention has been described in terms of a single preferred embodiment, those skilled in the art will recognize that the invention can be practiced with modification within the spirit and scope of the appended claims.
Claims (21)
(∇ƒ(x+αp)−c _{1}∇ƒ(x))^{T} p≦0,
Priority Applications (1)
Application Number  Priority Date  Filing Date  Title 

US10669722 US7433856B2 (en)  20030925  20030925  Optimization with unknown objective function 
Applications Claiming Priority (2)
Application Number  Priority Date  Filing Date  Title 

US10669722 US7433856B2 (en)  20030925  20030925  Optimization with unknown objective function 
US12173947 US8214840B2 (en)  20030925  20080716  Optimization with unknown objective function 
Publications (2)
Publication Number  Publication Date 

US20050080979A1 true US20050080979A1 (en)  20050414 
US7433856B2 true US7433856B2 (en)  20081007 
Family
ID=34421983
Family Applications (2)
Application Number  Title  Priority Date  Filing Date 

US10669722 Expired  Fee Related US7433856B2 (en)  20030925  20030925  Optimization with unknown objective function 
US12173947 Expired  Fee Related US8214840B2 (en)  20030925  20080716  Optimization with unknown objective function 
Family Applications After (1)
Application Number  Title  Priority Date  Filing Date 

US12173947 Expired  Fee Related US8214840B2 (en)  20030925  20080716  Optimization with unknown objective function 
Country Status (1)
Country  Link 

US (2)  US7433856B2 (en) 
Cited By (3)
Publication number  Priority date  Publication date  Assignee  Title 

US20070233991A1 (en) *  20060330  20071004  Yixin Diao  Method and apparatus for distributing memory in a data processing system 
US8250581B1 (en) *  20071028  20120821  HewlettPackard Development Company, L.P.  Allocating computer resources to candidate recipient computer workloads according to expected marginal utilities 
US9336516B2 (en)  20130814  20160510  International Business Machines Corporation  Scheduling for service projects via negotiation 
Families Citing this family (8)
Publication number  Priority date  Publication date  Assignee  Title 

US7555491B2 (en) *  20040114  20090630  Bmc Software, Inc.  Removing overflow rows in a relational database 
US7668788B2 (en) *  20050606  20100223  Wren William E  Resource assignment optimization using direct encoding and genetic algorithms 
US20070061429A1 (en) *  20050912  20070315  Microsoft Corporation  Optimizing utilization of application resources 
US7660773B1 (en) *  20051212  20100209  The Mathworks, Inc.  System and method for the use of an adaptive mutation operator in genetic algorithms 
US9208203B2 (en) *  20111005  20151208  Christopher M. Ré  Highspeed statistical processing in a database 
CN103425524B (en) *  20130717  20170208  北京邮电大学  A method of balancing the multiservice terminals and the polymerization system 
US20150100370A1 (en) *  20131008  20150409  Ford Global Technologies, Llc  Method and System for Generating Supplier Capacity Requirements 
US20160247100A1 (en) *  20131115  20160825  Hewlett Packard Enterprise Development Lp  Selecting and allocating 
NonPatent Citations (4)
Title 

California Scientific Software, Introduction to Neural Networks, 1991, California scientific software, 3rd. * 
Chandu Visweswariah, Optimization Techniques for HighPerfomance Digital Circuits, 1997, IEEE, 198205. * 
Paul Barford et al., On the Marginal Utility of Network Topology Measurements, 2001, ACM, 1581 13435501, 517. * 
Peter Grun et al., Memory Size Estimation for Multimedia Applications, 1997 NSF Grant MIP9708067. * 
Cited By (5)
Publication number  Priority date  Publication date  Assignee  Title 

US20070233991A1 (en) *  20060330  20071004  Yixin Diao  Method and apparatus for distributing memory in a data processing system 
US8122450B2 (en) *  20060330  20120221  International Business Machines Corporation  Method and apparatus for distributing memory in a data processing system 
US8250581B1 (en) *  20071028  20120821  HewlettPackard Development Company, L.P.  Allocating computer resources to candidate recipient computer workloads according to expected marginal utilities 
US9336516B2 (en)  20130814  20160510  International Business Machines Corporation  Scheduling for service projects via negotiation 
US9355388B2 (en)  20130814  20160531  International Business Machines Corporation  Scheduling for service projects via negotiation 
Also Published As
Publication number  Publication date  Type 

US20080276245A1 (en)  20081106  application 
US20050080979A1 (en)  20050414  application 
US8214840B2 (en)  20120703  grant 
Similar Documents
Publication  Publication Date  Title 

Meulman et al.  SPSS Categories 10.0  
Golany et al.  An application procedure for DEA  
Hamerly et al.  Simpoint 3.0: Faster and more flexible program phase analysis  
US6633882B1 (en)  Multidimensional database record compression utilizing optimized cluster models  
US5473773A (en)  Apparatus and method for managing a data processing system workload according to two or more distinct processing goals  
US6377907B1 (en)  System and method for collating UNIX performance metrics  
US6278989B1 (en)  Histogram construction using adaptive random sampling with crossvalidation for database systems  
US5734884A (en)  Database execution cost and system performance estimator  
US5881283A (en)  Job scheduling analysis method and system using historical job execution data  
US6256621B1 (en)  Database management system and query operation therefor, including processing plural database operation requests based on key range of hash code  
US6272389B1 (en)  Method and system for capacity allocation in an assembly environment  
Kazman et al.  Making architecture design decisions: An economic approach  
Kamakura et al.  Concomitant variable latent class models for conjoint analysis  
US6169983B1 (en)  Index merging for database systems  
US5831864A (en)  Design tools for highlevel synthesis of a lowpower data path  
Chen et al.  Computing efforts allocation for ordinal optimization and discrete event simulation  
US20050268063A1 (en)  Systems and methods for providing constrained optimization using adaptive regulatory control  
Brown et al.  Towards automated performance tuning for complex workloads  
US6732085B1 (en)  Method and system for sample size determination for database optimizers  
US20140215487A1 (en)  Optimizing execution and resource usage in large scale computing  
US6801903B2 (en)  Collecting statistics in a database system  
US5909669A (en)  System and method for generating a knowledge worker productivity assessment  
US6928398B1 (en)  System and method for building a time series model  
US7836058B2 (en)  Web searching  
US7421460B2 (en)  Method for determining execution of backup on a database 
Legal Events
Date  Code  Title  Description 

AS  Assignment 
Owner name: INTERNATIONAL BUSINESS MACHINES CORPORATION, NEW Y Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:WU, CHAI WAH;DIAO, YIXIN;REEL/FRAME:014552/0078 Effective date: 20030919 

REMI  Maintenance fee reminder mailed  
SULP  Surcharge for late payment  
FPAY  Fee payment 
Year of fee payment: 4 

REMI  Maintenance fee reminder mailed  
LAPS  Lapse for failure to pay maintenance fees  
FP  Expired due to failure to pay maintenance fee 
Effective date: 20161007 