WO2022245423A1 - Multi-stage optimization algorithm for resource management in systems with predictable uncertainties - Google Patents

Abstract

The use of multiple horizon optimization for vehicle deployment and/or resource management is provided. Long horizon optimization for a time period is performed, and an optimal value function is determined. Data is received such as information about the vehicle(s) that are or can be deployed and/or the resource(s) that are or can be managed. Short horizon optimization for the time period is performed using a rollout algorithm, the optimal value function, and the received data. Aspects of the vehicle(s) and/or resource(s) may be adjusted using results of the short horizon optimization.

Description

MULTI-STAGE OPTIMIZATION ALGORITHM FOR RESOURCE MANAGEMENT IN SYSTEMS WITH PREDICTABLE UNCERTAINTIES CROSS-REFERENCE TO RELATED APPLICATIONS [0001] This application claims the benefit of U.S. provisional patent application No. 63/190,331, filed on May 19, 2021, and entitled “MULTI-STAGE OPTIMIZATION ALGORITHM FOR RESOURCE MANAGEMENT IN SYSTEMS WITH PREDICTABLE UNCERTAINTIES,” the disclosure of which is expressly incorporated herein by reference in its entirety. STATEMENT OF GOVERNMENT SUPPORT [0002] This invention was made with government under grant/contract number DE- AR0000794 awarded by the Department of Energy. The government has certain rights in the invention. BACKGROUND [0003] An objective of the nonlinear dynamic optimization problem, formulated in the spatial domain, is to optimize the use of resources such as vehicle deployment and energy consumption. [0004] It is with respect to these and other considerations that the various aspects and embodiments of the present disclosure are presented. SUMMARY [0005] The systems and methods described herein remove the drawbacks associated with previous systems and methods. Certain aspects of the present disclosure relate to vehicle deployment optimization and resource management optimization using multiple horizon optimization. [0006] In an implementation, a method of using multiple horizon optimization for fleet management and energy management of buildings comprises: performing long horizon optimization for a duration of time; determining an optimal value function based on the long horizon optimization; and performing short horizon optimization using the optimal value function as a terminal cost. [0007] In an implementation, a system for fleet management and energy management of buildings comprises: a long horizon optimization module configured to perform a long horizon optimization for a duration of time; a deterministic optimization module configured to determine an optimal value function; and a short horizon optimization module configured to perform a short horizon optimization using the optimal value function as a terminal cost. [0008] This summary is provided to introduce a selection of concepts in a simplified form that are further described below in the detailed description. This summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter. BRIEF DESCRIPTION OF THE DRAWINGS [0009] The foregoing summary, as well as the following detailed description of illustrative embodiments, is better understood when read in conjunction with the appended drawings. For the purpose of illustrating the embodiments, there is shown in the drawings example constructions of the embodiments; however, the embodiments are not limited to the specific methods and instrumentalities disclosed. In the drawings: [0010] FIG. 1 is a diagram of an implementation of a multiscale eco-optimization framework that comprises three cascaded layers; [0011] FIG. 2 is an illustration of an exemplary environment for systems and methods for vehicle deployment and/or resource management using multiple horizon optimization; [0012] FIG.3 is an operational flow of an implementation of a method of using multiple horizon optimization for vehicle deployment and/or resource management; [0013] FIG. 4 is an operational flow of another implementation of a method of using multiple horizon optimization for vehicle deployment and/or resource management; and [0014] FIG. 5 shows an exemplary computing environment in which example embodiments and aspects may be implemented. DETAILED DESCRIPTION [0015] The following description of the disclosure is provided as an enabling teaching of the disclosure in its best, currently known embodiment(s). To this end, those skilled in the relevant art will recognize and appreciate that many changes can be made to the various embodiments of the invention described herein, while still obtaining the beneficial results of the present disclosure. It will also be apparent that some of the desired benefits of the present disclosure can be obtained by selecting some of the features of the present disclosure without utilizing other features. Accordingly, those who work in the art will recognize that many modifications and adaptations to the present disclosure are possible and can even be desirable in certain circumstances and are a part of the present disclosure. Thus, the following description is provided as illustrative of the principles of the present disclosure and not in limitation thereof. [0016] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood to one of ordinary skill in the art to which this invention belongs. As used in the specification and claims, the singular form “a,” “an,” and “the” include plural references unless the context clearly dictates otherwise. As used herein, the terms “can,” “may,” “optionally,” “can optionally,” and “may optionally” are used interchangeably and are meant to include cases in which the condition occurs as well as cases in which the condition does not occur. Reference in the specification to “one embodiment” or “an embodiment” or “an example embodiment” means that a particular feature, structure, or characteristic described is included in at least one embodiment described herein and does not imply that the feature, structure, or characteristic is present in all embodiments described herein. Publications cited herein are hereby specifically incorporated by reference in their entireties and at least for the material for which they are cited. [0017] Ranges can be expressed herein as from “about” one particular value, and/or to “about” another particular value. When such a range is expressed, another embodiment includes from the one particular value and/or to the other particular value. Similarly, when values are expressed as approximations, by use of the antecedent “about,” it will be understood that the particular value forms another embodiment. It will be further understood that the endpoints of each of the ranges are significant both in relation to the other endpoint, and independently of the other endpoint. It is also understood that there are a number of values disclosed herein, and that each value is also herein disclosed as “about” that particular value in addition to the value itself. For example, if the value “10” is disclosed, then “about 10” is also disclosed. [0018] In some aspects, the present invention relates to systems and methods for vehicle deployment and/or resource management using multiple horizon (multi-horizon) optimization. [0019] In many engineering applications, the resource consumed are functions of uncertainties in the system that can be predicted beforehand, and the prediction becomes more and more accurate as the decision time nears. Examples of such predictable uncertainties include weather, customer arrival, energy consumption, renewable generation, traffic conditions, traffic lights, etc. A multi-stage optimization algorithm is described herein that can take into account various predictions to reduce the resource consumption of the system. [0020] Applications where such techniques may be used include: 1) optimization of fleets of vehicles providing ridesharing service, where assignments are made to reduce passenger waiting time and empty vehicle miles traveled; 2) optimization of electric vehicle charging scheduling, duration, and costs with or without using renewable energy; 3) optimization of energy consumption of residential and commercial buildings; 4) optimization of energy consumption in electrically assisted bicycles or personal mobility devices; 5) optimization of energy consumption of factories and chemical plants with substantial HVAC loads; 6) optimization of energy consumption of thermal management systems, including but not limited to climate control systems for residential buildings, commercial buildings, freight transportation and storage, air, sea and land vehicles, energy storage systems (e.g., battery backup units, UPS); 7) bio-adapted automated drug delivery in patients (insulin pump, etc.); 8) optimization of electric storage and thermal loads in critical infrastructures such as data centers; 9) optimization of operations of large wind turbines and wind farms; and 10) optimization of operations of locomotive. [0021] Systems and methods described herein comprise formulating eco-matching, eco-routing, and eco-driving as a unified multi-layer, multi-scale optimization. Aspects include the integration of models predicting vehicle energy consumption and environmental metrics into all optimization layers, and a solution method for the combined eco-matching/routing/driving problem where information on energy use computed by each layer is transferred consistently across different time scales to improve accuracy. [0022] Comprehensive eco-optimization techniques are described for high-capacity rideshare services (HCRS) that aim at jointly solving the problem as a unified, multi-layer dynamic optimization over multiple time scales. The transformational nature of the approach lies in integrating the energy-aware dispatch of vehicles, energy-saving route planning, and efficient speed advisory into a single solution, and exploiting synergies among these features to maximize the energy savings across the fleet, while maintaining high quality of service. [0023] FIG. 1 is a diagram of an implementation of a multiscale eco-optimization framework 100 that comprises three cascaded layers 110, 140, 170. The long timescale optimization is a Pre-Allocation for Eco-Matching (PEM) 110, namely a fleet-level optimization over an entire day of service that uses historical data for travel demands, traffic and other pertinent factors. Conventional algorithms for pre-booking services may perform this optimization to develop ride plans maximizing sharing and reducing vehicle miles travelled (VMT). Inputs include travel demand distribution and traffic patterns (historic data). Outputs include initial eco- allocation of vehicles for full day of service, initial eco-route plan. The value function includes optimal route plan for entire day of service. Implementations include physics-based models predicting energy consumption and environmental impact of the fleet (including conventional, hybrid and battery electric vehicles) in the optimization. This layer outputs a route plan for scheduled rideshare services. [0024] The medium timescale optimization (Dynamic Eco-Routing - DER) 140 dynamically updates vehicle assignments and routing on a shorter time horizon (minutes to hours) in response to current travel demand and traffic conditions. Aspects include integrating the energy consumption computed by the PEM 110 (long timescale) as a terminal cost in the DER 140 (medium timescale). Inputs include the value function from the PEM 110, current (real-time) travel demand data and traffic data. Outputs include real-time eco-matching and eco-routing of fleet vehicles. The value function includes optimal real-time routing data for each vehicle in the fleet. This feature, consistent with the concept of Rollout Algorithms in Approximate Dynamic Programming, considerably improves energy savings, when computed across the fleet over the course of a service. [0025] The short timescale optimization is a Dynamic Eco-Driving (DED) 170 utilizes the most recent route information assigned to each vehicle to perform a sub-optimization of the speed profile, accounting for the immediate state of traffic and traffic lights while en route. Inputs include the value function form the DER 140, enhanced route information, data from V2V and V2I communications. Outputs include real-time eco-driving (recommended velocity profile for individual driver). This technology can be adapted to an energy-efficient velocity advisor. [0026] As an example, the consumption of electrical energy for heating, ventilation, and air conditioning (HVAC) loads in residential and commercial buildings depend on various external and internal uncertainties. Most of these uncertainties can be predicted with some degree of confidence. The external uncertainties include solar irradiation; weather – temperature, humidity, wet bulb and dry bulb temperature; renewable generation; local grid or substation constraints; and time of use prices. The internal uncertainties include building occupancy; number of electric vehicles connected to the charger; cooking, baking, and/or other sources of heat within the building; and front door opening and closing due to arrival/departure of customers/employees. [0027] Building energy management systems and methods described herein can take the predictions of these uncertainties into account to minimize the total HVAC energy consumption of the building. A technique implemented by the systems and methods can be divided into multiple horizons – long horizon and short horizon. The long horizon optimization will take into account the predictions of uncertainties to compute the value function for the 24-48 hour period. As new information about these uncertainties is acquired through sensors, the short horizon optimization is run using the value function from the long horizon optimization as the terminal cost. [0028] FIG. 2 is an illustration of an exemplary environment 200 for systems and methods for vehicle deployment and/or resource management using multiple horizon optimization. The environment 200 comprises one or more vehicles 203 and/or one or more resources 206 that may be deployed and/or managed. The environment 200 further comprises a long horizon optimization module 210, a short horizon optimization module 220, an optimization algorithm (based on a backward induction method, such as Dynamic Programming (DP) or Stochastic Dynamic Programming (SDP)) 230, and a rollout algorithm 240. A processor 250 (such as comprised within a computing device, such as the computing device 500 described with respect to FIG. 5) may also be included in the environment 200. The processor 250 may perform some or all of the operations described further herein, depending on the implementation. [0029] The long horizon optimization module 210, the short horizon optimization module 220, the backward induction optimization algorithm 230, the rollout algorithm 240, and the processor 250 may each be implemented using a variety of computing devices. A suitable computing device is illustrated in FIG. 5 as the computing device 500. [0030] A multi-horizon optimization approach for vehicle deployment and/or resource management is described that takes advantage of connectivity and automation to forecast the impact of future conditions. As described further herein, it is contemplated that long horizon optimization may be performed at the beginning of a time period and/or during a time period, depending on the implementation. Thus, for example, long horizon optimization may be computed or re-computed during a 24-48 hour period. That information may be stored. [0031] FIG. 3 is an operational flow of an implementation of a method 300 of using multiple horizon optimization for vehicle deployment and/or resource management. Further aspects and details are also described with respect to FIG. 4 and below. The method 300 may be implemented in the environment 200. [0032] At 310, prior to the beginning of a time period (or at one or more times during the time period, depending on the implementation), long horizon optimization may be performed on initial values and data available at the time, such as information about the vehicles 203 that are or can be deployed, the resources 206 that are or can be managed, etc. In this manner, an optimal value function may be determined. The long horizon optimization may be performed by the long horizon optimization module 210. [0033] At 320, the value function may be stored, e.g., in memory associated with the environment 200 or one of the environment’s components (e.g., the long horizon optimization module 210, the short horizon optimization module 220, the backward induction optimization algorithm 230, the rollout algorithm 240, the processor 250).

[0034] At 330, during the time period (e.g., after the time period has begun), data from one or more of the vehicles 203 and/or resources 206 are received from one or more of the vehicles

203 and/or resources 206. The data may be received at the processor 250 or other computing device, depending on the implementation.

[0035] At 340, short horizon optimization is performed, e.g., by the short horizon optimization module 220, using the rollout algorithm 240 (which combines the short horizon optimization module 220 to the long horizon optimization module 210), the optimal value function, and the data from the vehicle(s) 203 and/or resource(s) 206. The short horizon optimization module 220 is solved using the backward induction optimization algorithm 230.

[0036] At 350, aspects of the vehicle(s) 203 and/or resource(s) 206 (e.g., operating parameters, features, etc.) during the time period may be adjusted using the results of the short horizon optimization.

[0037] In an implementation, for a N_H horizon problem starting at j^th grid point, the value function in memory is applied as the terminal cost. The stage cost (or running cost) of the short horizon optimization contains the same terms as that of the long horizon optimization. The constraints fed to this short horizon dynamic optimization contains updated information about uncertainties, which is obtained from sensors and cloud-based service providers. The rollout algorithm is then run backward from the {j + N_H}^th grid point until the j^th grid point. The resulting optimal policy is then applied at the j^th grid point to make the system transition to the {j + 1}^th grid point. At the {j + 1}^th grid point, the same procedure is used to solve the N_H horizon problem.

[0038] FIG. 4 is an operational flow of another implementation of a method 400 of using multiple horizon optimization for vehicle deployment and/or resource management. The method 400 may be implemented in the environment 200.

[0039] At 410, at the beginning of a time period (or at one or more times during the time period, depending on the implementation, such as a day or a 24-48 hour period for example), long horizon optimization is run, and a value function is computed for the entire time period (or for the remainder of the time period, or for one or more segments, portions, or routes of the time period, depending on the implementation). The long horizon optimization may be performed by the long horizon optimization module 210. Thus, an initial optimization is performed at the start of the time period (or at one or more times during the time period, depending on the implementation). Information regarding the vehicle(s) 203 and/or the resource(s) 206 are fed to the long horizon optimization using data from cloud-based service providers (e.g., TomTom, Waze, etc.) and intelligent digital maps containing, for example, speed limits and elevation data. The optimal value function is used to provide approximate optimized trajectories for vehicle deployments and resource allocation and management during the time period. The long horizon optimization problem can be solved using one or more of a plurality of backward induction numerical methods, such as DP or SDP, for example, although this is not intended to be limiting.

[0040] At 420, the value function may be stored, e.g., in memory associated with the environment 200 or one of the environment’s components (e.g., the long horizon optimization module 210, the short horizon optimization module 220, the backward induction optimization algorithm 230, the rollout algorithm 240, the processor 250).

[0041] If there is variability or uncertainty in information from the vehicle(s) 203 and/or resource(s) 206 that occur during the time period, the optimization will need to be re-run with the updated information to reflect these changes. For online use, it becomes computationally impractical to periodically perform the full time period (or remaining time period) optimization, considering the limited on-board computational and memory resources. This is a motivation to convert the long horizon problem to a short (e.g., receding) horizon optimal control problem, solved using the backward induction and the rollout algorithm.

[0042] At 430, the optimization is re-run for a N_H-long horizon, starting at j^th grid point along the time period (where N_H is significantly shorter than the remaining portion of the time period). The stage cost (or running cost) of the short horizon optimization contains the same terms as that of the long horizon optimization, and the value function in memory is applied as the terminal cost. The constraints fed to this short horizon dynamic optimization contains updated time period information, which is obtained from vehicle-to-infrastructure/vehicle-to-vehicle (V2I/V2V) modules, cloud-based service providers, and dedicated short range communication (DSRC) units.

[0043] At 440, the N_H horizon optimization problem is solved in a backward manner using the rollout algorithm, starting from the terminal stage ({j + N_H]^th grid point) until the initial stage (j^th grid point) in the current receding horizon.

[0044] At 450, the resulting optimal policy is then applied at the j^th grid point to make the system transition to the {j + 1]^th grid point. [0045] At 460, at the {j + 1}^th grid point, the same procedure is applied recursively to solve the N_H horizon problem.

[0046] Consider a dynamic optimization problem with a long horizon, as encountered in full time period optimization. Considering the limited on-board computational resources available and that various conditions experienced en route (during the time period) are variable, this long horizon optimization has to be transformed to a short horizon problem.

[0047] The rollout algorithm is a method of value space approximation that makes use of the cost-to-go of some known suboptimal/heuristic policy, referred to as the base policy or base heuristic. Under suitable assumptions, it can be shown that when the base heuristic produces a feasible solution, the rollout algorithm also produces a feasible solution, whose cost is no worse than the cost corresponding to the base heuristic. This is called cost improvement with rollout policy, a result that can be proved using mathematical induction. If the base policy is chosen as a reference DP policy, then cost improvement guarantees that the online solution implemented is no worse than this reference. For cost improvement to be valid, it is important that the base heuristic and the rollout policy are computed over the same constraint set. In the context of the real-time problems considered, cost improvement is of relevance as the rollout algorithm is inherently robust to parametric uncertainties and modeling errors experienced en route.

[0048] An advantage of the rollout algorithm developed is that the optimal solution is updated periodically in response to changing conditions. Further, the reduced computational effort to compute a short horizon optimization while achieving close-to-optimal results is appealing. The cost improvement property of the rollout algorithm guarantees the desired performance of the resulting solution, especially when it is applied to real-world problems containing uncertainties.

[0049] In such classical building energy management problems, the running cost is considered to be the same as that of the original optimization problem, and the terminal cost is typically a function that penalizes deviation from a reference operating condition. The choice of the terminal cost is somewhat arbitrary and for convenience, it may be constructed as an affine or quadratic function. Some calibration effort is required to ensure that the given system achieves the desired behavior with the chosen penalty function. In an implementation, the terminal cost is determined in a systematic manner and relies on the principles of ADP, specifically the rollout algorithm.

[0050] The rollout algorithm is a method of value space approximation that makes use of the cost-to-go of some known suboptimal/heuristic policy, referred to as the base policy or base heuristic. Under suitable assumptions, it can be shown that if the base heuristic produces a feasible solution, the rollout algorithm also produces a feasible solution, whose cost is no worse than the cost corresponding to the base heuristic. This is called cost improvement with rollout policy, a result that can be proved using mathematical induction. If the base policy is chosen as a reference DP policy, then cost improvement guarantees that the online solution implemented is no worse than this reference. [0051] An advantage of the rollout algorithm developed is that the optimal solution is updated periodically in response to changing weather conditions, renewable generation, local grid constraints, time of use prices, and other internal uncertainties listed above. Further, the reduced computational effort to compute a short horizon optimization while achieving close-to-optimal results is appealing. The cost improvement property of the rollout algorithm is an important result that guarantees the desired performance of the resulting solution, especially when it is applied to real-world problems containing uncertainties. Another advantage is that the proposed algorithm can still operate in temporary absence of information about the uncertainties. Further, the frequency of execution of the short horizon optimization can be adjusted based on computational needs and frequency of updates from sensors. [0052] FIG. 5 shows an exemplary computing environment in which example embodiments and aspects may be implemented. The computing device environment is only one example of a suitable computing environment and is not intended to suggest any limitation as to the scope of use or functionality. [0053] Numerous other general purpose or special purpose computing devices environments or configurations may be used. Examples of well known computing devices, environments, and/or configurations that may be suitable for use include, but are not limited to, personal computers, server computers, handheld or laptop devices, multiprocessor systems, microprocessor-based systems, network personal computers (PCs), minicomputers, mainframe computers, embedded systems, distributed computing environments that include any of the above systems or devices, and the like. [0054] Computer-executable instructions, such as program modules, being executed by a computer may be used. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. Distributed computing environments may be used where tasks are performed by remote processing devices that are linked through a communications network or other data transmission medium. In a distributed computing environment, program modules and other data may be located in both local and remote computer storage media including memory storage devices. [0055] With reference to FIG. 5, an exemplary system for implementing aspects described herein includes a computing device, such as computing device 500. In its most basic configuration, computing device 500 typically includes at least one processing unit 502 and memory 504. Depending on the exact configuration and type of computing device, memory 504 may be volatile (such as random access memory (RAM)), non-volatile (such as read-only memory (ROM), flash memory, etc.), or some combination of the two. This most basic configuration is illustrated in FIG. 5 by dashed line 506. [0056] Computing device 500 may have additional features/functionality. For example, computing device 500 may include additional storage (removable and/or non- removable) including, but not limited to, magnetic or optical disks or tape. Such additional storage is illustrated in FIG. 5 by removable storage 508 and non-removable storage 510. [0057] Computing device 500 typically includes a variety of computer readable media. Computer readable media can be any available media that can be accessed by the device 500 and includes both volatile and non-volatile media, removable and non-removable media. [0058] Computer storage media include volatile and non-volatile, and removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Memory 504, removable storage 508, and non-removable storage 510 are all examples of computer storage media. Computer storage media include, but are not limited to, RAM, ROM, electrically erasable program read-only memory (EEPROM), flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by computing device 500. Any such computer storage media may be part of computing device 500. [0059] Computing device 500 may contain communication connection(s) 512 that allow the device to communicate with other devices. Computing device 500 may also have input device(s) 514 such as a keyboard, mouse, pen, voice input device, touch input device, etc. Output device(s) 516 such as a display, speakers, printer, etc. may also be included. All these devices are well known in the art and need not be discussed at length here. [0060] In an implementation, a method of using multiple horizon optimization for fleet management and energy management of buildings comprises: performing long horizon optimization for a duration of time; determining an optimal value function based on the long horizon optimization; and performing short horizon optimization using the optimal value function as a terminal cost. [0061] Implementations may include some or all of the following features. Performing the short horizon optimization further uses a rollout algorithm. The rollout algorithm is a method of value space approximation that makes use of the cost-to-go of a suboptimal/heuristic policy. The suboptimal/heuristic policy is a base policy or a base heuristic. The method further comprises adjusting an operation of a vehicle using a result of the short horizon optimization. The method further comprises adjusting management of a resource using a result of the short horizon optimization. The method further comprises receiving data from one or more of vehicles or resources, wherein performing the short horizon optimization uses a rollout algorithm, the optimal value function, and the received data. The data is received after the duration of time has begun and during the duration of time. The long horizon optimization is performed at least one of prior to the beginning of the duration of time or during the duration of time. Performing long horizon optimization comprises using backward induction optimization. [0062] In an implementation, a system for fleet management and energy management of buildings comprises: a long horizon optimization module configured to perform a long horizon optimization for a duration of time; a deterministic optimization module configured to determine an optimal value function; and a short horizon optimization module configured to perform a short horizon optimization using the optimal value function as a terminal cost. [0063] Implementations may include some or all of the following features. Performing the short horizon optimization further uses a rollout algorithm. The rollout algorithm is a method of value space approximation that makes use of the cost-to-go of a suboptimal/heuristic policy. The suboptimal/heuristic policy is a base policy or a base heuristic. The system further comprises a processor configured to adjust an operation of a vehicle using a result of the short horizon optimization. The system further comprises a processor configured to adjust management of a resource using a result of the short horizon optimization. The system further comprises a processor configured to receive data from one or more of vehicles or resources, wherein performing the short horizon optimization uses a rollout algorithm, the optimal value function, and the received data. The data is received after the duration of time has begun and during the duration of time. The long horizon optimization is performed at least one of prior to the beginning of the duration of time or during the duration of time. Performing long horizon optimization comprises using backward induction optimization. [0064] It should be understood that the various techniques described herein may be implemented in connection with hardware components or software components or, where appropriate, with a combination of both. Illustrative types of hardware components that can be used include Field-Programmable Gate Arrays (FPGAs), Application-specific Integrated Circuits (ASICs), Application-specific Standard Products (ASSPs), System-on-a-chip systems (SOCs), Complex Programmable Logic Devices (CPLDs), etc. The methods and apparatus of the presently disclosed subject matter, or certain aspects or portions thereof, may take the form of program code (i.e., instructions) embodied in tangible media, such as floppy diskettes, CD-ROMs, hard drives, or any other machine-readable storage medium where, when the program code is loaded into and executed by a machine, such as a computer, the machine becomes an apparatus for practicing the presently disclosed subject matter. [0065] Although exemplary implementations may refer to utilizing aspects of the presently disclosed subject matter in the context of one or more stand-alone computer systems, the subject matter is not so limited, but rather may be implemented in connection with any computing environment, such as a network or distributed computing environment. Still further, aspects of the presently disclosed subject matter may be implemented in or across a plurality of processing chips or devices, and storage may similarly be effected across a plurality of devices. Such devices might include personal computers, network servers, and handheld devices, for example. [0066] Although the subject matter has been described in language specific to structural features and/or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.

Claims

What is claimed: 1. A method of using multiple horizon optimization for fleet management and energy management of buildings, the method comprising: performing long horizon optimization for a duration of time; determining an optimal value function based on the long horizon optimization; and performing short horizon optimization using the optimal value function as a terminal cost.

2. The method of claim 1, wherein performing the short horizon optimization further uses a rollout algorithm.

3. The method of claim 2, wherein the rollout algorithm is a method of value space approximation that makes use of the cost-to-go of a suboptimal/heuristic policy.

4. The method of claim 3, wherein the suboptimal/heuristic policy is a base policy or a base heuristic.

5. The method of claim 1, further comprising adjusting an operation of a vehicle using a result of the short horizon optimization.

6. The method of claim 1, further comprising adjusting management of a resource using a result of the short horizon optimization.

7. The method of claim 1, further comprising receiving data from one or more of vehicles or resources, wherein performing the short horizon optimization uses a rollout algorithm, the optimal value function, and the received data.

8. The method of claim 7, wherein the data is received after the duration of time has begun and during the duration of time.

9. The method of claim 1, wherein the long horizon optimization is performed at least one of prior to the beginning of the duration of time or during the duration of time.

10. The method of claim 1, wherein performing long horizon optimization comprises using backward induction optimization.

11. A system for fleet management and energy management of buildings, the system comprising: a long horizon optimization module configured to perform a long horizon optimization for a duration of time; a deterministic optimization module configured to determine an optimal value function; and a short horizon optimization module configured to perform a short horizon optimization using the optimal value function as a terminal cost.

12. The system of claim 11, wherein performing the short horizon optimization further uses a rollout algorithm.

13. The system of claim 12, wherein the rollout algorithm is a method of value space approximation that makes use of the cost-to-go of a suboptimal/heuristic policy.

14. The system of claim 13, wherein the suboptimal/heuristic policy is a base policy or a base heuristic.

15. The system of claim 11, further comprising a processor configured to adjust an operation of a vehicle using a result of the short horizon optimization.

16. The system of claim 11, further comprising a processor configured to adjust management of a resource using a result of the short horizon optimization.

17. The system of claim 11, further comprising a processor configured to receive data from one or more of vehicles or resources, wherein performing the short horizon optimization uses a rollout algorithm, the optimal value function, and the received data.

18. The system of claim 17, wherein the data is received after the duration of time has begun and during the duration of time.

19. The system of claim 11, wherein the long horizon optimization is performed at least one of prior to the beginning of the duration of time or during the duration of time.

20. The system of claim 11, wherein performing long horizon optimization comprises using backward induction optimization.