WO2009111799A2 - Nuage d'informatique utilitaire distribué mondialement - Google Patents

Nuage d'informatique utilitaire distribué mondialement Download PDF

Info

Publication number
WO2009111799A2
WO2009111799A2 PCT/US2009/036579 US2009036579W WO2009111799A2 WO 2009111799 A2 WO2009111799 A2 WO 2009111799A2 US 2009036579 W US2009036579 W US 2009036579W WO 2009111799 A2 WO2009111799 A2 WO 2009111799A2
Authority
WO
WIPO (PCT)
Prior art keywords
instance
virtual
server grid
distributed application
virtual appliance
Prior art date
Application number
PCT/US2009/036579
Other languages
English (en)
Other versions
WO2009111799A3 (fr
Inventor
Peter Nickolov
Bert Armijo
Vladimir Miloushev
Original Assignee
3Tera, Inc.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 3Tera, Inc. filed Critical 3Tera, Inc.
Publication of WO2009111799A2 publication Critical patent/WO2009111799A2/fr
Publication of WO2009111799A3 publication Critical patent/WO2009111799A3/fr

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5061—Partitioning or combining of resources
    • G06F9/5072—Grid computing
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/10—Protocols in which an application is distributed across nodes in the network

Definitions

  • FIGURE 1 shows an example embodiment of a portion of an AppLogic(TM) Grid
  • FIGURE IA illustrates an example embodiment of a server system 80 which may be used for implementing various aspects/features described herein.
  • Figure IB shows an example embodiment of a global network portion 99 which may be used for implementing various aspects described herein.
  • FIG. 2A shows an example embodiment of a Cloudware System 200 which, for example, may be used to provide various types of Cloudware-related functionality described herein.
  • Figure 2B illustrates an example embodiment of a Cloudware Portal home page 290.
  • Figure 2C illustrates another example embodiment of a Cloudware Portal home page
  • FIG. 3 shows an example embodiment of a graphical user interface (GUI) 300 which may be used for implementing various Cloudware related aspects/features.
  • GUI graphical user interface
  • FIG. 4 shows an example embodiment of another graphical user interface (GUI) 400 which may be used for implementing various Cloudware related aspects/features.
  • GUI graphical user interface
  • FIG. 5 shows an example embodiment of another graphical user interface (GUI) 500 which may be used for implementing various Cloudware related aspects/features.
  • GUI graphical user interface
  • FIG. 6 shows an example embodiment of another graphical user interface (GUI) 600 which may be used for implementing various Cloudware related aspects/features.
  • Figure 7 shows an example embodiment of a graphical user interface (GUI) 700 which may be used for implementing various Cloudware related aspects/features.
  • FIG. 8 shows an example embodiment of graphical user interface (GUI) 800 which may be used for implementing various Cloudware related aspects/features.
  • GUI graphical user interface
  • FIG. 9 shows an example embodiment of graphical user interface (GUI) 900 which may be used for implementing various Cloudware related aspects/features.
  • GUI graphical user interface
  • FIG. 10 shows an example embodiment of graphical user interface (GUI) 1000 which may be used for implementing various Cloudware related aspects/features.
  • GUI graphical user interface
  • FIG 11 shows an example embodiment of graphical user interface (GUI) 1100 which may be used for implementing various Cloudware related aspects/features.
  • Figure 12 shows an example embodiment of graphical user interface (GUI) 1200 which may be used for implementing various Cloudware related aspects/features.
  • FIG. 13 shows an example embodiment of graphical user interface (GUI) 1300 which may be used for implementing various Cloudware related aspects/features.
  • Figure 14 shows an example embodiment of a graphical user interface (GUI) 1400 which may be used for implementing various Cloudware related aspects/features.
  • Figure 15 shows a flow diagram illustrating various information flows and processes which may occur at or between various entities of the Cloudware network.
  • Figures 16-17 illustrate example embodiments of various types of Cloudware metering features and interfaces.
  • Figure 18 shows an example embodiment of a geographically distributed cloud computing network 1800.
  • Figure 19 shows an example embodiment of an interaction diagram illustrating an example of a distributed application migration procedure between two geographically distributed server grids.
  • Figures 20-29B illustrate various example embodiments of different graphical user interfaces (GUIs) and/or virtualized components which may be utilized, for example, for enabling, accessing and/or implementing various types of global utility computing features and/or information described herein.
  • Figure 30 illustrates an example embodiment of a virtual machine manager.
  • Figure 31 illustrates an example embodiment of a virtual network interface.
  • Figure 32 illustrates an example embodiment of a virtual appliance.
  • Figure 33 illustrates an example embodiment of an instantiation of an application which has been implemented using a plurality of different virtual appliances.
  • Figure 34 illustrates an example embodiment of a property mechanism for virtual appliances.
  • Figure 35 illustrates an example embodiment of a composite virtual appliance.
  • Figure 36 illustrates an example embodiment of a structure of a distributed application.
  • Figure 37 illustrates an example embodiment of a user interface for defining virtual appliances.
  • Figure 38 illustrates an example embodiment of a user interface for application monitoring.
  • Figure 39 illustrates an example embodiment of a portion of a system architecture which may be used for implementing various aspects described herein.
  • Figures 40-81 illustrate various example embodiments of different graphical user interfaces (GUIs) and/or virtualized components which may be utilized, for example, for enabling, accessing and/or implementing various types of global utility computing features and/or information described herein.
  • Figure 82 shows an example embodiment of a Cloudware-enabled global network
  • a Cloudware network may be implemented as a unified, globally distributed computer system having functionality similar to a mainframe computing system.
  • all or selected portions of the Cloudware network may be configured or designed to function as a globally distributed mainframe computing cloud, wherein the user or client computer systems may be operable as individual terminals for providing interfaces with the mainframe computing cloud.
  • all or selected resources associated with selected computing grids may be aggregated, shared, and/or combined, and collectively represented (e.g., to end users) as a single entity which represents a virtual, globally distributed computing system (or computing cloud).
  • various embodiments of the Cloudware network may be operable to provide services for running various types desktop computer software, such as, for example, desktop computer operating system software (e.g,. Linux, MS Windows, MAC OS, Solaris, etc.), desktop computer applications, etc.
  • various aspects described herein are directed to different methods, systems, and computer program products for use in computing networks such as, for example, on-demand, grid and/or utility computing networks.
  • Examples of at least a portion of the techniques (and/or related features, aspects, and/or benefits) disclosed herein include: techniques for migrating virtual appliances from a first server grid to a second server grid via a communication network; techniques for migrating distributed applications from a first server grid to a second server grid via a communication network; techniques for delivering pre-packaged software in virtual appliances to computing systems for use in operating software applications; techniques for managing use of virtualized computing resources implemented in a computing network; exchange systems for renting or leasing computing resources provided over a computing network; techniques for offering, via a computing network, virtualized computing resources for use in deployment of one or more distributed applications at one or more server grids of a computing network; techniques for offering, via a computing network, distributed application components for use in deployment of one or more distributed applications at one or more server grids of a computing network; techniques for implementing
  • different embodiments of computing networks may include multiple different data centers and/or server grids which are deployed different geographic locations.
  • at least some of the server grids may be operable to provide on-demand, grid and/or utility computing resources for hosting various types of distributed applications.
  • a distributed application may be characterized as an application made up of distinct components (e.g., virtual appliances, virtual machines, virtual interfaces, virtual volumes, virtual network connections, etc.) in separate runtime environments.
  • different ones of the distinct components of the distributed application may be hosted or deployed on different platforms (e.g., different servers) connected via a network.
  • a distributed application may be characterized as an application that runs on two or more networked computers.
  • Devices that are in communication with each other need not be in continuous communication with each other, unless expressly specified otherwise.
  • devices that are in communication with each other may communicate directly or indirectly through one or more intermediaries.
  • AppLogic(TM) may be implemented as a grid operating system designed to enable utility computing for web applications. AppLogic(TM) may run distributed transactional and streaming applications on grids of commodity hardware. It does not require a SAN or other expensive shared storage, and may be open and vendor- neutral. Additionally, AppLogic(TM) may be completely compatible with existing web applications.
  • grid computing has been limited to running computational applications such as business intelligence, simulations, derivatives trading, etc.
  • computational applications such as business intelligence, simulations, derivatives trading, etc.
  • the vast majority of Internet services and business applications are not computational; instead, they process large numbers of relatively small concurrent transactions (transactional applications) and/or deliver content (I/O intensive applications).
  • AppLogic(TM) may be implemented as a grid operating system that may be designed for web applications and may be optimized for transactional and I/O intensive workloads. It uses advanced virtualization technologies to ensure complete compatibility with existing operating systems, middleware and applications. As a result, AppLogic(TM) makes it easy to move existing web applications onto a grid without modifications.
  • Figure 1 illustrates an example architecture of a portion of an AppLogic(TM) Grid
  • the system may run on a hardware grid assembled from commodity servers connected via Gigabit Ethernet interconnect, for example. Some (or all or selected) of the servers may include directly attached storage (such as, for example, inexpensive IDE/ATA/SATA hard drives) which AppLogic(TM) may use to provide a distributed storage pool for applications.
  • the AppLogic(TM) Grid Operating System may include one or more of the following subsystems (or combinations thereof): • Distributed Kernel - abstracts and virtualizes the grid hardware and provides system services.
  • Disposable Infrastructure Manager handles the infrastructure for each AppLogic(TM) application.
  • Grid Controller - provides a central point for managing and monitoring the grid.
  • Such subsystems provide a foundation for executing and scaling existing web applications on grids of commodity servers.
  • AppLogic(TM) may use virtualization to enable each disposable infrastructure component to run on its own copy of one or more selected operating systems, and focuses on providing the abstractions and services needed at the distributed application level. This approach results in a system that may be very robust, while capable of integrating and running existing software unchanged.
  • AppLogic(TM) may manage and/or use disposable infrastructure in order to provide, within the application, the necessary infrastructure which may be preferred to run a given application. For example, in one embodiment, whenever a given application is started, the system may automatically manufacture and assemble the infrastructure required to run it. Once the application is stopped, AppLogic(TM) may dispose of all or selected infrastructure associated with it. This dramatically simplifies both the construction and the operation of N-tier applications: building infrastructure for each individual application may be much simpler than building and managing shared infrastructure. More importantly, including the infrastructure within each application makes applications self- contained and portable and enables AppLogic(TM) to instantiate them on demand and migrate them from one grid to another.
  • AppLogic(TM) treats the entire N-tier application as a single logical entity that can be copied, instantiated, configured, started, stopped, cloned, exported, imported, etc.
  • AppLogic(TM) treats the entire N-tier application as a single logical entity that can be copied, instantiated, configured, started, stopped, cloned, exported, imported, etc.
  • a user can scale an instance of the application from a fraction of a server to dozens of servers simply by defining how much CPU, memory and bandwidth may be be be allocated to that specific instance. Any number of instances of the same application can be executed simultaneously on the same grid. Multiple, unrelated applications can share the grid.
  • an instance of an application can be cloned, together with its state, database and content, and exported to run on another grid that may be located half-way around the world.
  • AppLogic(TM) may be operable to provide a set of functions that for running mainstream web applications.
  • Such functions may include, but are not limited to, one or more of the following (or combinations thereof):
  • AppLogic(TM) may be operable to implement a variety of services that enable the building of real-world utility computing systems. Examples of such services may include, but are not limited to, one or more of the following (or combinations thereof):
  • Grid management system manages a datacenter as a single system.
  • AppLogic(TM)-based grids into a single scalable, highly available computing cloud that, for example, may be used to run distributed Web 2.0 applications.
  • Cloud Computing may be characterized as a pool of abstracted, highly scalable, and managed computing resources capable of hosting end-customer applications.
  • the cloudware techniques described herein may be implemented on software and/or hardware. For example, they can be implemented in an operating system kernel, in a separate user process, in a library package bound into network applications, on a specially constructed machine, over a utility computing grid, on a network interface card, etc.
  • the technique described herein may be implemented in software such as an operating system or in an application running on an operating system.
  • a software or software/hardware hybrid implementation of various cloudware related techniques may be implemented on a general-purpose programmable machine selectively activated or reconfigured by a computer program stored in memory.
  • Such programmable machine may be a network device designed to handle network traffic, such as, for example, a router, a switch and/or a server.
  • Such network devices may have multiple network interfaces including frame relay and ISDN interfaces, for example.
  • a general architecture for some of these devices will appear from the description given below.
  • some cloudware techniques described herein may be implemented on a general-purpose network host machine such as a personal computer, server, or workstation. Further, at least one embodiment may be at least partially implemented on a card (e.g., an interface card) for a network device or a general-purpose computing device.
  • FIGURE IA illustrates an example embodiment of a server system 80 which may be used for implementing various aspects/features described herein.
  • the server system 80 includes at least one network device 60, and at least one storage device 70 (such as, for example, a direct attached storage device).
  • server system 80 may be suitable for implementing at least some of the cloudware techniques described herein.
  • network device 60 may include a master central processing unit (CPU) 62, interfaces 68, and a bus 67 (e.g., a PCI bus).
  • the CPU 62 may be responsible for implementing specific functions associated with the functions of a desired network device. For example, when configured as a server, the CPU 62 may be responsible for analyzing packets; encapsulating packets; forwarding packets to appropriate network devices; instantiating various types of virtual machines, virtual interfaces, virtual storage volumes, virtual appliances; etc.
  • the CPU 62 preferably accomplishes at least a portion of these functions under the control of software including an operating system (e.g. Linux), and any appropriate system software (such as, for example, AppLogic(TM) software).
  • an operating system e.g. Linux
  • any appropriate system software such as, for example, AppLogic(TM) software
  • CPU 62 may include one or more processors 63 such as, for example, one or more processors from the AMD, Motorola, Intel and/or MIPS families of microprocessors. In an alternative embodiment, processor 63 may be specially designed hardware for controlling the operations of server system 80. In a specific embodiment, a memory 61 (such as non-volatile RAM and/or ROM) also forms part of CPU 62. However, there may be many different ways in which memory could be coupled to the system. Memory block 61 may be used for a variety of purposes such as, for example, caching and/or storing data, programming instructions, etc.
  • the interfaces 68 may be typically provided as interface cards (sometimes referred to as "line cards").
  • one or more of the interfaces 68 may be provided as on-board interface controllers built into the system motherboard. Generally, they control the sending and receiving of data packets over the network and sometimes support other peripherals used with the server system 80.
  • the interfaces may be FC interfaces, Ethernet interfaces, frame relay interfaces, cable interfaces, DSL interfaces, token ring interfaces, Infiniband interfaces, and the like.
  • various very high-speed interfaces may be provided, such as fast Ethernet interfaces, Gigabit Ethernet interfaces, ATM interfaces, HSSI interfaces, POS interfaces, FDDI interfaces, ASI interfaces, DHEI interfaces and the like.
  • Other interfaces may include one or more wireless interfaces such as, for example, 802.11 (WiFi) interfaces, 802.15 interfaces (including BluetoothTM), 802.16 (WiMax) interfaces, 802.22 interfaces, Cellular standards such as CDMA interfaces, CDMA2000 interfaces, WCDMA interfaces, TDMA interfaces, Cellular 3G interfaces, etc.
  • one or more interfaces may include ports appropriate for communication with the appropriate media. In some cases, they may also include an independent processor and, in some instances, volatile RAM. The independent processors may control such communications intensive tasks as packet switching, media control and management. By providing separate processors for the communications intensive tasks, these interfaces allow the master microprocessor 62 to efficiently perform routing computations, network diagnostics, security functions, etc.
  • some interfaces may be configured or designed to allow the server system 80 to communicate with other network devices associated with various local area network (LANs) and/or wide area networks (WANs).
  • Other interfaces may be configured or designed to allow network device 60 to communicate with one or more direct attached storage device(s) 70.
  • FIGURE IA illustrates one specific network device described herein, it is by no means the only network device architecture on which one or more embodiments can be implemented.
  • an architecture having a single processor that handles communications as well as routing computations, etc. may be used.
  • other types of interfaces and media could also be used with the network device.
  • network device may employ one or more memories or memory modules (such as, for example, memory block 65, which, for example, may include random access memory (RAM)) configured to store data, program instructions for the general- purpose network operations and/or other information relating to the functionality of the various cloudware techniques described herein.
  • the program instructions may control the operation of an operating system and/or one or more applications, for example.
  • the memory or memories may also be configured to store data structures, and/or other specific non-program information described herein.
  • machine-readable media that include program instructions, state information, etc. for performing various operations described herein.
  • machine-readable storage media include, but are not limited to, magnetic media such as hard disks, floppy disks, and magnetic tape; optical media such as CD-ROM disks; magneto-optical media such as floptical disks; and hardware devices that may be specially configured to store and perform program instructions, such as read-only memory devices (ROM) and random access memory (RAM).
  • Some embodiments may also be embodied in transmission media such as, for example, a carrier wave travelling over an appropriate medium such as airwaves, optical lines, electric lines, etc.
  • program instructions include both machine code, such as produced by a compiler, and files containing higher level code that may be executed by the computer using an interpreter.
  • Figure IB shows an example embodiment of a global network portion 99 which may be used for implementing various aspects described herein.
  • global network portion 99 may include a plurality of different data centers (e.g., 85a-c) which, for example, may reside at different physical and/or geographic locations.
  • data center 85a may be located in the United States
  • data center 85a may be located in the United States
  • each of the different data centers may be communicatively coupled to each other via a wide area network (e.g., WAN 90) such as, for example, the Internet or world wide web.
  • WAN 90 wide area network
  • each data center may include a respective plurality of server systems 80 (herein “servers") which may be communicatively coupled together via one or more local area networks (e.g., LANl 91 and/or LAN2 92).
  • servers may be communicatively coupled together via one or more local area networks (e.g., LANl 91 and/or LAN2 92).
  • at least a portion of the data center servers may include at least a portion of features and/or components similar server system 80 of Figure IA.
  • the data centers may include several different types of local area networks such as, for example, a backbone LAN (e.g., LANl 91) which may be utilized for providing localized communication between various local network elements within a given data center, and an internet LAN (e.g., LAN2 92) which, for example, may be utilized for providing WAN or Internet access to various local network elements within the data center.
  • a backbone LAN e.g., LANl 91
  • an internet LAN e.g., LAN2 92
  • one or more of the data centers may be operable to host a variety of different types of applications and/or other software. Examples of such applications may include, but are not limited to, one or more of the following (or combinations thereof):
  • web-based application services such as, for example, Amazon's S3 storage service or Google's adwords
  • video-conferencing services such as, for example, Cisco's WebEx
  • service-oriented architecture and XML web service components such as, for example, the Aivea eShip Web Service at www.aivea.com/eshipinfo.htm, and/or the GeoIP lookup service at www.hostip.info/use.html
  • e-commerce applications such as, for example, Amazon.com, e-Bay and Apple iTunes
  • one or more of the data centers may be operable to provide various types of database services such as, for example, data storage, database queries, data access, etc.
  • virtualization software such as 3TERA's AppLogic(TM)
  • one or more of the servers of a given data center may be implemented as a server grid which may be operable to enable utility computing for distributed applications.
  • multiple server grids from different data centers may be linked together to form a virtual global server grid which may be used to facilitate utility computing for distributed applications.
  • a distributed application may be characterize as an application made up of distinct components (e.g., virtual appliances, virtual machines, virtual interfaces, virtual volumes, virtual network connections, etc.) in separate runtime environments.
  • different ones of the distinct components of the distributed application may be hosted or deployed on different platforms (e.g., different servers) connected via a network.
  • a distributed application may be characterize as an application that runs on two or more networked computers.
  • users may be able to access one or more of the data centers via the WAN 90.
  • one or more of the data centers may include hardware/software for implementing a Cloudware System 98 which may be used to provide users and/or data centers with various types of different functionalities.
  • the global network 99 may be collectively referred to as a Cloudware network.
  • Cloudware may be implemented as a global service which, for example, may be operable to provide computing resources to users according to various pricing models (such as, for example, subscription model, pay-as-you-go model, etc.).
  • the service may be operated by a business entity (e.g., 3Tera) and the computing resources may be provided by the business entity's (3Tera's) hosting partners or other service subcribers.
  • Network may be used to characterize a first portion of functionality which may be provided by Cloudware.
  • Example A may be intended to help illustrate various aspects and/were features relating to the various Cloudware related techniques described herein.
  • the Cloudware home page may describe what Cloudware is and may offer the user access to log in and/or sign up.
  • the user may choose one or more a data center locations for instantiating and running one or more distributed application(s) selected by the user.
  • the Cloudware services may be offered at different geographic locations, such as, for example, Texas, Germany, Japan, etc..
  • one or more customized user dashboard page(s) may be displayed the user.
  • the user dashboard page may include status information (e.g., status summary) relating to one or more of the user's associated applications.
  • the user dashboard page may include one or more different types of messages sent to the user's account.
  • the user dashboard page may also include a list of the user's applications and/or other Cloudware network resources.
  • the user can also see a catalog of available application templates by expanding an appropriate side bar and/or by accessing an expanded dashboard GUI.
  • the user dashboard page(s) may include various functionality for allowing the user to perform a variety of operations such as, for example, one or more of the following (or combinations thereof):
  • a user may edit a selected application using an infrastructure editor, such as that shown, for example, in the example of Figure 14.
  • the user can assign resources to each component (appliance) of the user's application separately, as well as to the application as a whole.
  • a user may be charged for use of different Cloudware services and/or Cloudware resources.
  • a user may be charged for one or more of the following Cloudware services/resources (and/or combinations thereof):
  • CPU/memory time (e.g., $ fee per CPU core/1 GB RAM, per hour);
  • appliance use e.g., $ fee per instance or per resource used by a licensed appliance
  • the prices/fees may be based on many factors and may differ from data center to data center.
  • various services/resources may be bundled together.
  • different bundles may be offered which include fixed amounts of CPU/memory, storage (e.g., 10 GB per CPU-month) and/or bandwidth (e.g., 100 GB per CPU-month).
  • the use may only be charged for costs relating to the monthly account fee plus the CPU/memory (e.g., because most other resources fit in the bundle and don't require a separate charge).
  • Such bundling also allows for better planning of resources at the datacenters that provide the resources.
  • the various prices/fee structures may also be based on a periodic (e.g., monthly, yearly, etc.) payment plan plus overage schema, similar to mobile phone plans in the ONE (e.g., like T-mobile's Individual Max.).
  • the user can pre-pay for a certain amount of resources (e.g., 10,000 GB-hours of RAM and hours of CPU) at a lower per-resource price for the pre-paid resources and be charged a higher price for resources used in excess of the pre-paid resources.
  • This payment schema allows for better planning both on the user side and on the datacenter/service operator side.
  • the periodic plans may be for different time periods - e.g., from hours to years; as well as simply a pre-bought package of resources (e.g., 1 million GB-hours) that can be used over a specified time period, (e.g., much like a prepaid phone card).
  • the pre-pay and bundle approaches may be combined to achieve predictable and competitive pricing for the service(s) offered.
  • Cloudware and/or Cloudware Nimbus may provide various features and/or functionalities such as, for example, one or more of the following (or combinations thereof):
  • Global data center selection o multiple geographic locations of different data centers/grids o multiple grids possible in each data center o accounts may be bound by default to a shared grid in a selected data center o accounts can be migrated manually between grids, data centers, or even moved to a dedicated grid (e.g., a grouping of Cloudware-enabled servers which have been allocated for exclusive use by a given user/account)
  • a dedicated grid e.g., a grouping of Cloudware-enabled servers which have been allocated for exclusive use by a given user/account
  • Per-usage billing for resources o hourly billing o flexible resource assignment (CPU in 10% increments, memory in 128MB increments, storage in 25MB increments) o bundled storage and transfer resources
  • AppLogic preferably continues to exist as a product for standalone virtual private data centers
  • Nimbus supports both accounts going to shared grids and/or to dedicated grids
  • a dedicated grid may include a group of physical servers in the Cloudware network which have been allocated for exclusive use by a specified user/account.
  • Cloudware and/or Cloudware Nimbus may also provide other features and/or functionalities such as, for example, one or more of the following (or combinations thereof):
  • Cloudware Nimbus may be configured or designed to follow a shared grid design which allows it to support all (or selected) desired features of Cloudware.
  • Cloudware can be thought of as operating system for the world's first global distributed computer. In this sense, it has resources (e.g., grids located at various different locations over the world), a kernel and a user interface.
  • resources e.g., grids located at various different locations over the world
  • kernel e.g., a kernel
  • user interface e.g., a user interface
  • Cloudware Nimbus may include one or more of the following components (or combinations thereof):
  • KERNEL -- Cloudware Kernel o Controller - service controller o Metering - metering system o Authentication — authentication service o Worker Apps - worker applications (long-term tasks initiated by the controller, such as volume resizing, application export, etc.) o DC Manager — data center manager (preferably not required for Nimbus) o Repository - data repository (preferably not required for Nimbus) o Scheduler - data center and grid scheduler (preferably not required for
  • CUI and KERNEL may be configured or designed as applications running on a "core" grid.
  • account shells (Shell(s)) may be intended to be active only while someone is logged in on the account.
  • account shells may be kept running at all or selected times (e.g., with resources paid for by the user's monthly account fees).
  • At least one example embodiment described herein comprises an application model, a visual method and a system for rapid delivery of distributed applications.
  • the phrase "inventive system” refers to an example embodiment and/or to alternative embodiments described or referenced herein.
  • the application model defines several abstractions which, taken together, make it possible to express the structures and behavior of complete distributed applications.
  • those abstractions can be grouped in the following way: virtual resources, virtual appliances, composite appliances, catalogs of appliances, and applications.
  • Virtual Resources can be grouped in the following way: virtual resources, virtual appliances, composite appliances, catalogs of appliances, and applications.
  • the present invention uses resource virtualization to abstract the underlying hardware system and to make it possible to define the rest of the application in a hardware-independent way.
  • At least one embodiment described herein defines various types of virtual resources, such as, for example: virtual machines, virtual volumes and virtual network interfaces.
  • the hardware system comprises computing and/or storage nodes interconnected through a suitably fast network, with at least one node acting as a system controller.
  • Each node on the network preferably exposes one or more pools of virtual resources, one pool for each resource type.
  • the system controller aggregates multiple discrete resource pools, exposed by the various nodes in the system, into a single, distributed resource pool.
  • Virtual resources are allocated/created from their respective system pools and carry a system-wide identification which makes it possible to access a given instance of a virtual resource in a uniform fashion independent of where the resource is actually located.
  • FIGURE 30 illustrates an example embodiment of an architecture of a virtual machine management system, in which a virtual machine monitor 3030 partitions a physical host 3000 into multiple virtual machines, such as the virtual machines 3010 and 3020, and manages the access from virtual devices 3013, 3014, 3023 and 3024 to physical devices 3040, 3050 and 3060.
  • Each virtual machine is capable of booting a general-purpose operating system, such as 3011 and 3021, and any other software that it may be configured to run.
  • Most virtual machine managers virtualize access to at least two types of peripheral devices, namely network interfaces and block storage devices.
  • peripheral devices namely network interfaces and block storage devices.
  • When configuring an individual virtual machine one can specify a set of virtual network devices and a set of virtual storage devices for that virtual machine, and define how those virtual devices should be mapped to the actual physical devices of the host.
  • some virtual machine managers make it possible to map a virtual device of a given virtual machine to a logical device (network interface or disk volume) implemented by an operating system in another virtual machine.
  • Virtual machine managers also allow individual virtual machines to be migrated from one host to another, transparently to the software that runs inside the virtual machine.
  • An example of such prior art virtual machine manager is Xen, described in [Xen].
  • virtual machines are assigned a set of execution attributes that determine the minimum and maximum amounts of processing power, memory and network bandwidth that can be allocated to a given instance of a virtual machine, as well as to permit or prohibit the migration of the virtual machine.
  • Virtual storage volumes are logical block devices exposed by one or more hosts on the system and accessible from virtual machines running on the same or on other hosts. Virtual volumes are persistent, named objects, the size of which is defined at the time of creation and which reside on the system until explicitly destroyed.
  • a virtual volume defined and exposed by one node is accessible from any node in the system, thereby allowing a virtual machine that uses the volume to be migrated freely to any node.
  • One way to implement virtual volumes is by configuring [NBD] so that each individual virtual volume is stored in a file on one of the hosts, shared on the network as an NBD volume and accessed from the other hosts using the NBD client.
  • a virtual volume is typically accessed exclusively by a single virtual machine. This makes it possible and desirable to cache volume contents aggressively on the host on which the virtual machine accessing the volume is being executed. Such caching is easily accomplished, for example, by layering on top of the NBD client a block device driver that uses a file on a local physical disk to store copies of blocks recently accessed by the virtual machine.
  • Another aspect described herein is the ability to create multiple instances of the same virtual volume. Those are useful whenever there is a need to share a large set of data among multiple virtual machines in such a way as to permit each virtual machine to make relatively small number of modifications to the common set of data for its own use. Instantiable virtual volumes can be implemented by simply replicating the common volume for each virtual machine.
  • an instantiable volume is implemented by a combination of a "master" virtual volume which is common to all instances and contains the common data, and a "differential" virtual volume for each virtual volume instance, which accumulates the modifications made to the specific instance.
  • the master volume and the differential volume are presented to the client virtual machine as a single block device, for example, by layering an appropriate block device driver over an NBD client that can access both virtual volumes.
  • FIGURE 31 illustrates the inventive virtual network interfaces provided by the present invention.
  • Virtual network interfaces are used to abstract the structure of the network interconnect inside the distributed application.
  • a pair of virtual network interfaces such as VNIl and VNB, is used to create a "virtual wire" between virtual network adapters vNICl and vNIC3, which belong to virtual machines VMl and VM2, respectively.
  • the virtual wire operates in a manner equivalent to a cross-over cable that connects two physical network interface cards directly: it transfers packets from one of the cards to the other and vice-versa.
  • virtual network interfaces are implemented by combining two types of objects, a virtual interface factory, such as VNFACl, and a virtual interface instance, such as VNIl.
  • the virtual interface factory is preferably attached to each virtual machine and creates one virtual interface instance for each virtual network adapter configured on its virtual machine.
  • the factory configures each virtual interface instance with the MAC address of its respective virtual network adapter, thereby allowing the instance to intercept all outbound traffic from that adapter.
  • the virtual interface instance VNIl is also configured with information sufficient to establish connection with its counterpart, the virtual interface instance VNI3 using the physical network available in the hardware system.
  • VNIl intercepts outgoing traffic from vNICl and forwards it to VNI3 which channels the packets into vNIC3, optionally modifying packet headers to support the tunneling abstraction. Traffic in the opposite direction is handled the same way.
  • virtual wire VCl can be implemented by tunneling application traffic (packets) between two virtual network interfaces through a TCP connection, UDP datagrams, InfiniBand reliable connection, or as direct memory-to-memory transfer whenever both VNIl and VNI3 happen to be located on the same host, all of which is completely transparent to the communicating virtual machines VMl and VM2. Indeed, it is possible to move the virtual wire VCl from, for example, a TCP connection over Gigabit Ethernet, to a reliable connection over 10 Gigabit InfiniBand on the fly, transparently to the communicating virtual machines.
  • Virtual Appliances application traffic
  • FIGURE 32 illustrates the inventive virtual appliance.
  • the virtual appliance 3200 comprises a boundary, boot volume 3240, and interior.
  • the boundary comprises the execution attributes 3210, the terminals 3220, 3221 and 3222, the properties 3230, the content volume 3241.
  • the interior comprises operating system 3250, configuration files 3280, software services 3260 and the application service 3270.
  • virtual appliances are defined by building a descriptor such as the descriptor 700 illustrated in FIG. 7 of U.S. Pub. No. 20070078988.
  • virtual appliances are created by first defining a virtual appliance class using descriptor similar to 700 and then creating one or more virtual appliance instances that execute on the target system.
  • the class is used as a template for creating instances.
  • FIGURE 33 illustrates the process of creating multiple virtual appliance instances from one class.
  • the system first creates a virtual machine with one virtual network adapter for each terminal, such as 3381 and 3382, and an instance of a virtual network interface for each of the adapters.
  • the system creates one virtual block device for each volume 3360.
  • the system next creates a virtual volume instance 3360 by either replicating the class volume 3310 or by creating a differential volume using the class volume 3310 as a master, as described above, and binds it to the corresponding block device created above.
  • the virtual machine of the instance is created using the specific values assigned to the execution attributes.
  • the instance is configured with the values 3370 of the properties 3320, preferably by modifying the configuration files 3351 residing on the volume 3360. Since volume 3360 is an instance of the master volume 3310, the modifications are private to the instance 3350.
  • the system then proceeds to execute the virtual machine, resulting in the booting the operating system 3352 and starting the various services 3353.
  • the inventive process for defining virtual appliance classes and instances makes it possible to separate (a) the information and configuration that are common to all virtual appliances of a given class, such as the operating system and the application service code, and the configuration required to make them work together; and (b) the configuration and connection data that are specific for each instance of the virtual appliance based on its role in the distributed application.
  • each class of virtual appliances would have configuration parameters that are specific to the function and the implementation of the class.
  • the present invention provides a mechanism for exposing the desired set of such configuration parameters to be modified by the application designer through a universal property interface modeled after properties of software components (such as Microsoft ActiveX controls).
  • the designer of a virtual appliance class defines the set of properties 3320, preferably by defining the name, data type and default value of each property as part of the class descriptor.
  • the virtual appliance designer specifies the names of one or more configuration files 3351, into which the values of the properties need be transferred at the time of instance creation.
  • FIGURE 34 illustrates an example embodiment of a mapping of virtual appliance property values into configuration file settings and scripts that execute inside an instance of a virtual appliance.
  • scripts 3400 for each property defined in the appliance class an example embodiment provides an environment variable named after that property and initializes such variable to the value of the property with which the instance was configured.
  • a parameter 3411 is set to a specific value 3414.
  • the designer of the appliance adds a comment to the configuration file with a tag 3412, identifying the appliance property name 3413, which is to be mapped to the parameter 3411.
  • Terminals of Virtual Appliances In order to visually build structures of virtual appliances, the present invention defines the notion of terminals as connection points that represent endpoints for logical interactions between appliance instances.
  • the inventive terminals are designed so that already existing software packages used inside virtual appliances can communicate through terminals without requiring modifications.
  • a terminal could be an input, such as the input 3220, or an output, such as the outputs 3221 and 3222.
  • An input terminal is a terminal for accepting network connections; an output terminal is a terminal for originating network connections.
  • both types of terminals allow bi-directional transfers.
  • a terminal preferably comprises a name, a virtual network adapter and a virtual network interface.
  • the system When an output terminal of one virtual appliance instance is connected to an input terminal of another instance, the system creates a virtual wire between their respective virtual network interfaces, and assigns virtual IP addresses to both ends of the connection.
  • the virtual applianceVAl has a virtual machine VMl and an output terminal OUTl, comprising vNICl and VNIl. This terminal is connected to the input terminal IN of the virtual appliance VA2 through the virtual wire VCl.
  • the inventive system will provide it with the virtual IP address assigned to the opposite end of the virtual wire VCl which is connected to the terminal IN. This has the effect of binding the network host name "OUTl" in VAl to the IP address of the terminal IN of VA2.
  • Each instance of the inventive virtual appliances has at least one volume from which it boots operating system and other software. These volumes may be provided as part of the class definition of the appliance and instantiated for each virtual appliance instance. In many cases, virtual appliances may have additional volumes that are not part of the class definition but are explicitly configured on each instance of the virtual appliance.
  • the boot volume 3240 may contain software and configuration necessary to boot a Linux operating system and run an Apache web server; this volume is part of the class definition and is instantiated for each instance of the appliance 3200.
  • the volume 3241 may contain data specific to a given web site, for example, HTML files, images and JavaScripts.
  • appliance 3200 While the class definition for appliance 3200 includes a reference to the specific volume 3240, it only defines a placeholder for the volume 3241, indicating that each instance of the appliance 3200 may be explicitly configured with a reference to such volume. Instantiating the appliance 3200 and configuring the instance with a reference to the volume 3241 has the effect of producing an instance of an Apache web server that serves the particular web site the content of which is located on volume 3241. In addition, defining a property on the appliance 3200 through which the appliance can be configured with a directory name on the volume 3241 from which it would access the content allows multiple different instances of the appliance 3200 to be configured with the same volume 3241 but serve different content located in different directories on the volume.
  • the same pattern can be applied to design a generic J2EE server appliance that can be configured with a volume containing the EJB code packages for a particular application function, or a generic database server configured externally with a volume containing a specific database.
  • a generic J2EE server appliance that can be configured with a volume containing the EJB code packages for a particular application function, or a generic database server configured externally with a volume containing a specific database.
  • FIGURE 33 illustrates a presentation tier of a web application implemented as a structure of virtual appliances.
  • the structure comprises one instance of a load balancer appliance 3301, and three instances of a web server appliance, the instances 3302, 3303 and
  • the outputs 3310, 3311 and 3312 of the load balancer 3301 are connected to the inputs
  • load balancer 3301 is parameterized with a value for its TIMEOUT property 3330
  • web server instances are parameterized with a cache size value for their CACHE properties 3340
  • Arbitrarily complex structures of virtual appliances can be described in a uniform way by capturing the set of instances that participate in them, configuration parameters for each instance and the connections between their terminals. This allows the inventive system to instantiate such structures automatically, by interpreting such structure descriptions, instantiating virtual appliances, configuring them with the provided values and establishing virtual wires through which the appliances could interact.
  • each described instance is assigned a human-readable name that identifies the role that such instance plays within the structure.
  • a composite appliance comprises a boundary and an interior.
  • the boundary of a composite appliance is defined in the same way as the boundary of a regular virtual appliance, and the interior of a composite appliance comprises a structure of virtual appliances.
  • FIGURE 35 illustrates the inventive composite virtual appliance. It defines a new, composite appliance class 3500 that implements a scalable web tier of a distributed application as a single appliance.
  • the boundary of the appliance 3500 comprises an input terminal 3510 and two output terminals 3511 and 3512, as well as properties 3520 and 3521.
  • the interior of the appliance 3500 comprises the load balancer instance 3530 and two instances of a web server, the instances 3540 and 3550.
  • the input terminal 3510 is connected to the input terminal 3531 of the load balancer; the outputs 3532 and 3533 of the load balancer are connected to the input terminals 3541 and 3551 of the web servers 3540 and 3550, respectively.
  • the outputs 3542 and 3552 of the web servers are connected to the output 3511 of the composite; while the outputs 3543 and 3553 are connected to the output 3512.
  • property 3521 of the composite is redirected to the property 3535 of the load balancer 3530, while the property 3520 of the composite is redirected to the properties 3545 and 3555 of the web servers.
  • the resulting composite appliance 3500 can be used in any structure or application in the place of a web server such as 3540, without having to know anything about its interior or even the fact that it is a composite appliance. Unlike the web server 3540, it will deliver increased performance and increased resilience to hardware failures (since it can operate with one of the web servers 3540 or 3550 having failed), without increased visible complexity in the target application.
  • FIG. 14 of U.S. Pub. No. 20070078988 An example embodiment of a text descriptor form of a composite appliance (e.g., similar to the composite appliance 3500) is illustrated, for example, at FIG. 14 of U.S. Pub. No. 20070078988.
  • the descriptor preferably assigns a name to the appliance class, identifies properties, terminals and volumes visible on the boundary of the appliance, lists the subordinate instances that form the structure of the appliance, assigning a name to each instance, identifying the class of the instance, and configuring each instance by assigning values to one or more properties, attributes and/or volumes; and describes the connections between terminals of subordinate appliances, as well as between the terminals defined on the boundary of the composite appliance and terminals of its subordinates.
  • an example embodiment of a descriptor provides a simple way to "redirect" a property of the composite appliance to one or more of its subordinates.
  • the property "cache_sz" of the web_tier composite appliance (assembly) is redirected to the property "cache_sz" of its subordinates "webl” and “web2” by means of specifying "$.cache_sz” in place of an explicit value in the configuration section of each of those subordinates. This has the effect of configuring each of the webl and web2 subordinates with the actual value with which the web_tier composite is ultimately configured in the target application.
  • the inventive system preferably implements a property mechanism that redirects properties of the composite to one or more properties of its subordinate instances, by redirecting configuration values set on an instance of a composite appliance to properties of the appropriate subordinates, as defined by the interior structure; and a terminal mechanism that forwards the configuration information required to create virtual wires received by the terminals of the composite appliance to the terminals of the appropriate subordinates to which they are connected.
  • a property mechanism that redirects properties of the composite to one or more properties of its subordinate instances, by redirecting configuration values set on an instance of a composite appliance to properties of the appropriate subordinates, as defined by the interior structure
  • a terminal mechanism that forwards the configuration information required to create virtual wires received by the terminals of the composite appliance to the terminals of the appropriate subordinates to which they are connected.
  • Such mechanisms can be implemented by the system runtime support similar to [XDL] or, preferably, by a structure linker utility that resolves property and terminal forwarding references prior to instantiating the application.
  • the present invention defines a way to package multiple classes of virtual appliances into class libraries called Catalogs.
  • the catalogs can be used in multiple applications.
  • Each virtual appliance class preferably comprises a class descriptor and one or more volume images referenced by the descriptor.
  • Each composite appliance class preferably comprises a class descriptor similar to the class descriptor of the regular virtual appliance classes and an interior descriptor that captures the structure that implements the composite.
  • a catalog preferably comprises a catalog package descriptor that identifies the classes included in the catalog and the class descriptors, volume images and interior descriptors of those classes.
  • a catalog can be implemented as a shared directory on a network in which all descriptors and volume images reside. Alternatively, a catalog may be exposed through a web or ftp interface on the Internet.
  • FIGURE 36 illustrates the inventive catalog structure. It includes the external catalog
  • the classes 3610 and 3620 are regular virtual appliances and contain no references to other classes.
  • the class 3630 is a composite virtual appliance and contains at least one instance of the class 3620 and, therefore, has a reference 3631 to the class 3620.
  • Classes included in catalogs preferably have names that are unique within the catalog.
  • the name of that class is sufficient to resolve the reference.
  • the name of the catalog is preferably pre-pended to the name of the class to form a name that is unique within the inventive system.
  • FIGURE 36 also illustrates the structure of the inventive application.
  • the application 3650 is described as a package that comprises a local catalog 3660, a MAIN singleton class 3670, and another singleton class 3680, as well as the application volumes 3690, 3691 and 3692.
  • the local catalog 3660 is a catalog containing the classes 3661 and 3662 which are specific to the application 3650 and are not intended to be used outside of it.
  • the present invention defines a singleton class as a class of which only a single instance may be created. Singletons may not exist outside of the scope of an application and cannot be included in shared catalogs.
  • Each application preferably has at least one singleton, the MAIN 3670, which includes the top-level structure of the application. In addition to the MAIN singleton, other singletons can be used to define subsystems of the application that are not intended to be instantiated by design. All singletons in an application preferably reside directly in the application package and outside of the local catalog.
  • Each application preferably contains one or more virtual volumes that are not directly associated with any virtual appliance class. Such volumes may be used to store application- specific content, code packages, libraries and databases, in a layout convenient for access by the operator and are bound by configuration to virtual appliance instances that require access to such data.
  • the abstractions defined in the application model are sufficient to describe constructively the structure of an arbitrary distributed application without references to the hardware system on which it would execute, and without explicit dependencies on the actual software functionality encapsulated in each of the virtual appliances.
  • the structure and configuration of the application defined in the terms of the application model can be easily expressed through a set of static descriptors using a structure descriptor language such as XML.
  • a structure descriptor language such as XML.
  • Various example embodiments of structure description language are illustrated, for example, in FIG. 7 and FIG. 14 of U.S. Pub. No. 20070078988.
  • this language may be semantically equivalent to XML but is less verbose and more suitable for direct editing by humans.
  • an arbitrarily complex distributed application can be described in a set of text files, such as, for example, one or more of the following (or combinations thereof): (1) virtual appliance descriptors; (2) composite appliance boundary descriptors; (3) composite appliance interior (assembly) descriptors, and (4) package descriptors.
  • this set of descriptors, together with the images of class volumes and application volumes, is sufficient to instantiate and execute the application on any hardware system that supports resource virtualization and other services defined by the present invention.
  • one example embodiment of a method of practicing at least one embodiment described herein is to design, implement, integrate and deploy applications in a visual manner. This takes full advantage of the fact that all abstractions defined in the application model - virtual appliances, structures of appliances, composite appliances and whole applications - are easy to visualize and most operations with them are easy to implement as visual operations on a computer user interface.
  • the primary functionality of the user interface is implemented by an application editor that makes it possible to create, edit and save the descriptor files that comprise a distributed application.
  • the editor is preferably implemented as a web-browser based user interface, allowing access to the editing functionality from any workstation having network connection to the inventive system.
  • An example embodiment of an application editor with a distributed e-commerce application displayed on the canvas is illustrated, for example, in FIG. 20 of U.S. Pub. No. 20070078988.
  • the property sheet screens and behavior of the editor may be similar to most desktop windowed applications, such as Microsoft Windows Explorer property sheets and follow similar visual design guidelines.
  • scopes e.g., composite appliances
  • the editor preferably supports both visualization options.
  • Operations in the editor may be implemented so that they can be applied to a single component or to a selected set that contains multiple components.
  • Such operations preferably include at least drag and move on the canvas; cut, copy and delete; and modifications achieved through property sheets.
  • the windows displayed by the editor have titles that preferably contain the name of the component being edited, the type of editor and the name of the application. It is also preferable that the editor performs basic file locking on descriptor files on which it presently operates, similar to the locking schemas employed typically by text editors, such as the "vi" editor in Linux. This allows multiple users to safely view and/or edit one and the same application.
  • the editor preferably does not save any modifications to the application made by the user until the user explicitly chooses a "save" operation. If, while navigating through the application, the user tries to close a window or navigate away from the modified component, and changes would be lost, the editor preferably prompts the user, giving him an option to save or discard the changes.
  • the editor preferably implements a different screen for each type of entity being edited. These screens preferably include: a list of available applications, a virtual appliance editor, a composite appliance boundary editor and an assembly (interior) editor. In addition, the editor preferably allows visual operations between entities, such as dragging virtual appliances from a catalog onto the application canvas and vice-versa.
  • the Application List is preferably the first screen that the user sees after logging in.
  • This screen preferably contains the list of applications available for editing and provides the ability to select and open for editing or viewing one of these applications.
  • the screen preferably provides ability to execute certain actions over whole applications, such as creating a new application, deleting a whole application, renaming an application, etc.
  • Each entry in the application list preferably includes the name of the application, a human-readable description and a unique identifier.
  • the virtual appliance editor (also known as the component editor) is preferably a property sheet window for editing virtual appliance classes. All information available in this editor is obtained from and stored in the component descriptor file of the edited virtual appliance class. The appearance of the editor is preferably distinctly different from other property sheets, especially from the instance settings property sheet of the assembly editor.
  • FIGURE 37 illustrates an example embodiment of a visual interface of the virtual appliance editor.
  • the virtual appliance editor preferably displays a preview of the appliance's graphical shape, showing the correct size and color, as well as the terminals, their names and positions.
  • the virtual appliance editor preferably comprises the following sections, with each section implemented as a separate property sheet tab: a general section, an interfaces section, a volumes section, a resources section, a properties section and a configuration files section.
  • the general tab preferably contains common class attributes, as well as some visual attributes.
  • An example of the fields available through this section includes the class name, a description, operating system type, whether instances of this class can be migrated live from one server to another, as well as visual shape, size and color.
  • the interfaces tab preferably allows the user to view, edit and create the set of virtual appliance interfaces, including both terminals and virtual network adapters. It preferably displays a list of terminals showing, for each terminal, its name, direction (input or output), communication protocol for connections on that terminal and a "mandatory" attribute that defines whether the terminal may be connected in order for the appliance to operate. For "raw" virtual network adapters - those that are not associated with a terminal - the editor may allow defining and editing the MAC address. Using the interfaces tab, the users can add, delete or rename terminals in the list. The terminal's position, such as the side of the component's shape on which the terminal appears, and its order among other terminals on that side, may be editable as well.
  • the editor preferably allows the user to insert gaps between terminals, so that terminals can be visually grouped, as convenient.
  • the volumes tab preferably defines the set of volumes to be used by instances of the virtual appliance class being edited.
  • the list includes both class volumes, which are to be instantiated with the appliance, and placeholders for application volumes, which are to be parameterized on each instance of the appliance.
  • the editor preferably allows the user to define a logical name that determines the role of the volume within the appliance, a mount path under which this volume will be visible to the software inside of the appliance, and a boot attribute defining whether this volume is the boot volume for the appliance.
  • the user can add, delete and rename volumes in the volume list.
  • the volumes tab preferably allows the user to define a variety of attributes for each volume.
  • attributes may include class vs. placeholder, a "mandatory" attribute for placeholders that defines whether the appliance may be parameterized with a valid volume in order to operate.
  • the editor preferably makes it possible to restrict the access of the appliance instances to a given volume to read-only access, as well as to express constraints, such as "high-bandwidth access” and "local access only” that allow the inventive system to optimize the placement of the volumes and virtual machines that comprise appliance instances.
  • the resources tab preferably allows the user to set minimum and maximum values for each hardware resource required to execute an instance of the virtual appliance. Such resources include at least CPU time, memory size and network bandwidth. The system can use these values to ensure that sufficient resources are available for each virtual appliance instance, as well as to prevent any particular instance from depriving the rest of the executing instances of any particular resource.
  • the property tab preferably allows the user to define, view and edit the list of properties made available on each instance of the edited virtual appliance class. It preferably contains a list of properties, specifying for each property its name, data type, whether setting this property is preferred on each instance, a default value, and optionally, constraints, such as range for integer properties, maximum length for strings, and enumerated list of allowed values.
  • the user can add, delete and rename properties on the list, as well as edit any of the attributes of each property.
  • the configuration files tab preferably lists the set of configuration files contained within the virtual appliance to which property values are to be applied at instantiation.
  • the tab preferably includes the logical name of the volume (as defined in the volumes tab) on which the file is to be found, the path of the file relative to the specified volume, and additional information, if needed, such as special character escaping rules for that file.
  • the user preferably can add and delete configuration files, and edit the information for each file.
  • the boundary editor is preferably a property sheet that allows the user to define the boundary and other elements of a composite appliance that are not related to appliance's interior structure. This editor is visually and semantically similar to the virtual appliance editor, except that it operates on composite appliances.
  • the editor preferably operates in read-only mode for all classes except singletons included directly in the application package, and is preferably divided into several sections (tabs).
  • the general tab contains common class attributes, as well as visual attributes. Those preferably include the class name, a description, shape color, size and style.
  • the terminals tab preferably allows the user to view, define and edit the set of terminals exposed on the boundary of the composite appliance. It preferably contains a list of terminals, including, for each terminal, its name, direction (input or output), and a "mandatory" attribute. The user can add, delete and rename terminals, as well as edit the data related to each terminal.
  • the terminal's visual position on the appliance shape, such as side and order of terminals, can be edited as well; gap insertion is preferably supported if it is supported for virtual appliances.
  • the properties tab preferably allows the user to define the set of properties that is to be exposed on the boundary of the composite appliance. It preferably includes a list of properties, defining, for each property, name, default value and an optional "mandatory" attribute. The user can add, delete and rename properties, as well as edit data related to each property.
  • the volumes tab allows the user to define a set of volume placeholders that can be configured with references to application volumes on the boundary of the instances of the edited composite appliance class.
  • the tab preferably provides name, an optional "mandatory" attribute, as well as other attributes, such as shared or readonly.
  • the user can add, rename, delete or edit list elements.
  • the assembly editor is the main screen of the application editor. It allows users to view and edit the interior structures of composite appliances. This includes adding or removing subordinate instances, configuring each of those instances, and creating the structure by interconnecting their terminals.
  • the assembly editor preferably supports the ability to customize virtual appliance classes in a convenient visual way.
  • the assembly editor preferably provides the means for opening the other editors, such as the virtual appliance editor, the boundary editor, etc.
  • the assembly editor provides a drawing canvas on which appliance instances, virtual or composite, are configured and assembled into structures.
  • the editor preferably includes one or more palettes that make it possible to select the classes of virtual appliances to be included in the structure from a catalog, recycle bin, etc.
  • the user preferably selects an appliance class from a palette and drags it onto the canvas. If the selected class is a virtual or composite appliance, the editor will create an instance of that class. If a special "blank" class is selected, the editor will preferably create a new singleton class and place it directly in the application package; as well as create an instance of this class. In addition, the editor will generate automatically a name for the instance and/or, optionally, for the singleton, so that the name is unique within the structure being edited.
  • the editor preferably displays each instance as a rectangular shape with attached terminals.
  • the color, style and size of the shape, as well as the positions of the terminals, are as specified when defining the virtual appliance class to which this instance belongs.
  • the editor preferably displays the class name within the body of the instance, the instance name outside of the body, the name and direction of each terminal within the terminal, and zero or more selected attributes that apply to this appliance.
  • the editor allows the user to drag it freely around the canvas, changing its position, and preferably preventing the user from placing it directly over another instance.
  • the terminals of the instance can be connected by preferably clicking on one of the terminals and dragging a connection to the other terminal.
  • the editor preferably allows output terminals to be connected only to input terminals and input terminals only to output terminals. Each output is preferably connected to only one input, while many outputs can be connected to the same input.
  • the editor routes connections automatically by default, and preferably allows the user to re-route any connection manually by dragging moveable lines and corners of connections, and by adding or deleting line segments.
  • the editor allows the user to select one or more instances and apply various operations to them. Whenever a selected instance or group is moved, their connections are preserved as much as possible; this includes preserving all connections between the selected instances, and re-routing any connections from a selected instance to a non-selected instance.
  • FIG. 17 of U.S. Pub. No. 20070078988 An example embodiment of the interior of a composite appliance "Web Tier" opened in the assembly editor is illustrated, for example, in FIG. 17 of U.S. Pub. No. 20070078988.
  • the terminals of the composite appliance may be visualized on the canvas as small, pseudo-appliances, with one terminal each, indicating the name and direction of the respective terminal, and can be connected to the interior structure.
  • the user can preferably add text box annotations on any place on the canvas.
  • the editor will preserve such annotations as comments in the structure describing the appliance interior.
  • the editor preferably allows the following operations over selected appliance instances: cut, copy, paste, view/edit class boundary, view/edit class interior (for composite appliances), configure instance, and branch class. Those operations may be selected by a right- button click on the instance shape, which opens a context menu and selecting the desired operation from the menu.
  • the semantics of the cut, copy and paste operations are the same as in any windowed editor; viewing class boundaries and/or interiors is accomplished by starting the appropriate editor over the class of the selected instance.
  • Configuring instances is accomplished by displaying a special instance settings property sheet that is preferably part of the assembly editor and displays and modifies data within the same structure descriptor. Catalog Palettes
  • the visual editor preferably provides a set of palettes, one for each catalog made available to the user.
  • the user is preferably able to select a subset of catalogs to be displayed at any time.
  • Each palette displays an icon for each appliance class found in the respective catalog, with the icon visually resembling the shape of the component as much as possible.
  • the icons displayed may be grouped by category of appliance they represent, such as servers, infrastructure, gateways, load balancers, etc. Dragging an icon from the catalog onto the canvas preferably has the effect of including a new instance of the selected class into the edited structure.
  • Dragging a special "blank” appliance or a "blank” composite appliance from the palette preferably creates a singleton class included directly in the application package, and an instance of this class included into the edited structure.
  • a right-button mouse click on an icon in the catalog preferably opens a menu that gives the user options, such as deleting or renaming the class, creating an instance of the class (same as drag to canvas), copying the class, moving the class to another catalog or converting it to a singleton, viewing the appliance boundary and interior (if the appliance is a composite).
  • double-clicking on an appliance icon in the catalog palette preferably opens up the respective editor to display detailed boundary information about that class.
  • Branching a class involves creating a copy of the class of the selected instance, designating such copy as a singleton class, placing the singleton class directly in the application package, and changing the class of the selected instance to the new singleton class. Branching creates a tightly coupled pair comprising an instance and a singleton class, which can be edited as single entity. Adding a Class to a Catalog
  • the user preferably converts a singleton into a class. To do this, the user selects the instance of the singleton on the canvas and drags it into the desired catalog's palette. The editor then creates the appropriate class within the catalog structure, copies and/or moves all class data and volumes into the catalog, and preferably deletes the singleton. In addition, the instance that was selected to initiate the operation is preferably removed from the structure. Instance Settings
  • the instance settings property sheet allows users to configure a subordinate instance in a structure of virtual appliances. Unlike in appliance and boundary editors, in which changes apply to the all future instances of the edited class, instance settings apply only to the selected instance. Instance settings override any default values specified in the class.
  • the editor allows the user to specify a "reference" to a property of the composite that contains that instance. If such reference is specified, the system will substitute it at the appropriate time with a value assigned directly or indirectly to the respective property of the composite. This makes it possible to "redirect" a property, attribute or volume of the composite instance to one or more properties, attributes or volumes of its subordinate appliances.
  • the instance settings may be divided into several sections (tabs).
  • the attributes tab contains the instance name, as well as a set of attributes that apply to that instance.
  • the tab preferably includes the class name and may include optional attributes, such as a start order, migrateable, standby, etc.
  • the resources tab preferably makes it possible to override the resource constraints specified in the class of the virtual appliance to further reduce the range of resources available to the particular instance, if desirable.
  • FIG. 18 of U.S. Pub. No. 20070078988 illustrates an example embodiment of a design of the instance settings volumes tab. It allows the user to configure the instance, so that it can access a specific application volume. To achieve this, the instance is preferably configured with the name of the desired application volume.
  • FIG. 19 of U.S. Pub. No. 20070078988 illustrates an example embodiment of a design of the instance settings properties tab. It allows the user to set property values that configure and specialize the instance for its role within the structure. For each property defined on the class, the user may view the default value, if any, and override it if desired. In addition, the user may select one or more properties and their values to be displayed by the editor in the vicinity of the instance's shape on the canvas, thereby improving the readability of the diagram.
  • the visual editor preferably allows users to define application-level configuration parameters that can be used to modify the behavior of the application as a whole, bind it to a particular hardware configuration, etc.
  • the application configuration property sheet is preferably divided into several sections (tabs).
  • the general tab describes the application as a whole, including name, version, human- readable description, comments, unique ID, etc.
  • the application resources tab defines a subset of the hardware resources within the inventive system that are to be assigned to the given application.
  • the tab preferably contains two general groups of fields, one for hardware resources, and the other for IP network settings.
  • Hardware resources may be specified in terms of number of CPUs, total amount of memory and bandwidth to be committed to the application. In some embodiments of the system, it may be preferable to specify the hardware resources in an alternative fashion, such as total number of servers assigned to the application or a list of specific servers designated to run it.
  • the IP network settings group preferably defines the range of IP addresses to be allocated for internal use by the inventive system when running this application.
  • the property tab is preferably similar to the instance settings property tab discussed above, and makes it possible to configure the application as a whole in a manner similar to configuring any other composite appliance.
  • the application volumes tab preferably enables the user to create and manage a set of application volumes associated with the given application, assign their names and sizes, and configure the application in using them.
  • the user can add, rename and delete volumes; and assign reference to volumes to volume placeholders exposed on the boundary of the application in a manner preferably similar to configuring any other composite appliance.
  • the present invention teaches a visual method for rapid design, construction and deployment of distributed applications using the application model and visual interface described herein.
  • steps comprise creating a virtual appliance, assembling a composite appliance from existing appliances, creating a new appliance class in a catalog and creating the application.
  • this section covers related topics such as troubleshooting applications designed with the inventive system and monitoring their execution. Creating a Virtual Appliance
  • the user preferably opens the application editor and drags a blank virtual appliance onto the editor canvas. This creates a new, automatically named singleton class and an instance of that class. The user then selects the new instance and opens the virtual appliance editor on its class.
  • the virtual appliance editor defines the new virtual appliance by specifying appropriate class name, and a set of properties, terminals, interfaces and volumes.
  • the user selects appropriate values for hardware resources, properties and execution attributes that will be used as defaults for new instances of this class.
  • the user creates one or more application volumes that will be later used as class volumes for the new virtual appliance and then installs or copies the desired combination of operating system, add-on software packages and configuration data for the appliance.
  • the user further configures the various software packages that may operate together inside the appliance in accordance with their documentation.
  • the user selects configuration files and parameters within them that are to be exposed for configuring the virtual appliance and maps them to properties using one of the property mechanism methods described herein.
  • the user configures the software packages within the appliance to use the names of the terminals defined on the boundary of the appliance. If the appliance does not have multiple input terminals with the same protocol, the software within the appliance is configured to listen for incoming network sessions in the conventional way (e.g., by port number only). If two or more input terminals are defined with the same protocol, for each such terminal, the user has to configure the software so that it will listen for network sessions using the name of the desired terminal as a network host name. For output terminals, the user configures the appropriate software packages as if the name of the respective output terminal was the name of a remote network host to which the package is expected to establish a communication session.
  • the volumes are bound to the appliance being created by opening the instance settings property sheet on the appliance instance and configuring each volume placeholder with the name of its respective application volume.
  • the user drags a blank composite appliance onto the editor canvas, thereby creating a singleton composite class with an automatically generated name and an instance of that class. The user then selects the newly created instance and opens the boundary editor on its class.
  • the user defines the new class by selecting an appropriate name for it, and defining its terminals, properties and volume placeholders, as desired.
  • a new editor window opens providing a canvas for defining the interior, on which the terminals of the composite have already been placed.
  • the user creates the desired structure, by: (a) adding appliance instances by selecting appropriate appliance classes from a catalog and dragging them on the canvas, (b) configuring each instance through the instance settings property sheet, and (c) connecting the terminals of the instances and the terminals of the composite into the desired structure.
  • an input terminal of the composite behaves as an output (e.g., it is connectable to exactly one input of a subordinate appliance), and an output terminal of the composite behaves as an input (e.g., multiple outputs of various subordinates may be connected to it).
  • the user redirects properties and/or volumes of the composite to properties and/or volumes of one or more subordinates, by referencing them in configuration of the instance settings of the subordinates as described above.
  • a virtual appliance or a composite appliance can be dragged onto one of the available catalogs to create a catalog class from which multiple instances can be created.
  • the act of dragging the appliance onto the catalog converts the singleton into an identically named catalog class, includes that class in the package of the desired catalog, and deletes the instance used to create and edit the new appliance.
  • application volumes that are configured as class volumes of the new class are converted into instantiable class volumes by the inventive system and removed from the list of application volumes accessible by the user.
  • the inventive system preferably implements an application as a combination of a package descriptor, a singleton composite appliance named "MAIN", and an optional catalog. Assuming that all required appliance classes already exist in one or more available catalogs, assembling the application is equivalent to creating the interior of the MAIN composite.
  • the MAIN composite preferably has no terminals, since the application is expected to interact with the rest of the computer network through one or more virtual network adapters defined on one or more instances of virtual appliances included in the application. Such interactions may be carried out by means of standardized input and output "gateway" appliances, thereby isolating most of the application from having to know the details and settings of such interactions.
  • the act of creating an application in general comprises an iterative process that combines top-down decomposition of the desired functionality into subsystems, which are expressed as composite appliances, with the bottom-up assembly of subsystems and other composites from available appliance classes.
  • creating a new virtual appliance class is required to best express a sub-function of a given subsystem; in this case the appropriate class is created either from scratch or, more often, by branching and customizing an existing appliance class.
  • the design of the new application is complete when the MAIN singleton is fully assembled and all subordinates included in it exist and are properly configured.
  • the set of descriptors and volumes that comprises the application designed as the present invention teaches contains all necessary software packages, data, configuration parameters and structural information required to create a running instance of the application under the control of the inventive system.
  • any subset of the application being it a single virtual appliance, an incomplete structure of virtual appliances, a finished application subsystem such as a database cluster or a web tier, or an application that is not completely configured, can be started on the inventive system subject only to the software packages included in the existing virtual appliances having sufficient configuration and connectivity to operate.
  • the application is a hierarchical structure of composite appliances and is itself a composite appliance
  • it is beneficial to design the application so that any properties, volumes or attributes that may be desired to change when deploying the application on different systems and locations, are exposed as properties, volumes and attributes of the application (e.g., of the MAIN composite).
  • Troubleshooting and Monitoring When executing an application built using the present invention the inventive system constructs the running image of the application from virtual resources, using structural and configuration information captured in virtual appliances and composites.
  • This way of deploying and executing applications has a significant added benefit in that all structural information captured throughout design and development is available to the system at run time. This makes it easy to correlate monitoring data captured as the application runs with the logical structure of the application, and significantly simplifies the process of troubleshooting and tuning applications and monitoring the execution in production by making it intuitive.
  • FIGURE 38 illustrates the monitoring and troubleshooting user interface in an example embodiment.
  • each virtual appliance is dedicated to serving a particular function within the application; monitoring the resource usage of the appliance, such as CPU load, memory and bandwidth, provides an excellent indication about the operation of that function.
  • it is easy to design virtual appliances so that each terminal represents a distinct incoming or outgoing logical interaction; the result of such design being that most, if not all, connections within the application structure represent distinct logical interactions between different functions in the application. Since each terminal is preferably constrained to a specific connection and protocol type, it is easy to interpret the traffic along any connection to determine key characteristics such as requests per second and response time per request.
  • All of this monitoring data pertains to individual virtual appliances, connections or terminals, and can be easily overlaid on the visual layout of the application structure.
  • the inventive system presents the user with a live view of the application design, reflecting the state, the load and communication patterns of the application as they develop.
  • the inventive system also provides easy means to define thresholds of normal behavior on appliance instances and connections, and detect and display abnormal behaviors on the visual layout of the application. This enables the user to formulate and execute corrective actions directly in the terms of the application logic rather than having to continuously translate such actions into the terms of the physical infrastructure on which the application executes. Change Management and Version Control
  • the present invention enables a simple and effective approach to change management in distributed applications by making it possible to apply technology that is well understood and proven over decades of use to the problem.
  • the inventive system captures the complete structure and configuration of the application, including installed images of operating systems, application software, configuration files, network settings, scripts and user data, sufficient to execute the application on any instance of the inventive system, and retains this data in the form of collection of text files (descriptors) and logical volume images.
  • This makes it possible to use a commercial version control system developed for use in software code development, such as ClearCase or Microsoft Visual SourceSafe, to effectively implement version control of distributed applications during design and development, as well as for change management in the later stages of application delivery and deployment.
  • the disclosed visual method makes it possible to construct distributed applications of arbitrary complexity by visually defining a model of the target application that is simple and yet sufficiently complete to allow the inventive system to deploy and execute the application on a variety of target hardware without any further human intervention. This greatly simplifies all activities related to designing, constructing, deploying and managing large distributed applications by eliminating the need for constant manual translation from application logic to hardware configuration and vice-versa. 4. EXAMPLE SYSTEM EMBODIMENT
  • the present invention includes a system that implements the necessary support for the abstractions defined in the application model and for practicing the visual method.
  • the system provides runtime support for deploying, executing, monitoring and managing applications constructed as the present invention teaches.
  • FIGURE 39 illustrates the architecture of the inventive system.
  • the system comprises a system controller 3900 and one or more servers 3910 and/or one or more server blades 3920.
  • the system may include a storage area network (SAN) 3940, in which case one or more of the servers, such as the servers 3930 would act as gateways providing access to the SAN 3940 for the rest of the system. All nodes in the system are interconnected through the network 3950 which is assumed to have sufficient bandwidth to carry the operation of the system.
  • the servers 3910 may have hard disks or other storage media attached to them, while the server blades 3920 may be diskless.
  • all elements of the inventive system reside on a single server such as 3910, and use the storage attached directly to the server.
  • Servers 3910 and blades 3920 are configured to boot a suitable host operating system and a virtual machine manager 3980 or 3981, which enables them to be partitioned into multiple virtual machines 3911.
  • those servers are configured to execute a virtual resource manager 3970 or 3971, which interacts with the controller 3900.
  • the inventive virtual resource manager implements support for virtual network interfaces 3990 and 3991, and for virtual storage volumes 3960 and 3961, sufficient to implement the application model.
  • each virtual resource manager 3970 controls its local virtual machine manager 3980 and extends its functionality as may be necessary to provide sufficient support for the application model.
  • the virtual resource manager 3970 makes the hardware resources of the server available to the controller 3900 as three distinct pools of virtual resources, including virtual machines 3911, virtual network interfaces 3990 and virtual volumes 3960.
  • the server blade 3920 has no storage and so the virtual resource manager 3971 is configured to make its resources available to the controller 3900 as two pools of virtual resources: virtual machines 3921 and virtual network interfaces 3922.
  • the servers 3930 are configured to provide access only to the storage resources of the SAN 3940. Accordingly, they do not have a virtual machine manager and their local virtual resource manager 3972 interfaces with a suitable SAN management system 3963 to provide a pool of virtual volumes 3960 and 3963 which are physically located on the SAN 3940 and accessed via a FibreChannel interface 3964.
  • the controller 3900 can access all servers 3910, 3920 and 3930 over the network 3950 and can, therefore, create, control and access virtual machines, virtual volumes and virtual network interfaces, as applicable, on any and all of the above servers.
  • the controller includes a resource aggregator 3901, an execution control module 3901 and a user interface system
  • the resource aggregator 3901 provides unified access to the virtual resources exposed by the servers 3910, 3920 and 3930, creating thereby three uniform distributed and scalable pools of virtual resources, one for each type of resource.
  • the resource aggregator preferably abstracts the actual location (e.g., server) on which each instance of a virtual resource resides from the rest of the controller, and also preferably manages the creation of such resources, determining on which server to create each particular resource instance and interacting with that server to this purpose.
  • the execution control module 3902 uses the resource aggregator 3901 to create, access and manage virtual resources. It provides runtime support for the application model allowing virtual appliances to be instantiated, configured, interconnected, started, executed, migrated from one server to another and monitored. In addition, the execution control module provides the necessary support for composite appliances and applications.
  • the execution control module 3902 may further interface with external software, making such application available for management by conventional data center management software, and forwarding alerts and other events related to the running application to such software.
  • the user interface system 3903 has two key functions: (a) it implements command line and visual interface to the execution control module and the rest of the inventive system, and (b) it implements the visual user interface (editors) for practicing the method taught by the present invention.
  • FIG. 2A shows an example embodiment of a Cloudware System 200 which, for example, may be used to provide various types of cloudware-related functionality described herein.
  • CUI Cloudware User Interface (CUI 220) Subsystem
  • the Cloudware User Interface subsystem provides interface to the Cloudware service.
  • the interface may be both for human users and for programmatic use.
  • the CUI subsystem 220 comprises the portal application (e.g., Portal 222), the account shells (e.g., Shell(s) 224) and the API gateway(s) (e.g., API(s) 226).
  • portal application e.g., Portal 222
  • account shells e.g., Shell(s) 22
  • API gateway(s) e.g., API(s) 2236
  • the Portal application provides the common, non-account- specific portion of the Cloudware web site. Its functionality comprises: service home page, new user registration, account creation and management, billing, service login, forums, documentation, support helpdesk, corporate site and brochures, etc.
  • the portal application may also be responsible for activating account shells, for example either at account creation and during login.
  • Portal makes Shell(s) instances and may also make instances of API(s).
  • the Shell(s) account shell provides the account user interface, including list of applications, infrastructure editor, monitoring, application and appliance control, etc — pretty much the current grid controller user interface of AppLogic(TM).
  • the Shell(s) application may be named this way due to its similarity to a "shell/desktop" in a traditional computer operating system - this may be the face of the "global computer” for any particular user.
  • the Shell(s) application may be designed to be instantiated per account (one instance per account); other options obviously include one Shell(s) app for the service (maybe scalable), as well as one Shell(s) instance per user login (like a shell in a traditional OS). Shell(s) ideally provides both GUI and text-based shell.
  • the API(s) gateway provides programmatic access to
  • Cloudware services It provides more or less the same set of services and functions as the Shell(s), allowing programmatic access/control to the same functions that humans use the Shell(s) shell to achieve.
  • the API(s) gateway provides a web-services interface (e.g., via SOAP).
  • the CUI subsystem may also be responsible to ensure a secure and authenticated channel between arbitrarily located end-user (e.g., anywhere on the
  • the intended measures may include, but are not limited to, one or more of the following (or combinations thereof):
  • RFC4346 connection security for all or selected secure portal pages (login, account/billing info, etc.) as well as all or selected pages of the account shell(s).
  • Server-side certificates signed by a well-known authority can be used to mitigate the dangers of man-in-the middle and similar attacks. Wildcard SSL certificates can be used, for example, to protect the Shell(s) pages (otherwise, there may be a need for a separate certificate issued by a known authority for each instance of Shell(s) - which may not be cost-effective and may delay account creation until a trusted cert can be issued; another option may be to provide signed certifications ("certs").
  • SSL-based security for the API(s) gateway functions all or selected programmatic interactions with the Cloudware service may occur over SSL. At least server-side certs may be used. Client-side certs may also be used instead of user name and password.
  • SSH Secure shell
  • All or selected SSH connections may preferably use SSH key-based authentication instead of user names and passwords.
  • Text-based shell(s) may also be provided via a web browser, for example, using the AppLogic web-based shell.
  • the CUI subsystem may not require persistent state (except, possibly cached data). It also may not drive composite operations - for example, application migration may be handled by the kernel, with the CUI initiating and then reporting progress of the operation.
  • the Cloudware service may be fully operational if the CUI subsystem may be inoperative for short periods of time - for example, everything may work except it may not be controllable (e.g., temporarily).
  • the CUI subsystem works as a set of AppLogic applications on one or more grids, in one or more data centers.
  • KERNEL Cloudware Kernel
  • the Cloudware Kernel subsystem may provide core controlling functionality of Cloudware. For example, in one embodiment, it may be responsible for performing all or selected operations that define various Cloudware services/resources.
  • the KERNEL subsystem comprises the service controller (e.g., Controller 206), metering system (e.g., METER 212), authentication service (e.g. Authentication 216), data center manager (e.g., DC Manager 214), and repository (e.g., Repository 218).
  • service controller e.g., Controller 206
  • metering system e.g., METER 212
  • authentication service e.g. Authentication 216
  • data center manager e.g., DC Manager 214
  • repository e.g., Repository 2128.
  • the Repository stores all or selected metadata for the Cloudware service, such as, for example, one or more of the following (or combinations thereof): account structure, users and their permissions, applications, catalogs, etc. It may be a hierarchical repository, organized by entity, as described elsewhere in the Cloudware docs. Repository may be highly available and replicated geographically for disaster recovery. The Cloudware service can survive Repository restart with only temporary loss of controlling services and without impacting running applications (this may also be true for other subsystems and/or components of Cloudware described herein).
  • the Repository may be implemented using an lighweight directory access protocol (LDAP) implementation, such as OpenLDAP (at www.openldap.org), including its directory replication mechanisms for achieving redundancy and geographic distribution.
  • LDAP lighweight directory access protocol
  • the DC Manager 214 may be responsible to manage the set of data centers and grids in them, as well as their relationships, associated resources (such as IP addresses), and metadata (e.g., resource prices and costs). It aggregates the multiple data centers and grids, and provides a secure and reliable channel to them. All or selected interactions except volume transfers may pass through the DC Manager.
  • DC Manager may be configured or designed to not store persistent data. In such embodiments, DC Manager may use the Repository for all or selected storage needs (including storage of transient states).
  • the metering system 212 may be responsible for tracking all or selected resource usage, such as, for example, one or more of the following (or combinations thereof): CPU time, memory, storage, bandwidth, licensed appliances, etc.
  • the metering system may also be operable to timely (e.g., real time) report resource usage information to billing system 230.
  • the authentication service 216 may be responsible for authenticating users as well as Cloudware entities.
  • authentication provides authentication services for logging in users.
  • it also manages the user and account relationships in a directory service.
  • Authentication provides secure authentication between Cloudware subsystems and components, including between components that may be geographically distributed and may need to communicate securely over public/insecure networks. Authentication allows for maintaining a single, unified user login across all or selected content and services of AppLogic, so that users authenticate once and obtain access to all or selected aspects of their accounts, such as, for example: applications, billing, forums, helpdesk, etc.
  • the service controller 206 may be responsible for controlling various aspects relating to Cloudware resources/services/information. In one embodiment, it may implements all or selected operations and may provide the abstraction/entities defined by Cloudware. It may use other services to perform their appropriate functions, and may also use grids to operate and/or manage applications.
  • Controller 206 comprises various subsystems such as, for example, one or more of the following (or combinations thereof):
  • Worker Apps (209) - worker applications array.
  • Worker Apps may be an internal AppLogic application that the controller starts to perform specific operations that take long time and require a lot of resources (e.g., volume copy, migration, etc.)
  • Worker Apps actually represents a set of different application templates; Controller provisions the particular one it needs for the task and destroys it upon completion. It may also be possible to maintain a pool of pre- provisioned Worker Apps applications that Controller can allocate for tasks that it needs to perform. (This concept may be akin to the "helper" applications used in
  • the Worker Apps applications may not be visible/controllable/accessible to end-users directly. However, resources used by such internal worker applications may be preferably accrued to the user (definitely network transfer, possibly CPU/memory) •
  • Scheduler (208) application scheduler. Scheduler determines where an application may be placed - in which data center and on which grid — based on account preferences, user location, application configuration, resource settings, data center/grid capabilities and available/spare capacity. Scheduler does not deal with scheduling within a grid — that function may be responsibility of the grid itself (the same way as scheduling within a server may be responsibility of the server itself).
  • Grid Controller(s) (not shown) - operable to manage one or more grids in the
  • Cloudware network and/or Cloudware resources and/or services associated with such grids may reside locally at the same data center(s) as the grid(s) which they control.
  • portions of functionality of the Grid Controller(s) may be incorporated into DC Manager 214.
  • the KERNEL subsystem may be highly available and replicated for disaster recovery.
  • the applications running on the Cloudware service may be fully operational if the KERNEL is down for short periods of time - everything may work except it may not be controllable and some aspects of high availability may be delayed until the KERNEL subsystem is restored.
  • the KERNEL subsystem works as a set of AppLogic applications on one or more grids, in one or more data centers. Grids
  • Cloudware preferably operates end-user applications on a set of grids (e.g., AppLogicTM-based utility computing grids) located in multiple data centers. Each data center may have one or more grids, possibly with different dimensions (e.g., ratio of CPU cores to memory; size of storage per server, etc.)
  • Cloudware may use grids to host the Cloudware-specific components themselves - such as, for example, the CUI and KERNEL subsystems of the Cloudware System 200. It may also be possible to operate the external services or portion thereof on grids.
  • the grids used by Cloudware and/or Cloudware System for its operation may not be among the grids that Cloudware would schedule end-user applications. This allows the Cloudware service to be maintained with minimal impact on the end user applications.
  • Cloudware may eventually operate grids running different versions of utilizing computing grid software such as AppLogicTM. It may also be possible and likely to have grids with different versions in the same data center, as well as to have a single account that runs applications on grids with different versions concurrently (e.g., one app on AppLogic 2.5, another on 2.6).
  • Cloudware components use loose coupling for their interfaces, especially for the interface between the Cloudware KERNEL and the grids on which user applications operate.
  • this interface may be implemented using Simple Object Access Protocol ("SOAP") based functionality, and may fully utilize the flexibility provided by SOAP (including, for example, optional fields, must-understand attributes, forwarders, etc.).
  • SOAP Simple Object Access Protocol
  • the Cloudware System may include (and/or utilize) other external services to provide functions which, while not speecific to Cloudware, may be preferable for its operation.
  • other services include one or more of the following (or combinations thereof), which,::
  • a billing system e.g,. 230
  • a billing system which keeps user's (account's) billing information, history; issues invoices and facilitates money transfers: charging users for monthly and usage fees through a variety of mechanisms — credit card, account sweeps, etc.
  • the billing system also facilitates charging license fees, as well as crediting accounts with licensed appliance usage fees, paying hosting providers for the resources they provide in the service based on actual usage, etc.
  • the billing system includes at least an account/user portal (e.g., for the user to keep his/her information up-to-date, make payments, review invoices and payment history), as well as an accounting portal (e.g., for the Cloudware operator); in addition, it may have a provider portal (e.g., which may be used by hosting providers, appliance/application providers and other service providers to access their billings).
  • a globally accessible data storage system (e.g., 240), which keeps large pieces of data that may be desirable to be accessible to one or more accounts.
  • this may include the class volumes of catalog appliances - those may be treated by the service controller and the grids as a resource storage.
  • the storage system may be a BLOB store - it has no notion of what it stores - just a bunch of globally accessible and persistently/uniquely identifiable binary blocks (BLOBs).
  • Cloudware may work with multiple such systems concurrently, for example, Amazon S3 and The Grid Layer's DynaVol; Grids can also be used as a storage system.
  • the Cloudware System may include (and/or may be communicatively coupled to) a licensing management system (not shown) which, for example, may be operable to perform one or more of the following functions:
  • the Cloudware System may be operable to acquire (e.g., at the request of the user) the desired license(s) from the licensing entity (e.g., third-party publisher) on behalf of the user, and may be operable to coordinate and/or manage all billing/payment activities (e.g., relating to the acquired license for the virtual appliance) without the user ever having to deal directly with the third-party publisher/licensing entity.
  • the licensing entity e.g., third-party publisher
  • the Cloudware System may further be operable to track and report (e.g., to the user and/or third party licensing entity) usage information relating to the usage of the licensed virtual appliance by the user's application, which may be running at one or more different server grids.
  • At least a portion of the other services described above may be implemented as internal services (e.g., internal to the Cloudware system/network) and/or as external services (e.g., external to the Cloudware system/network).
  • Arbitrage mechanism e.g., included as part of controller 206 functionality for arbitrating among the needs (e.g., resource needs) of the various different Cloudware grid networks as well as the needs of the Cloudware System.
  • FIG. 3 shows an example embodiment of a graphical user interface (GUI) 300 which may be used for implementing various Cloudware related aspects/features.
  • GUI 300 may be implemented as a web page which may be accessible to users via conventional Web browsers such as Microsoft Internet Explorer, Mozilla Firefox, etc.
  • GUI 300 may correspond to a data center operator (DCO) profile page which may be associated with a particular DCO, namely NetClime, Inc.
  • the DCO profile page may be accessible to various entities or Cloudware customers such as, for example: data center operators (e.g, employees/agents of the DCO), end users, publishers (e.g., publishers of applications, appliances), etc.
  • data center operators e.g, employees/agents of the DCO
  • end users e.g., publishers of applications, appliances
  • publishers e.g., publishers of applications, appliances
  • DCO profile page 300 includes a variety of different types of content which may be related to or associated with NetClime 's data center. Examples of such content may include, but are not limited to, one or more of the following (or combinations thereof):
  • Resource metering content relating to the DCO' s resources/operations.
  • resource metering content may include one or more of the following (or combinations thereof):
  • Information e.g., 308a
  • the DCO' s currently available and/or total available resources such as, for example, computing resources, storage resources, bandwidth resources, etc.
  • Information e.g., 308b
  • Information relating to histories of available resources at the DCO over one or more time intervals (e.g., daily, weekly, monthly, yearly, average, min, max, etc.).
  • time intervals e.g., daily, weekly, monthly, yearly, average, min, max, etc.
  • graphical data may be used to convey a running history of the DCO's computing, storage and bandwidth resources of its data center over the past 7 days.
  • • Content e.g., 310) relating to various online forums.
  • at least a portion of the forums may relate to products and/or services provided by the DCO.
  • other portions of the forms may relate to topics which the DCO (and/or agents of the DCO) has subscribed to an/or selected for monitoring.
  • Content relating to the DCO's various account settings such as, for example, billing history, billing information, account options, etc.
  • some or all of the DCO's account settings content may be flagged as private, and only viewable to authorized DCO employees/agents.
  • Content relating to various types of network accessible virtual appliances and/or applications such as, for example, one or more of the following (or combinations thereof): application catalogs 314a, applications 314b, appliance catalogs 314c, appliances 314d, etc.
  • Content relating to various user (e.g., 316b) and/or user accounts (e.g., 316a).
  • the account content 316b may include user account information relating to accounts created for users who may be authorized by the DCO to have various types of privileges/access.
  • Content e.g., 318, relating to messages associated with the DCO and/or associated with one or more DCO employees/agents.
  • the content 302-318 may be editable by the account owner, so that the account owner can modify what appears on the profile page 300 for the account owner, as well as for other users of the service.
  • a data center may have a plurality of different types of resources associated therewith. Examples of such resources may include, but are not limited to, one or more of the following (or combinations thereof):
  • VPN Virtual Private Network
  • the DCO may charge their customers on a per-resource basis, wherein a customer may be charged fees based on the various resources which that customer uses. For example, in one embodiment, a customer may be charged a separate fee each time the customer (or the customer's application, which may be being hosted at the data center) makes use of one or more specified data center resources.
  • the pricing structures of various fees for data center resource utilization may be based upon a variety of different criteria such as, for example, one or more of the following (or combinations thereof):
  • Time based fees such as, for example, charges for use of a resource (or given group of resources) for a specified time period (e.g., $1 for every 1 CPU-hour of use);
  • Quantity based fees such as, for example, charges for use of a resource (or given group of resources) for a given quantity (e.g., $1 for every 1 GB of data transfered);
  • Operation based fees such as, for example, charges for one or more operations and/or transactions relating to a given resource (or given group of resources) (e.g., $1 for every data base access request, $1 for every n transactions performed, etc.);
  • One-time fees such as $100 for using a service (e.g., for a given time period such as, for example, a particular month); • Per-seat fees, such as $1 for each user accessing the service (e.g., management and monitoring services);
  • Reservation based fees such as, for example, charges to ensure a minimum resource availability (e.g. $1000 per month to ensure the ability to access up to 20 CPUs simultaneously at any time);
  • License fees for software that the DCO makes available to customers running in their datacenter e.g., Microsoft Windows licenses, published appliance and application licenses
  • special pricing e.g., lower price than the typical or retail published price of the software (e.g., appliance and/or applications)
  • a DCO may group different resources together to offer one or more bundled groups of resources for specified fees.
  • a packaged or bundled group of resources may be referred to as a "virtual resource bundle" (VRB).
  • VRB virtual resource bundle
  • a DCO may offer a customer (e.g., for a specified fee) a virtual resource bundle which, for example, may include: 1 CPU, 1 GB RAM, 250 GB storage, and 125 Mbps bandwidth.
  • the customer may be presented with a single fee arrangement for use of the entire virtual resource bundle (e.g., for a specified duration of time).
  • a virtual resource bundle is a BCU.
  • a BCU may refer to a "basic computing unit.”
  • a BCU may be defined as having a fixed or predetermined amount of computing resources.
  • a BCU may be defined to include a CPU core (e.g., the equivalent of a 1.86 MHz single core CPU) and RAM (e.g., the equivalent of 1 GB RAM).
  • a BCU may be defined to include other combinations of resources such as, for example, one or more of the following (or combinations thereof): CPU(s), RAM, storage (e.g., 250 GB disk storage), bandwidth (e.g., 500GB transfer), etc.
  • different portions of the resource metering content may represent different types of DCO resources such as, for example, one or more of the following (or combinations thereof):
  • the data center resource metering content may be displayed to a user via a graphical user interface.
  • one or more application catalogs (e.g., 314a) and/or applications (e.g., 314b) may be published (e.g., for display on DCO profile page 300) by the DCO, by user(s), and/or by appliance publishers (e.g., which have an affiliation or relationship with the DCO).
  • one or more application catalogs may include pre- configured sets of applications (e.g., organized according various criteria such as, for example, theme, functionality, etc.) for use in designing and/or implementing distributed applications, for example.
  • application content portion 314b may be operable to display a customized list of user selected/preferred applications.
  • one or more appliance catalogs may include pre- configured sets of appliances for use in designing and/or implementing distributed applications, for example.
  • content/information relating to one or more of the appliance catalogs may be globally accessible, for example, via the Cloudware System and/or WAN.
  • the appliance catalogs (and related content) may be stored in a centralized location, which may be accessible to all (or selected) data centers and/or users (e.g., via the Cloudware System). Accordingly, in at least one embodiment, the appliance catalogs which may be accessible via different data centers may be standardized across the multiple different data centers.
  • the content/relating to one or more appliance and/or application catalogs may be available globally using one or more of the following approaches: • a distributed global store, such as, for example via the use of a dispersed storage technology such as www.cleversafe.org, a peer-to-peer file sharing network such as Bittorent, a global file system such as Google's Global File System or RedHat GFS, and/or a cloud storage service such as Amazon S3; • a content delivery network (such as, for example, Akamai or Limelight
  • appliances that are recently used on a particular grid or datacenter may be cached on that grid or datacenter; appliances may be cached in a datacenter either on demand (e.g., when a first application tries to use the appliance) or pushed to the datacenter when the appliance becomes available.
  • the delivery and caching of catalog appliances and applications may be preferably handled using a common approach.
  • appliance content 314d may be operable to display a customized list of user selected/preferred appliances.
  • one or more of the forums may be organized around various types of infrastructure (such as, for example, cloudware related infrastructure, AppLogicTM related infrastructure, etc.).
  • infrastructure such as, for example, cloudware related infrastructure, AppLogicTM related infrastructure, etc.
  • Other examples of such infrastructure may include, but are not limited to, one or more of the following (or combinations thereof):
  • a user may click (e.g., right click) on an icon of a specific application or appliance in order to access one or more specific forums relating to that specific application/appliance.
  • the user may be presented with a menu for accessing one or more online forums which may be related to the selected application/appliance.
  • One benefit of this approach may be that the user can submit the question or feedback directly with the entity to which it applies (and/or who is responsibe for servicing such questions and/or feedback), without having to figure out which company published or supports the entity.
  • GUI 400 may be implemented as a web page which may be accessible to users via conventional Web browsers such as Microsoft Internet Explorer, Mozilla Firefox, etc.
  • GUI 400 may correspond to a profile edit page relating to data center operator (DCO) such as, for example, NetClime, Inc.
  • DCO data center operator
  • the DCO profile edit page may be accessible to various users (e.g., selected DCO employees/agents) who may be provided with sufficient authorization/privileges to access the DCO profile edit page.
  • DCO profile edit page 400 includes a variety of different types of content which may be related to or associated with NetClime' s data center.
  • Portions of the content illustrated in the example DCO profile edit page of Figure 4 may be similar to corresponding portions of content illustrated in the example DCO profile page 300 of Figure 3, and therefore will not be described in greater detail.
  • various portions of content (e.g., 450) illustrated in the example DCO profile edit page of Figure 4 may be edited and/or modified by an appropriate user (such as, for example, a DCO employee/agent).
  • the content of Figure 4 may be an alternative representation of the content of Figure 3.
  • other portions of content e.g., 410, 440, 430-438, etc.
  • a NetClime employee/agent may be given permission to perform a variety of different editing operations such as, for example, one or more of the following (or combinations thereof): • editing/modifying the display and/or access to various application content (e.g.,
  • resource metering content e.g., 436
  • GUI 500 may be implemented as a web page which may be accessible to users via conventional Web browsers such as Microsoft Internet Explorer, Mozilla Firefox, etc.
  • GUI 500 may correspond to a virtual appliance profile page which is associated with a given virtual appliance.
  • Examples of various virtual appliances may include, but are not limited to, one or more of the following (or combinations thereof): • Apache Web Server
  • virtual appliance profile page 500 is associated with a Simple Web Server virtual appliance, which, for example, may provide functionality for implementing a Web Server application.
  • an entity which creates a customized virtual appliance may publish the customized virtual appliance (and/or other information relating to the customized virtual appliance) to one or more Cloudware Appliance catalogs.
  • the virtual appliance profile page may be accessible to various entities or Cloudware customers such as, for example: data center operators (e.g, employees/agents of the DCO), end users, publishers (e.g., publishers of applications, appliances, etc.), etc.
  • data center operators e.g, employees/agents of the DCO
  • publishers e.g., publishers of applications, appliances, etc.
  • a user NetClime as illustrated at 501
  • at least a portion of the content of virtual appliance profile page 500 has been dynamically generated for that particular user.
  • virtual appliance profile page 500 includes a variety of different types of content which may be related to or associated with the particular virtual appliance (e.g., Simple Web Server virtual appliance). Examples of such content may include, but are not limited to, one or more of the following (or combinations thereof): • Content (e.g., 502) relating to a name of the virtual appliance.
  • Content e.g., 506 relating to reviews and ratings of the virtual appliance.
  • Content e.g., 508 relating to typical usage of the virtual appliance (which, for example, may include various descriptions and/or drawings relating to the virtual appliance).
  • Content e.g., 510 relating to other documentation which may relate to the virtual appliance (such as, for example, software installed in the virtual appliance, list of attributes and properties, resource requirements for usage, etc.).
  • Content e.g., 512
  • statistics associated with the virtual appliance such as, for example, number of copies in use, average/min/max resources provisioned, MTBF, average downtime, etc.
  • • Content e.g., 514.
  • at least a portion of the forums may relate to aspects/features of the virtual appliance.
  • the virtual appliance profile page 500 may include a summary portion (e.g., 520) which may include content which provides a summary of various aspects and/or features relating to the virtual appliance. Such summary content may make it easier for users to choose an appliance and/or to dynamically compare features of similar type appliances.
  • summary portion e.g., "At A Glance”
  • 520 may include a variety of different types of information relating to the virtual appliance such as, for example, one or more of the following (or combinations thereof):
  • Documentation information e.g., 522 relating to the virtual appliance.
  • portion(s) of the documentation information may also be accessible under the Appliance Documentation (510) portion and/or other portions of the virtual appliance profile page 500.
  • at least a portion of the documentation information may be provided by the creator of the virtual appliance and/or may be automatically determined and/or provided by the Cloudware System.
  • the appliance documentation information indicates that the virtual appliance has the following properties/characteristics :
  • At least a portion of the virtual appliance uses statistics information may be automatically determined, tracked, generated and/or provided by the Cloudware System.
  • the virtual appliance uses statistics information may include, but are not limited to, one or more of the following (or combinations thereof) properties/characteristics :
  • the Cloudware System may be operable to track operational data (e.g., including failure data) relating to all (or selected) instances of the virtual appliance at all (or selected) data centers of the Cloudware network, and may be further operable to use at least a portion of the operational data to dynamically and/or automatically calculate or determine a current MTBF value for the virtual appliance.
  • operational data e.g., including failure data
  • the current estimate of the MTBF value for an instance of the Simple Web Server virtual appliance is 553.5 hours (e.g., meaning that, on average, it is anticipated that a failure may occur for an instance of this virtual appliance about once every 553.5 hours).
  • the Cloudware System may identify different instances of the virtual appliance running at one or more data centers of the Cloudware, and may analyze various types of connectivity information relating to the various virtual appliance instances.
  • the Cloudware System may identify other appliances which are connected to different instances of the virtual appliance, and/or may identify patterns of connectivity, such as, for example, which of the other appliances are most commonly connected to a given instance of the virtual appliance.
  • the Cloudware System may automatically and/or dynamically generate information (e.g., 526) relating to other appliances and/or applications which may be typically related to or associated with the virtual appliance.
  • information e.g., 526
  • the appliances MySQL and LoadBalancer are most commonly connected to (or used in association with) instances of the Simple Web Server virtual appliance.
  • the appliances MySQL and LoadBalancer may been identified at 526 as being related to the Simple Web Server virtual appliance.
  • all or selected portions of the information and/or content provided in summary portion 520 may be automatically updated in real-time. In other embodiments, all or selected portions of the information and/or content provided in summary portion 520 may be automatically updated at periodic intervals (e.g., daily) and/or upon user request.
  • a user may access the virtual appliance profile page GUI 500, for example, by clicking (e.g., right clicking) on an object or image representing the virtual appliance (such as, for example, graphical image 503) which, for example, may be displayed in one or more other GUIs described or referenced herein.
  • an object or image representing the virtual appliance such as, for example, graphical image 503
  • one unique feature of the Cloudware network is the ability of the Cloudware System to monitor, track, analyze, and/or process (e.g., in real-time) various types of operational and/or configuration information relating to all (or selected) instances of virtual appliances and/or applications across all (or selected) data centers of the Cloudware network.
  • Another unique aspect is the ability of the Cloudware System to automatically and/or dynamically generate (e.g., in real-time) compiled information/content (such as, for example, Appliance Usage Statistical Information, Related Appliance/Application Information, etc.) relating to various virtual appliances, applications and/or other aspects relating to the global Cloudware network.
  • compiled content may be based, at least in part, upon actual or existing uses of virtual appliances and/or applications across all (or selected) data centers of the Cloudware network.
  • the Cloudware System is able to analyze various types of operational and/or configuration information relating to all (or selected) instances of virtual appliances and/or applications across all (or selected) data centers of the Cloudware network, and is further able to use the analyzed information to automatically and dynamically generate one or more of the following (or combinations thereof) types of information (which, for example, may correspond to real-time information):
  • Related Appliance/ Application Information relating to other appliances and/or applications which may be identified (e.g., based on existing usage statistics/paterns/analysis) as having a relationship or association with the virtual appliance.
  • this may include appliances that are typically used in conjunction with the particular appliance (e.g., a MySQL database server, a load balancer, a firewall, etc.).
  • the content may further provide information on how related appliances are typically connected: for example, that the MySQL virtual appliance is frequently connected to the WEB server virtual appliance by having WEB servers's DB terminal connected to MySQL's IN terminal.
  • this information can be further used when creating infrastructures for an application, by automatically placing, or proposing to place the related appliance and connect it as it is most frequently connected (and/or based on various other selected criteria).
  • Information relating to types and/or quantities of resources e.g., number of CPUs, memory size, number and sizes of storage volumes, etc.
  • resources e.g., number of CPUs, memory size, number and sizes of storage volumes, etc.
  • such information may be based, at least in part, upon actual or existing uses of virtual appliances and/or applications across all (or selected) data centers of the Cloudware network.
  • Such information may also be represented in a graphical form, as well as include details, such as resource distributions by count (e.g., that 50% of appliance instances use between 256 and 512 MB RAM, while 10% of instances use above 2 GB RAM).
  • Information relating to start/stop time of one or more selected virtual appliances and/or applications • Information related to frequency of available updates or new versions of the appliance; information related to activity on the forums and other aspects of the appliance, such as, for example, to allow users to evaluate the available level of support for the appliance;
  • GUI 600 shows an example embodiment of another graphical user interface (GUI) 600 which may be used for implementing various Cloudware related aspects/features.
  • GUI 600 may be implemented as a web page which may be accessible to users via conventional Web browsers such as Microsoft Internet Explorer, Mozilla Firefox, etc.
  • GUI 600 may correspond to a virtual appliance expanded profile page which is associated with a given virtual appliance.
  • virtual appliance expanded profile page 600 is associated with a Simple Web Server virtual appliance, which, for example, may provide functionality for implementing a Web Server application. Accordingly, in at least one embodiment, at least a portion of the content of the virtual appliance expanded profile page 600 of Figure 6 may be similar to the content of virtual appliance profile page 500 of Figure 5.
  • the virtual appliance expanded profile page may be accessible to various entities or Cloudware customers such as, for example: data center operators (e.g, employees/agents of the DCO), end users, publishers (e.g., publishers of applications, appliances, etc.), etc.
  • data center operators e.g, employees/agents of the DCO
  • publishers e.g., publishers of applications, appliances, etc.
  • Figure 6 it is assumed that a user has accessed the Cloudware System, and that at least a portion of the content of virtual appliance expanded profile page 600 has been dynamically generated for that particular user.
  • virtual appliance expanded profile page 600 includes a variety of different types of content which may be related to or associated with a given customized virtual appliance (e.g., Simple Web Server virtual appliance). Examples of such content may include, but are not limited to, one or more of the following (or combinations thereof):
  • boundary information which may be associated the virtual appliance.
  • at least a portion of the boundary information may include functional specification information relating to the virtual appliance.
  • Content e.g., 617) relating to content setup details and/or share file storage details associated with the virtual appliance.
  • Content e.g., 618, relating to typical or recommended usage details associated with the virtual appliance.
  • the boundary content 610 may include one or more of the following type of information (or combinations thereof):
  • Resource information relating to various resource usage recommendations and/or requirements associated with the virtual appliance.
  • the resource information may include MAX, MIN and/or DEFAULT parameters relating to various types of resources associated with the virtual appliance such as, for example, one or more of the following (or combinations thereof): CPU resources, memory resources, bandwidth resources, storage resources, interface resources, etc.
  • Terminal information e.g., 614 relating to various types of terminals (and/or related parameters) associated with the virtual appliance.
  • terminal information may include descriptive information (such as, for example, Terminal Name, Terminal Direction, Terminal Protocol, Terminal Description/Functionality, etc.) relating to various types of terminals and/or interfaces associated with the virtual appliance.
  • Property information relating to the various configurable properties associated with the virtual appliance may include descriptive information, such as, for example, Property Name, Property Type (string, numeric, IP address, etc.), Description, Default Value, Constraints (e.g., such as whether the property is Mandatory, Read-only, etc.).
  • Volume information relating to the various volumes associated with the virtual appliance Such information may include volume name, description, size and file- system type constraints, etc.
  • Statistics Counters relating to the various performance and operation statistics counters provided by the appliance to aid monitoring, performance tuning and troubleshooting, for example. Such information may include, for example, counter name, description, unit of measure, frequency of updates, etc.
  • the virtual appliance expanded profile page 600 may include usage details, such as the memory usage content (616), setting up the content and shared file storage (617), etc.
  • the page 600 may include detailed typical usage info (618) which provides example usage of the appliance in various application use cases. This information may include, for example, graphical representation of the infrastructure of such application, textual description of the purpose and specialization of the usage case, as well as details on the role, configuration and/or limitations of the appliance in such use case.
  • the page 600 may include additional notes (619), as well as links and references to other appliances and other documents that may be useful in conjunction with the appliance.
  • the virtual appliance expanded profile page 600 may include a summary portion (e.g., 620) which may include content which provides a summary of various aspects and/or features relating to the virtual appliance.
  • summary portion e.g., "At A Glance”
  • 620 may include a variety of different types of information relating to the virtual appliance such as, for example, one or more of the following (or combinations thereof):
  • Documentation information relating to the virtual appliance may be similar to portion(s) of the documentation information (522) described in the example embodiment of Figure 5.
  • Appliance Usage Statistical Information (e.g., 624) relating to usage statistics relating to the virtual appliance.
  • Appliance Usage Statistical Information e.g., 624
  • at least a portion of the Appliance Usage Statistical Information may be similar to portion(s) of the Appliance Usage Statistical Information (524) described in the example embodiment of Figure 5.
  • the virtual appliance uses statistics information may include, but are not limited to, one or more of the following (or combinations thereof) properties/characteristics:
  • MTBF Mean time between failure
  • a user may access the virtual appliance expanded profile page GUI 600, for example, by clicking (e.g., right clicking) on an object or image representing the virtual appliance which, for example, may be displayed in one or more other GUIs described or referenced herein.
  • GUI 700 shows an example embodiment of a graphical user interface (GUI) 700 which may be used for implementing various Cloudware related aspects/features.
  • GUI 700 may be implemented as a web page which may be accessible to users via conventional Web browsers such as Microsoft Internet Explorer, Mozilla Firefox, etc.
  • GUI 700 may correspond to a user dashboard page which is associated with a particular Cloudware user (or customer).
  • the current logged in user's ID is "Joe User”.
  • the user dashboard page may be accessible to various entities or Cloudware customers such as, for example: data center operators, end users, publishers (e.g., publishers of applications, appliances), etc.
  • entities or Cloudware customers such as, for example: data center operators, end users, publishers (e.g., publishers of applications, appliances), etc.
  • a user e.g., Joe User
  • Joe User has logged into the Cloudware System, and that at least a portion of the content of user dashboard page 700 has been dynamically generated for that particular user.
  • user dashboard page 700 includes a variety of different types of content which may be related to or associated with one or more applications which Joe User is running on the Cloudware network. Examples of such content may include, but are not limited to, one or more of the following (or combinations thereof):
  • System status content relating to one or more applications (e.g., associated with the user and/or assocated with the organization/account which the user belongs to) which are running on the Cloudware network.
  • the system status content may include a variety of different types of information such as, for example, one or more of the following (or combinations thereof):
  • User-related Application Information relating to one or more applications which are currently active or running in the Cloudware network (e.g., 9 applications currently running in Cloudware network which are associated with the user).
  • User-related Cloudware Resource Information relating to resources used by one or more of the user' s applications at one or more data centers in the Cloudware network.
  • the user-related cloudware resource information may include aggregated values relating to various types of resources used by one or more of the user's applications at one or more data centers in the Cloudware network. Examples of such User-related Cloudware Resource Information may include, for example: current (or real-time) total BCU resources being used, current (or real-time) total storage resources being used, current (or real-time) total bandwidth resources being used, etc.
  • System status information relating to the operational status of one or more user- related applications implemented in the Cloudware network.
  • Data Center Content relating to various data centers which are part of the Cloudware network.
  • the data center content may include a variety of different information such as, for example, one or more of the following (or combinations thereof):
  • different graphical objects may be used to represent different geographic data center locations throughout the global Cloudware network.
  • Information relating to data center resources may be used to represent different graphical objects (e.g., 704a, 704b) and different characteristics (e.g., shapes, sizes, colors, etc.), which, for example, may be used to represent relative resource availability at different data center locations throughout the global Cloudware network.
  • characteristics e.g., shapes, sizes, colors, etc.
  • 704a may indicate that the data center associated with object 704b has relatively more resources available to the user than the data center associated with object 704a.
  • different graphical objects e.g., 704a, 704b
  • characteristics e.g., shapes, sizes, colors, etc.
  • the relatively larger size of object 704b as compared to object 704a may indicate that more resources are being utilized by the user at the data center associated with object 704b than the resources being utilized by the user at data center associated with object 704a.
  • graphical objects e.g., 704a, 704b
  • characteristics e.g., shapes, sizes, colors, etc.
  • a data center object represented in the color green may indicate that all systems and applications are functioning normally
  • a data center object represented in the color yellow may indicate that some systems and/or applications have experienced errors in the past 24 hours
  • a data center object represented in the color red may indicate that one or more systems and/or applications are not functioning normally.
  • the user may select (e.g., click on) a specific data center object (e.g., 704a) to access additional information (e.g., resource information, status information, services, fees, etc.) relating to that data center.
  • additional information e.g., resource information, status information, services, fees, etc.
  • data center object 704a is represented using a red color, and that the user has clicked on (or hovered a pointer over) object 704a in order to access additional information relating to the data center status which, in this example, indicates that an application error has currently been detected at the data center relating to a "bugtracker2" application running at that data center.
  • Message Content e.g., 706
  • various messages may be generated by various entities of the Cloudware network such as, for example: the Cloudware
  • At least a portion of the messages may relate to various types of subject matter such as, for example, one or more of the following (or combinations thereof):
  • Application related subject matter e.g., relating to one or more of the user's applications which are running on the Cloudware network.
  • Virtual appliance related subject matter e.g., relating to one or more virtual appliances associated with one or more of the user's applications which are running on the Cloudware network).
  • Sales and/or support requests originated by the user or the user' s account • Other subject matter which may be of interest to the user and/or related to the user's activities in the Cloudware network.
  • Content relating to various types of network accessible virtual appliances and/or applications such as, for example, one or more of the following (or combinations thereof):
  • appliances • service catalogs and/or services available to the user
  • user dashboard page 700 may provide functionality for enabling the user to initiate various actions or operations such as, for example, one or more of the following (or combinations thereof):
  • Content for enabling the user to access various types of network information (such as, for example, various types of information relating to the Cloudware network and/or its resources).
  • network information may include, but are not limited to, one or more of the following (or combinations thereof): users, data centers, DCOs, people, publishers, organizations, applications, virtual appliances, catalogs, etc. Additional details relating to various types of network information which may be accessed by the user are further described with respect to Figure 12.
  • • Content e.g., 714) for enabling the user to access various types of messages and/or message related functionality. Additional details relating to messages and/or message related functionality which may be accessed by the user are further described with respect to Figure 12.
  • Content e.g., 750
  • region 750 of the GUI 700 may include graphical objects (e.g., 750a, 752a, 752b, 752c, etc.) and/or associated text which may be used to represent different instances of applications which may be instantiated at one or more data centers of the
  • the user may modify or arrange the display of the application objects (e.g., in region 750) as desired.
  • various different application objects e.g., 752a, 752b, 752c
  • 752a, 752b, 752c may be grouped together (e.g., via user manipulation and/or via automated mechanisms).
  • different graphical objects e.g., graphical objects
  • Application type e.g., application category type
  • Application status e.g., normal, stopped, standby, error, initializing, requires manual intervention, etc.
  • Application runtime e.g., cumulative runtime since application was instantiated and started
  • Application owner e.g., whether started by the current user or by another user with permissions on the account
  • Application availability to the user whether the user has permissions to perform various actions, such as start/stop/view/change/manage the application; whether the application is currently in use by another user;
  • • Content relating to searching/filtering functionality which, for example, may be operable to enable the user to initiate and/or perform searches/filtering for various types of applications using a variety of different search/filtering criteria such as, for example, one or more of the following (or combinations thereof):
  • Application state criteria e.g., 730a
  • Keywords criteria e.g., 730b
  • Time started • Resource usage (e.g., above 3 CPU, less than 200 MB RAM, more than 1 TB disk storage, etc.)
  • Application name or catalog template name e.g., TWiki
  • FIG 8 shows an example embodiment of graphical user interface (GUI) 800 which may be used for implementing various Cloudware related aspects/features.
  • GUI 800 may be implemented as a web page which may be accessible to users via conventional Web browsers such as Microsoft Internet Explorer, Mozilla Firefox, etc.
  • GUI 800 may correspond to an alternate embodiment of a user dashboard page which is associated with a particular Cloudware user (or customer).
  • the user dashboard page may be accessible to various entities or Cloudware customers such as, for example: data center operators, end users, publishers (e.g., publishers of applications, appliances), etc.
  • entities or Cloudware customers such as, for example: data center operators, end users, publishers (e.g., publishers of applications, appliances), etc.
  • Figure 8 it is assumed that a user has logged into the Cloudware System, and that at least a portion of the content of user dashboard page 800 has been dynamically generated for that particular user.
  • user dashboard page 800 includes a variety of different types of content which may be related to or associated with one or more applications which the user is running on the Cloudware network. Examples of such content may include at least a portion of the various content previously described and illustrated with respect to Figure 7.
  • portion 850 of the GUI 800 may include different types of content relating to various applications which may be instantiated at the Cloudware network.
  • region 850 includes an application information table which provides various types of information relating to different instances of applications which may be instantiated at one or more data centers of the Cloudware network.
  • the user may modify or arrange the display of the application objects (e.g., in region 850) as desired.
  • the user may elect to display and/or sort the information displayed in the application information table according to various criteria such as, for example, one or more of the following (or combinations thereof):
  • BCU usage criteria e.g., 808
  • storage/disk usage criteria e.g., 810
  • CPU usage criteria e.g., allocated and/or actual load
  • time-related criteria such as, for example, date/time application was last started, e.g., 814)
  • location criteria e.g., location of the data center(s) hosting the application
  • user information e.g., identity of user who last started or last modified the application; identity of user that is currently modifying or managing the application
  • a user may select a record or entry (e.g., 801) in the application information table in order to access additional information relating to the application associated with the selected entry.
  • a user may select entry 801 of the application information table in order to access additional information/features associated with the SiteKreator 2.0 application which, for example, is instantiated in the Orangeburg data center. Examples of the various types of additional information/features which may be accessed by the user are illustrated, for example, in Figures 9-11 of the drawings.
  • GUI 900 shows an example embodiment of graphical user interface (GUI) 900 which may be used for implementing various Cloudware related aspects/features.
  • GUI 900 may be implemented as a web page which may be accessible to users via conventional Web browsers such as Microsoft Internet Explorer, Mozilla Firefox, etc.
  • GUI 900 includes various types of content and/or features which are similar to the content/features described previously with respect to GUI 800 of Figure 8.
  • a user has selected entry 801 of the application information table in order to access additional information/features associated with the SiteKreator 2.0 application which, for example, is instantiated in the Orangeburg data center.
  • a secondary GUI e.g., Application Settings GUI 920
  • a secondary GUI is displayed to the user which enables the user to access and/or modify various settings relating to the selected application instance such as, for example, one or more of the following (or combinations thereof): • Resource settings (e.g., 902)
  • Advanced resource settings e.g., minimum/maximum resource ranges, defaults, disk storage capacity, bandwidth, etc.
  • Application details e.g., application name, description, tag fields, billing codes, unique ID, application group/category name, etc.
  • Application control settings e.g., field engineering code, debug/normal start mode, custom boot timeout values, appliance class/catalog version selections
  • the resources portion (e.g., 902) of the Application Settings GUI 920 may include various types of content (e.g., 910) relating to different types of resources (and their current parameter values) associated with the selected application instance.
  • Examples of such resource types may include, but are not limited to, one or more of the following (or combinations thereof):
  • computing resources e.g., CPU, memory and/or BCU
  • storage resources e.g., RAM, RAM, memory and/or BCU
  • the displayed resource information may include one or more of the following types of information (or combinations thereof):
  • Min/Max resource parameter values may be automatically and/or dynamically determined by the Cloudware System, for example, based on real-time resource availability data associated with the data center(s) which is currently hosting the selected application instance.
  • the user may use the GUI 920 to adjust or modify the settings of one or more resource types (e.g., by increasing or decreasing their current parameter values) in order to cause the identified resource types to be dynamically changed to a new parameter value as specified by the user.
  • the user may input the new desired computing resource parameter of 10 BCU (e.g., by sliding the computing resource button from 5 to 10), and click "save".
  • the Cloudware System may processes the user's resource modification instructions, and initiate appropriate actions to automatically and dynamically modify the computing resources associated with the identified SiteKreator 2.0 application instantiated in the Orangeburg data center in accordance with the user's instructions.
  • GUI 920 may be operable to allow the user to create multiple different application setting profiles (e.g., 903) for a given application. Additionally, in some embodiments, GUI 920 may be operable to allow the user to define one or more conditions and/or events (and/or other criteria) (e.g., as shown at 905) which may automatically trigger the application of specific application setting profiles for the given application upon the occurrence of such events/conditions.
  • GUI 920 may be operable to allow the user to create multiple different application setting profiles (e.g., 903) for a given application. Additionally, in some embodiments, GUI 920 may be operable to allow the user to define one or more conditions and/or events (and/or other criteria) (e.g., as shown at 905) which may automatically trigger the application of specific application setting profiles for the given application upon the occurrence of such events/conditions.
  • the user may define a first application setting profile (e.g., Profile 1) to serve as the default profile under normal operating conditions, and may define a second application setting profile (e.g., Profile 2) to be implemented upon the occurrence of one or more specified events/conditions (such as, for example, the occurrence of a primary power failure at the data center where the application is instantiated; or, as another example, when the application needs additional processing capacity, for example, during specified peak hours and/or based on actual real-time requirements/needs.
  • a first application setting profile e.g., Profile 1
  • Profile 2 e.g., Profile 2
  • specified events/conditions such as, for example, the occurrence of a primary power failure at the data center where the application is instantiated
  • additional processing capacity for example, during specified peak hours and/or based on actual real-time requirements/needs.
  • FIG. 10 shows an example embodiment of graphical user interface (GUI) 1000 which may be used for implementing various Cloudware related aspects/features.
  • GUI 1000 includes various types of content and/or features which are similar to the content/features described previously with respect to GUI 900 of Figure 9.
  • a secondary GUI e.g., Application Settings GUI 1020
  • Application Settings GUI 1020 is displayed to the user which enables the user to access and/or modify various settings relating to the hosted location(s) of the selected application instance.
  • the location portion of the Application Settings GUI 1020 may include various types of content relating to different data centers of the Cloudware network, such as, for example, one or more of the following (or combinations thereof): • Content relating to data center locations.
  • Content relating to data center resources such as, for example, one or more of the following (or combinations thereof):
  • GUI 1020 includes graphical object 1002 which represents the data center (and associated location) which is currently hosting or running the selected application instance.
  • Content relating to the recommended location for running the application (not shown), for example, based on application's resource requirements, traffic source, customer-specified objectives (quality, cost, communication latency, etc.);
  • GUI 1020 may show the available data centers in a list or table (not shown), and/or as a geographical map. It may also allow zooming in on specific regions, so that additional detail and resolution on the available data centers and their offerings (such as grids with different costs) may be selected by the user - similar in visual operation to speedtest.net, Google Maps, Google Earth, Microsoft Live Virtual Earth, etc. In another usage example, the user may select the continent, zoom in to the country, state/city, further zoom on a particular data center, and/or the zoom on a section of the data center with specific characteristics, then select an individual grid on which the application runs or should run.
  • GUI 1020 may additionally allow the user to specify filters (e.g., in account settings, permanently for the account; for a specific search; etc.) in order, for example, to select a subset of the available data centers based on some criteria, such as geographical area, service level agreement, CPU/memory ratio, price range, etc.
  • filters e.g., in account settings, permanently for the account; for a specific search; etc.
  • GUI 1020 may also be operable to display various types of target user content one or more target user bases for whom the selected application may be targeted toward.
  • Examples of such target user content may include, but are not limited to, one or more of the following (or combinations thereof):
  • mapping content relating to the locations and/or densities of one or more target user bases for which the selected application instance may be targeted toward.
  • Ping time information or communication latency information e.g., associated with one or more data centers which may be generated, for example, based on portions of the target user base information.
  • GUI 1020 may have different characteristics (e.g., different shapes, sizes, colors, etc.), which, for example, may be used to represent different types of information which may be of use to the user, such as, for example, one or more of the following (or combinations thereof):
  • the size of the data center object may be used to indicate the relative availability of resources at the corresponding data center.
  • Information relating to relative resource costs/fees associated with different data centers throughout the global Cloudware network may be used to indicate the relative estimated cost for hosting an instance of the selected application at the data center corresponding to that data center object.
  • Information relating to the current location of the application For example, additional graphics/objects (such as the circle around the dot of object 1002) may be used to indicate the location of the data center where the application resides (whether running or not), to be distinguished from all other data center locations.
  • a padlock shape may be displayed next to a displayed application object to indicate a status of the application (such as, for example, the application cannot be moved).
  • star shaped icons displayed next to DC locations may indicate relative preferred DC locations (e.g., one-to-five stars as DC rating), one or more currency icons (e.g., $$$ or € €) may be used to indicate relative cost of running in a given data center, etc.
  • the color of the data center object may be used to indicate the relative preference for hosting an instance of the selected application at the data center corresponding to that data center object.
  • the data center preference information may be based upon a variety of different criteria such as, for example, one or more of the following (or combinations thereof):
  • Criteria relating to data center resource availability • Criteria relating to data center operational statistics
  • a user may select (e.g., click and/or mouseover) an object displayed in GUI 1020 in order to access additional information and/or features relating to the data center (or other entity) associated with the selected object, including cost and other data center characteristics discussed herein.
  • the user when the user clicks on data center object 1004, the user may be presented with additional information (e.g.,
  • the user may be presented with an option to move or migrate the selected application from its current data center location (e.g., 1002) to the newly selected data center (e.g., 1004).
  • user may only be allowed or in able to move or migrate applications which are owned by or managed by that user.
  • GUI 1020 may also present the user with an option (e.g., via Map/List icons 1007) for displaying various content presented in GUI 1020 via a list or table.
  • GUI 1020 may present the user with other visual or graphical representations of the data center and location information (as well as other selected data center features, services, resources, attributes, etc.).
  • FIG 11 shows an example embodiment of graphical user interface (GUI) 1100 which may be used for implementing various Cloudware related aspects/features.
  • GUI graphical user interface
  • GUI 1100 may enable a user to configure and/or assign various properties of the selected application before starting or re-starting an instance of the selected application. As illustrated in the example embodiment of Figure 11, the properties portion of the
  • Application Settings GUI 1120 may include various types of content relating to different properties and/or parameters to be applied to a running instance of the selected application.
  • the user may update or modify at least a portion of the application properties/parameters via GUI 1100.
  • Example of various different types of application properties and/or parameters are illustrated in the example of Figure 11.
  • Example of other types of application properties and/or parameters are described, for example, in U.S. Patent Application Serial No. 11/522,050 (Attorney Docket No. TERAP004), by Miloushev et al., entitled "APPARATUS, METHOD AND SYSTEM FOR RAPID DELIVERY OF DISTRIBUTED APPLICATIONS,” previously incorporated herein by reference.
  • GUI 1200 graphical user interface 1200 which may be used for implementing various Cloudware related aspects/features.
  • GUI 1200 may be implemented as a web page which may be accessible to users via conventional Web browsers such as Microsoft Internet Explorer, Mozilla Firefox, etc.
  • GUI 1200 includes various types of content and/or features which are similar to the content/features described previously with respect to GUI 700 of Figure 7. Additionally, it is assumed that the user has elected to access various types of Cloudware network related information, for example, by selecting the Network Tab 712.
  • GUI portion 1210 displays various types of content to the user for enabling user to access information and/were features relating to the Cloudware network such as, for example, one or more of the following (or combinations thereof):
  • Content e.g., 1220
  • appliances e.g., virtual appliances
  • Content relating to grids (e.g., utility computing grids) in the Cloudware network.
  • GUI portion 1210 may also include content (e.g., 1204) relating to searching/filtering functionality which, for example, may be operable to enable the user to initiate and/or perform searches/filtering using a variety of different search/filtering criteria such as, for example, one or more of the following (or combinations thereof): • Keyword criteria
  • GUI 1200 may also include content portion 1202, which, in at least one embodiment, may be operable to display dynamically generated, customized content relating to the user's preferred network resources and/or other information.
  • content portion 1202 may be operable to display dynamically generated, customized content relating to the user's preferred network resources and/or other information.
  • the user may browse through various content displayed in GUI portion 1210, and selectively add/delete/modify desired network content/resources to/from the user's customized (or personalized) network resource content portion 1202.
  • the various network resources which may be displayed in GUI portion 1210 and/or selectively add/deleted to/from the user's customized (or personalized) network resource content portion 1202 may include various different types of network resources which may aid or facilitate the user in implementing and/or performing various activities at the Cloudware network.
  • examples of different types of network resources may include, but are not limited to, one or more of the following (or combinations thereof):
  • the user can drag a resource found in the search results shown in content of GUI portion 1210 into his customized network content 1202. In one embodiment, this will make the selected resource (e.g., appliance, application, catalog, service, datacenter, etc.) available in user's "personalized" network, which may also able it to be more easily accessible for selection, for example, when the user is performing various other functions (e.g., creating new application instances from a catalog, editing application infrastructure, moving applications from one datacenter to another, etc.).
  • the Cloudware network may be configured or designed to allow the user to utilize only those entities which are part of the user's personalized network (sometimes referred to as "favorites").
  • a user's personalized or customized network may include, for example, one or more of the following (or combinations thereof): lists of approved applications, lists of approved appliances and/or catalogs, lists of approved/preferred data centers, etc.
  • the Cloudware network may be configured or designed to automatically analyze and select a preferred data center for placing (e.g., initial placement, or subsequent migration) a given application.
  • the Cloudware network may be operable to give preference to particular data centers which are part of the user's (or which are part of an account's) personalized network and/or may give preference to particular data centers which meet selection criteria specified by the user/account holder.
  • a user can remove resources from his personalized network 1202, for example, by selecting them with the mouse and requesting a "remove" operation.
  • a resource may be removed by dragging the resource's icon (or other object representing the resource) out of the content 1202 and dropping it outside (e.g., dragging it back into GUI portion 1210).
  • at least a portion of the various content provided in one or more of the GUIs illustrated in Figures 3-17 may be generated by the Cloudware System and/or may be accessed by a user via the Cloudware network.
  • GUI portion 1210 may also be made available as search results through general-purpose search engines such as Google, Yahoo and MSN.
  • the Cloudware System may publish profiles of the resources and/or content available at the Cloudware network (e.g., datacenters, accounts, catalogs, applications, appliances, services, etc.) as static web pages, and may submit them to search engines for indexing.
  • This approach allows those who search for a particular solution (e.g., clustered MySQL) to find solutions available on the Cloudware service even if they themselves are not yet a Cloudware service member/user, thereby attracting additional customers and content providers to the services provided by or offered at the Cloudware network, as well as providing additional demographic data of those interested in a particular solution.
  • the Cloudware System may further collect statistics on such searches and page hits, and provide suggestions for targeted ads.
  • the content published as static web pages and searchable on the web without customer login may be limited in certain ways, for example without links or details that are available only to logged in users (e.g., pricing info may be unavailable without knowing the type of customer: for profit, non-profit, reseller, academic).
  • GUI 1300 shows an example embodiment of graphical user interface (GUI) 1300 which may be used for implementing various Cloudware related aspects/features.
  • GUI 1300 may be implemented as a web page which may be accessible to users via conventional Web browsers such as Microsoft Internet Explorer, Mozilla Firefox, etc.
  • GUI 1300 includes various types of content and/or features which are similar to the content/features described previously with respect to GUI 700 of Figure 7. Additionally, it is assumed that the user has elected to access various types of Cloudware message related information, for example, by selecting the Messages Tab 714.
  • GUI portion 1304 displays various types of content to the user for enabling user to access various types of message related information and/or features such as, for example, one or more of the following (or combinations thereof): • Messaging activity relating to various people and/or organizations (e.g., within the Cloudware network).
  • a user when a user utilizes a given resource of the Cloudware network (such as, for example, a given application, appliance, data center, etc.), the user may be eligible to subscribe to (and/or may be automatically subscribed to) receive messages relating to that particular resource.
  • a user may be eligible to subscribe to (and/or may be automatically subscribed to) receive different messages relating one or more of the entities/resources included in the user's personalized and/or customized network resource list (e.g., 1202). This may allow the user to not have to determine which organization(s) publishes or maintains a given resource of the Cloudware network, while still allowing the user to receive messages and/or other information related to the resource.
  • the user's/account's subscription may be automatically updated (e.g., by the Cloudware System) to the new publisher, without requiring user interaction.
  • messages relating to a given resource may be generated by various different entities such as, for example, users, the Cloudware System, publishers, DCO' s, applications, appliances, grids, data centers, and/or other network entities.
  • the Cloudware network may include a message distribution system which may include various functionality such as, for example, one or more of the following (or combinations thereof):
  • a user may generate a message relating to a particular network element (such as, for example, a virtual appliance), and may send the message to the virtual appliance, whereupon the message may then be automatically distributed to subscribers of the virtual appliance.
  • a particular network element such as, for example, a virtual appliance
  • Such subscribers may include, for example, people and/or non-human network entities.
  • a user may generate a message relating to a particular network element (such as, for example, a virtual appliance, an application, a grid or a data center), and may send the message to that element (or to the element's designated proxy), whereupon the message may then be automatically routed to the organization that provides support services for the selected element and/or for the user's account.
  • a particular network element such as, for example, a virtual appliance, an application, a grid or a data center
  • the message may then be automatically routed to the organization that provides support services for the selected element and/or for the user's account.
  • the user may be spared the burden of performing various tasks such as, for example: separately searching for a support organization, figuring out whether his account has a set relationship with a support provider; determining which support provider supports the selected element, etc.
  • the Cloudware System may further arrange for a payment scheme between the user and the service provider. For example, in one embodiment, no payment may be required if the account already has a support contract. Alternatively, one-time payments and/or per-incident payment may be arranged through the user's standard payment method(s), thus simplifying and accelerating user's support request(s).
  • a message sent to a network element may also (or instead) be posted to a community bulletin board, so that information about the element may be shared seamlessly between those who use the element (e.g., tips, suggestions, use cases, success stories, etc.), thereby facilitating community-based support mechanisms.
  • GUI portion 1304 may also include content relating to searching/filtering functionality which, for example, may be operable to enable the user to initiate and/or perform searches/filtering using a variety of different search/filtering criteria such as, for example, one or more of the following (or combinations thereof):
  • the messages shown in the content portion 1304 may be further grouped by search criteria, such as the list in the preceding paragraph.
  • the content portion 1304 may further have action buttons, such as 1304, for replying to a message, for viewing a message and other message actions; message actions may include typical message actions available in e-mail and messaging systems (such as Microsoft Outlook, Mozilla Thunderbird, etc.).
  • GUI 1300 may also include content portion 1302, which, in at least one embodiment, may be operable to display dynamically generated, customized content relating to the organization of messages and/or subscriptions relating the user's preferred network resources and/or other information.
  • the content portion 1302 may provide a hierarchical list of folders in which messages may be organized.
  • the folders include and Inbox for organizing incoming messages, a Message Archive for organizing older archived messages, and a Sent Messages folder for organizing messages sent by the user and/or account.
  • the content portion 1302 may include content related to actions that can be performed by the user, such as a New Message action that allows users to write and send a new message.
  • the content portion 1304 may include messages from systems outside of the Cloudware network.
  • Such systems may include, for example, RSS subscriptions, blog entries, podcasts and other subscriptions, news services, etc.
  • At least a portion of the various content provided in one or more of the GUIs illustrated in Figures 3-17 may be generated by the Cloudware System and/or may be accessed by a user via the Cloudware network.
  • the message content 1300 may also be made available through external systems, such as, for example, webmail (e.g., Google Gmail), social networks (e.g., Facebook), business systems (Plaxo, Linkedln), RSS and other syndication feeds, etc.
  • the Cloudware network may provide gateways to off -Internet systems, such as, for example, Simple Message Service (SMS), fax, dial-up networks, bulletin board systems (BBS), etc.
  • Figure 14 shows an example embodiment of a graphical user interface (GUI) 1400 which may be used for implementing various Cloudware related aspects/features.
  • GUI 1400 may be implemented as a web page which may be accessible to users via conventional Web browsers such as Microsoft Internet Explorer, Mozilla Firefox, etc.
  • GUI 1400 may correspond to an infrastructure editor page which, for example, may be used to enable a user (and/or other entity) to create, configure, edit, and/or manage various appliances and/or applications. More specifically, in the example of Figure 14, it is assumed that a user has accessed GUI 1400 (e.g., via the Cloudware network) in order to configure/edit a distributed application (e.g., 1420) which, for example, may be comprised of a plurality of different virtual appliances (e.g., 1422, 1424, 1426, etc.).
  • a distributed application e.g., 1420
  • a plurality of different virtual appliances e.g., 1422, 1424, 1426, etc.
  • the infrastructure editor page may be accessible to various entities or Cloudware customers such as, for example: data center operators, end users, developers, IT staff, system administrators, publishers (e.g., publishers of applications, appliances), etc.
  • entities or Cloudware customers such as, for example: data center operators, end users, developers, IT staff, system administrators, publishers (e.g., publishers of applications, appliances), etc.
  • infrastructure editor page 1400 includes a variety of different types of content which may be related to or associated with one or more applications and/or appliances. Examples of such content may include, but are not limited to, one or more of the following (or combinations thereof): • Content (e.g., 1402) relating to properties of the infrastructure to be edited/managed.
  • infrastructure editor page 1400 is being used to access/edit/manage a portion (e.g., "Main" section) of a SugarCRM application.
  • Content e.g., 1410 relating to various types of network accessible virtual appliances and/or applications such as, for example, one or more of the following
  • appliance catalogs • appliances
  • training materials e.g., documents, how-to's, videos, podcasts
  • Application/Appliance editor GUI (e.g., 1420) operable to allow a user to create, edit, and/or modify various features and/or characteristics associated with one or more applications and/or appliances.
  • infrastructure editor page 1400 may provide functionality for enabling the user to initiate various actions or operations such as, for example, one or more of the following (or combinations thereof):
  • GUI portion 1420 may be operable to allow a user to create, edit, and/or modify various features and/or characteristics associated with one or more applications and/or appliances.
  • GUI portion 1420 may be operable to enable a user to perform a variety of different activities/operations such as, for example, one or more of the following (or combinations thereof):
  • a user may click (e.g., right click) on an icon of a specific appliance (e.g., webserver virtual appliance 1422) in order to access various types of information/content relating to the selected appliance such as, for example, one or more of the following (or combinations thereof): • Appliance overview content (such as that illustrated and described with respect to
  • Resource information relating to the selected appliance (such as that illustrated and described with respect to Figures 5 and 6, for example).
  • Resource information relating to selected appliance(s) such as, for example, one or more of the following (or combinations thereof): CPU; BCU; memory; storage; network bandwidth; historical data; present/real-time data; configured and recommended values of resources; etc.
  • Operations information relating to selected appliance(s) such as, for example, one or more of the following (or combinations thereof): state (running/stopped/in error), presence of expected and/or unexpected traffic, resource use, etc.
  • a user may click (e.g., right click) on a specific application (e.g., the "Main" section of the SugarCRM application as shown at 1420, or on other locations of the editor canvas) in order to access various types of information/content relating to the selected application such as, for example, one or more of the following (or combinations thereof):
  • a specific application e.g., the "Main" section of the SugarCRM application as shown at 1420, or on other locations of the editor canvas
  • Figure 15 shows a flow diagram illustrating various information flows and processes which may occur at or between various entities of the Cloudware network.
  • a user e.g., accessing client computer system 1502
  • desires to start a running instance of a distributed application e.g., "Test" application
  • a user utilizes client computer system 1502 to access the Cloudware network.
  • client computer system 1502 may be implemented as a personal computer or workstation which is able to access the Cloudware network via the internet using, for example, conventional Web browsers such as Microsoft Internet Explorer or Mozilla Firefox.
  • client computer system 1502 may acquire access to the Cloudware network via a Cloudware portal system 1504.
  • An example of a Cloudware portal system is described previously with respect to Cloudware User Interface 220 of Figure 2A.
  • Start Application e.g., Start "Test” Application
  • the Start Application request may be forwarded (3) to the Cloudware controller 1506 via the Cloudware portal 1504.
  • An example of a Cloudware controller system is described previously with respect to Cloudware System controller 206 of Figure 2A.
  • the Cloudware controller processes the received Start Application request.
  • the processing of the Start "Test" Application request may include performing one or more of the following operations (or combinations thereof):
  • each separate instance/copy of the Test application may have its own, respective, unique ID, so that each instance of the application can be controlled separately.
  • one complex application e.g., Test application
  • different portion(s) of the app may each have their own respective "section” ID.
  • the Test Application is to be instantiated and started at a specific network grid which is managed by grid controller system 1510.
  • the grid controller system 1510 may reside at the same data center (e.g., where the grid(s) it controls are located).
  • portions of functionality relating to the grid controller system 1510 may be incorporated into DC Manager of the Cloudware System, such as, for example, DC Manager 214 of Figure 2A.
  • the Cloudware controller 1506 may generate and send instructions to grid controller 1510 to initiate a running instance of the Test Application at the specified grid.
  • the Cloudware controller 1506 may initiate and complete a migration of the application to the target grid controller.
  • the grid controller 1510 processes the received Start Test
  • the processing of the Start "Test” Application instructions may result in the grid controller 1510 performing one or more of the following operations (or combinations thereof):
  • the grid controller may query (17) storage system 1508 to verify and/or to provide a list of the current boot volumes and/or class information to be associated with components in the instance of the Test Application which is to be instantiated/started at the specified grid.
  • storage system 1508 An example of such a storage system is described previously with respect to storage system 240 of Figure 2A.
  • the boot volume query provided from the grid controller 1510 to the storage system 1508 includes a list of boot volumes which the grid controller has identified as the appropriate boot volumes to be associated with the components in the instance of the Test Application to be started at the designated grid.
  • the grid controller 1510 may send this query to the Clowdware Controller 1506, and the Clowdware Controller 1506 may be configured or designed to handle such requests/queries.
  • the storage system 1508 processes the Test Application boot volume query sent from the Cloudware controller.
  • the processing of the Test Application boot volume query may result in the storage system (and/or other entity of the Cloudware System) performing one or more operations, including, for example one or more of the following (or combinations thereof): • Identifying and/or determining appropriate boot volumes to be associated with component instances in the Test Application.
  • the storage system may generate and send a query response to the grid controller. For example, in one embodiment, the storage system (and/or Cloudware System) may verify that the list of Test Application boot volumes identified by the grid controller is current/up-to-date. In some embodiments, if it is determined that that the list of Test Application boot volumes identified by the grid controller is not current/up-to-date, the query response may include updated boot volume information which includes information relating to the current/up-to-date boot volumes which are to be associated with the Test Application. In addition, if the component class descriptors and/or the boot volumes are found to be out-of- date, the Grid Controller 1510 may identify and/or obtain the most current versions from the Storage System 1506.
  • the grid controller may take appropriate action(s) for starting a running instance of the Test Application at the designated grid.
  • the grid controller may utilize at least a portion of information provided in the query response to start the running instance of the Test Application.
  • the grid controller may be operable to periodically determine and/or generate (25) application status information relating to one or more running applications (such as, for example, the Test Application). Further in at least one embodiment, at least a portion of the application status information may be forwarded (e.g., 27, 29) to the Cloudware system, client system, and/or other entities of the Cloudware network (and/or entities of external networks).
  • Cloudware may be defined as a global utility computing environment.
  • a utility computing service running Cloudware may combine multiple AppLogic(TM)-based grids into a single scalable, highly available computing cloud that may be used to run distributed Web 2.0 applications, for example.
  • the individual grids that comprise the cloud can be located anywhere on the net and/or at various different geographic locations across the world, and may be managed by different hosting providers.
  • Cloudware may span continents and provide a truly global computing utility, which, for example, may be accessible with just a browser.
  • customers may interact with a Cloudware-based service through a web portal and an account controller.
  • the portal may be used to create and manage their accounts, track charges and make payments, browse and search the documentation, forums and catalogs of appliances and applications, view on-line tutorial sessions, open support tickets, etc.
  • the account controller may be used to create, edit and run applications, create new appliances, publish appliances and applications, rate and review other publicly available appliances and applications, etc.
  • Cloudware may be implemented via a system
  • each individual application instance may reside on a particular grid.
  • different instances belonging to the same customer account can run on different grids.
  • the Cloudware System may decide automatically on which grid the application may reside and run.
  • the user may be asked to select preferred regions in which the application is to run (e.g. Western ONE, Texas, Germany, UK, etc.), and the Cloudware System may then automatically determine on which of the grids located in the selected region(s) to provision/create instances of the application.
  • all or selected applications of the same account running in the same datacenter may have access to a private virtual LAN (VLAN).
  • VLAN virtual LAN
  • communications which may be implemented on this VLAN may be free (or charged a fee), even between applications running on different grids.
  • the Cloudware System may first attempt to satisfy the request on the same grid on which the application is running. If that grid has sufficient resources, the request may be executed in a manner similar to the techniques described in U.S. Patent Application Serial No. 11/522,050 (previously incorporated by reference). However, if it is determined that the grid does not have sufficient resources,
  • Cloudware may identify a nearby grid with sufficient resources to handle the request and take appropriate action to migrate the application to the identified grid.
  • migration to different grids within the same datacenter and/or different data centers may be handled with a minimal interruption of service.
  • the total interruption of service may be substantially the same as the downtime which may occur when restarting the application on the same grid.
  • the definition of an application in AppLogic(TM) may be extended for Cloudware.
  • the Cloudware-based applications may have properties/characteristics similar to appliances: for example, in one embodiment, the users can instantiate applications by dragging them from a catalog onto a canvas and to configure, start, stop, logon and monitor applications with a single click from a right-button menu.
  • One goal may be to eliminate completely the need to look inside the application: a customer who just wants to use an application, rather than modify/customize its structure or behavior, may be able to do so without even having to know what an infrastructure editor is and/or how to use it.
  • Cloudware supports a global repository of appliances and applications. Customers can create their own globally accessible catalogs, and publish both appliances and ready application templates in them. Cloudware supports paid appliances and applications with different payment models such as, for example, one or more of the following (or combinations thereof): one time charge per account, one time charge per instance, usage charge per instance, usage charge per resource use (bcu, storage, bandwidth), etc.
  • various pricing methods may include one or more of the following (or combinations thereof):
  • per instance-time e.g., $1 per instance per hour
  • max. number of instances running concurrently at any time during a period e.g., $1 per instance per month
  • volume discount scales e.g., $l/instance for 1-10 instances; $0.90/instance for
  • the Cloudware System may bill the user based on his usage and/or other terms of the pricing model, and remit the collected amount to the publisher, possibly retaining a commission.
  • Cloudware may provide detailed usage statements both to the users of published appliances, as well as to the publishers. Additional business opportunities may include, but are not limited to, one or more of the following (or combinations thereof): targeted advertising for published appliances (e.g., publisher pays for ads or clicks; ads may be selected based on criteria specified by publisher, such as use of similar or complementary application or appliance); etc..
  • targeted advertising for published appliances e.g., publisher pays for ads or clicks; ads may be selected based on criteria specified by publisher, such as use of similar or complementary application or appliance
  • ads may be selected based on criteria specified by publisher, such as use of similar or complementary application or appliance
  • the Cloudware account controller may implement a REST and/or SOAP API which may provide full access to all or selected capabilities available on a shell (e.g., command line shell, or GUI shell), including, for example, ability to create and start application instances, etc.
  • a shell e.g., command line shell, or GUI shell
  • the API may enable users to register handlers for receiving system events.
  • the API may be accessible via SSL from anywhere on the net, including from appliances that run in the Cloudware network on behalf of a specified account. This latter embodiment allows applications to self-manage and/or to manage other applications. Support for building dynamic global services
  • the user may also be able to convert an application into a service with terminals, and visually assemble a global web operation from instantiable services running in the Cloudware network.
  • the Cloudware service(s) may be AppLogic(TM) based application with a boundary that includes, for example, properties, terminals, proper life cycle, and/or other aspects for using the application as a component for building large-scale web systems.
  • services may be fully dynamic - e.g., customers may be able to instantiate, configure and/or connect services on the fly, either using the visual tools and/or by invoking the API.
  • a Cloudware service may have a very low entry barrier.
  • customers may pay a monthly subscription fee (e.g., $19.95/month) plus the actual amounts of resources they have used during the month.
  • BCU basic computing unit
  • one BCU may be equated to the following resources: IGB of RAM, 50% of a CPU and 20GB of highly available storage.
  • initial prices may be $0.10 per BCU/hr, $2 per month for each GB of additional storage and $0.20 per GB of network transfer. Transfer within the same datacenter may be free, transfer between datacenters (including migration and inter-application communications) may be billed.
  • the services and resources uses may also be billed using other methods, such as one or more of those described herein (including but not limited, per individual resource, bundles, pre-pay, etc.).
  • the Cloudware server may be operable to provide one or more of the following features (or combinations thereof):
  • Open system connects Web 2.0 companies, hosting providers and/or independent software vendors (ISVs)
  • Service may be scalable to 1 ,000,000 processors and more • Seamless global operation • Easy to migrate apps to virtual private datacenters or in-house
  • Cloudware may be architected for participation:
  • the network effect in the system may be based on one or more of the following (or combinations thereof):
  • the central service manager which may be operable to track all or selected workloads and/or all or selected resources available for them, and may be further operable to create value by:
  • the data center operators can make money with greatly reduced sales, marketing and support expense or expertise.
  • data center operators can leverage system integrators, ISVs, support professionals who specialize in operating online services, and/or other services one needs to run an online service through the Cloudware service.
  • the ISVs may have access to a ready market to which they can publish their applications and/or appliances in a manner which is extremely easy to demo, test, evaluate, thereby reaching customers with greatly reduced costs of sales and marketing compared to traditional business models, even if their product itself does not lend itself to viral marketing.
  • they can leverage system integrators, ISVs, support professionals who specialize in operating online services, and other services one needs to run an online service through the Cloudware service.
  • the Web 2.0 and SaaS companies may end up with a complete solution for building and operating their online services.
  • they can leverage system integrators, ISVs, support professionals who specialize in operating online services, and other services one needs to run an online service through the Cloudware service.
  • the Cloudware technology provider (such as, for example, 3TERA) owns the distribution network and the billing system.
  • the resulting uniform price can be determined in real time, hour by hour, resulting in real-time optimization of resource usage.
  • a secondary market can be created for futures, allowing customers to save money by buying capacity in bulk ahead of time, and allowing providers to optimize their loads by selling capacity ahead of time. Inevitably, the market may determine the fair price.
  • a customer may choose to run their database service in one place, where storage may be less expensive, and take advantage of the fact that front end apps can be moved easily from one place to another to leverage lower cost "offshore” (e.g. Canada, Mexico, Eastern Europe) resources to run the front end of their service.
  • offshore e.g. Canada, Mexico, Eastern Europe
  • the resulting model may not be unlike Google, which brings together three parties - consumers, who search and read online content, advertisers who pay for ads, and content providers who serve ads on the content network. By sharing revenue with the content providers, Google increases their reach while remaining in control of the whole system as well as of the pricing on each individual ad.
  • commercial hosting provider partners e.g. Layered Technologies, The Planet.
  • system integrators data center operators, ISVs, SaaS providers, enterprise data center outsourcers, system integrators, etc. to sell products and services from the system under their own brand.
  • the legal foundation of this may be a consortium comprising 3TERA and/or selected data center operators.
  • the membership of the consortium may be expanded to include system integrators, ISVs, consultants, support professionals, enterprise data center outsourcers, and others. It may allow one to aggregate a common marketing budget and focus it on promoting the service.
  • one aspect disclosed herein relates to techniques and mechanisms for automatically migrating instances of distributed applications between geographically different server grids and/or data centers.
  • Figure 18 shows an example embodiment of a geographically distributed cloud computing network 1800 which includes at least two different data centers (e.g., Data Center A 1810, Data Center B 1820) which are each deployed different geographic locations.
  • Data Center A may be physically deployed in California and Data Center B may be physically deployed in New York.
  • Data Center A may be physically deployed in Texas (USA) and Data Center B may be physically deployed in Tokyo (Japan).
  • ⁇ B are each operable to communicate with each other via a wide area network 1802 (such as, for example, the Internet, a private network, etc.).
  • a wide area network 1802 such as, for example, the Internet, a private network, etc.
  • multiple different server grids from the different data centers may be linked together to form a virtual global server grid which may be used to facilitate utility computing for distributed applications.
  • the cloud computing network 1800 may include a Cloudware (and Billing) System 1806 having components and/or functionality similar to those described, for example, with respect to Figure 2A.
  • Data Center A includes at least one server grid (e.g., Server Grid A 1812) which is configured or designed to enable utility computing for distributed applications (e.g., via the use of virtualized computing resources such as those described herein).
  • Data Center B includes at least one server grid (e.g., Server Grid B 1822) which is also configured or designed to enable utility computing for distributed applications.
  • FIG. 19 shows an example embodiment of an interaction diagram illustrating an example of a distributed application migration procedure between two geographically distributed server grids.
  • a user e.g., a user
  • Server Grid A may be referred to as the "Source Grid”
  • Server Grid B may be referred to as the “Target Grid”
  • Application Al may be referred to as the “Source Application”
  • Application A2 may be referred to as the "Target Application.”
  • a user accesses the target server grid (Server Grid B) in order to Initiate Application Migration of the identified application (“Test" App) from Source Grid 1812 to Target Grid 1822.
  • the user may log into the Cloudware System to obtain access to Server Grid B.
  • the application migration procedure may be manually or automatically initiated by the Cloudware System, by one of the server grids (e.g., grid controller of Server Grid B) or by the application itself.
  • one or more of the following entities may be involved in the migration of an application from the Source Grid to a the Target Grid: • Client (e.g., user)
  • a request, command or instruction for initiating an application migration may include one or more of the following types of information:
  • source application identifier information e.g., name, identifier
  • Target Grid identifier information • name/IP address of Target Grid (Server Grid B)
  • Server Grid B may process the received command relating to the application migration task, and in response, may optionally perform one or more of the following operations (or combinations thereof):
  • any specified/required preconditions e.g., verify that the target grid isn't already deploying an application with the same name/identifier as the source application, verify that the Source Application exists on Server Grid A etc.
  • the Target Grid commences with the initiation of the application migration. Accordingly, at (20) the Target Grid may query the source grid for information relating to the Source Application (and/or elements associated therewith). In response, the Source Grid may provide to the Target Grid various types of information relating to the identified.
  • Target Grid uses a least a portion of the received Source
  • the different set(s) of elements may include, but are not limited to, one or more of the following (or combinations thereof):
  • application descriptor(s) which identify the components (e.g., virtual appliances, etc) of the Source Application, and their associated connections, volumes, and/or configuration settings (e.g., properties, resources, etc)
  • appliance classes that need to be migrated e.g., which, for example, may be a part of the application and/or a part of a wider scope catalog
  • volumes that need to be migrated e.g., volumes that hold application-specific data, which, for example, may be in addition to the volumes that will be migrated with the appliance classes/descriptors
  • the Target Grid may request the Source Application descriptor(s) from the Source Grid, and the Source Grid may respond by transferring (26) the requested application descriptor(s) to the Target Grid.
  • the Target Grid may request additional descriptor(s) and/or other information relating to the Source Application, and the Source Grid may respond by transferring (30) the requested additional descriptor(s) and/or other information to the Target Grid.
  • the Source Application descriptor(s), additional descriptor(s) and/or other information relating to the Source Application may include, but are not limited to, one or more of the following (or combinations thereof):
  • connections e.g., virtual networks
  • the Target Grid may request virtual volume information from the Target Grid
  • the Source Grid may respond by transferring (34) the requested virtual volume information to the Target Grid.
  • the virtual volume information may include, but are not limited to, one or more of the following types of information (or combinations thereof):
  • class volumes e.g., if any appliance classes were identified for migration
  • application-specific data volumes e.g., a volume that belongs to application rather than a specific appliance (or appliance class) of the application ); • etc.
  • the Target Grid may optionally configure the Target Application (e.g., which represents a substantially identical instance of the Source Application) at the Target Grid.
  • the Target Grid may modify IP addresses (as needed to be compatible with Data Center B), resources, etc.
  • such configurations may be performed using the transferred application descriptor(s) and/or other migrated information.
  • such configurations may be performed during the process of migrating the application descriptor and/or other application related information.
  • the Target Grid may optionally start the Target Application at the Target Grid. In at least one embodiment, this may be desirable in situations where conditions warrant the starting of the application at the Target Grid (such as, for example, in situations where an instance of the application has been migrated to the Target Grid in order to assist in responding to detected high traffic load conditions.
  • the application migration procedure may be implemented as a live application migration procedure wherein the Source Application is running at the Source Grid and continues to run concurrently during the migration procedure.
  • multiple successive transfers of virtual volume information may be performed (e.g., using an incremental approach) in order, for example, to allow the current states of the virtual machine(s) of the Target Application to be substantially synchronized with the current states of the virtual machine(s) of the Source Application (running at the Source Grid).
  • the Target Grid may optionally initiate deletion of the instance of the Source Application at the Source Grid (e.g., so that the migration procedure is implemented as a "move” rather than “copy”).
  • the Cloudware during the application migration procedure, the Cloudware
  • System may optionally be involved with (and/or may optionally perform) one or more of the following (or combinations thereof):
  • the grid API may be based on the 3TERA shell of AppLogic(TM) 2.x grids.
  • c-object descriptor compiled object descriptor
  • all or selected volumes may be provided as class volume references. It may be up to the grid to decide whether and when to instantiate a volume (this may be true both for appliance class volumes, as well as for application volumes coming the first time from an application class).
  • the grid gets the c-obj descriptor on application creation.
  • the descriptor may be replaced one or more times using the application configure operation.
  • the grid On configure and/or activate, the grid may complete volume instantiation (incl. volprep), assign MAC/IP ADDRESSES and do volfix (note: volfix may not be needed if using an alternative configuration method, such as the DHCP/HTTP-based configuration introduced in AppLogic 2.3).
  • the grid may internally create a qobj descriptor for its own controller.
  • the c-obj ect descriptor preferably includes sufficient data to perform all or selected of the above without having to refer to catalog or classes.
  • the catalog and clas s entities may be deprecated. They may be still available at the shell level but may not be integrated with Cloudware and may not be exposed through the Representational State Transfer (REST) API.
  • REST Representational State Transfer
  • a new entity, re spool (resource pool), may be added.
  • aldo may be implemented as a software package used in conjunction with AppLogic by various data center partners. Aldo may be installed by system operators on a server in the data center on the grid backbone, and may performs various operations on AppLogic like grid installs, adding and removing servers, etc.
  • bindings to outgoing requests/notifications may be configured through this operation.
  • the operation allows setting multiple grid parameters with the same command:
  • grid config prints the full configuration (maybe except security sensitive parameters); — batch option may be supported, as may be the — stdin option, thus allowing exporting and importing the grid settings via UDL file.
  • the reboot and shutdown operations may also be changed to use IPMI or similar remote power control mechanisms, where available, shutdown turns off power after the server shuts down successfully, reboot brings up a server from power down, and also does a powercycle if the server does not come back from a soft reboot.
  • the reboot and shutdown operation preferably report these actions and intermediate results to the service controller, for example, so that Operations Dept can inspect and track down the reasons for which such operations were invoked, as well as to track downtime and service level agreement (SLA). Respool
  • One preferred approach may be to have the respool not deal with a fragment list, but only with the total amount of various resources (but also include the size of the smallest fragment).
  • a resource pool on a grid may be defined as the following 3 elements: • type: ⁇ storage, computing ⁇
  • min_frag size of the smallest fragment in the pool (in the same units as the size field)
  • info operation returns both the total size of the pool and the smallest fragment.
  • Resource pools can be used for individual applications (one app per pool) or for multiple apps (run multiple apps in the same pool). To be able to start an app on the grid, you do may need to have a pool.
  • configure also may create any needed instance volumes (removing unneeded ones as well) and do volfix (if needed). If all or selected succeeds, it marks the state of the application as configured.
  • the — skipvol option skips all or selected the volume ops but may leave the application in unconfigured state, activate, if it finds the application in unconfigured state, may complete the configuration before activating the app.
  • activate may be given a resource pool ID to use (all or selected of it or any part of it, so that multiple apps may share the resource pool).
  • the SSH operation may be forwarded once through the service portal/shell (such as the CUI 220) and a second time through the grid controller of the grid on which the component executes.
  • the service portal/shell such as the CUI 220
  • apps that are in "development” mode may have the external interface of any appliance turned on for the purpose of outbound internet access (e.g., to download/install new software).
  • the exact command/mechanism my vary.
  • the mount and unmount operations may be removed.
  • share exposes a volume on the boundary of the grid, so that it can be accessed by other grids (or any other entity); this may be done, for example, to assist low- downtime migration of applications between grids, such as illustrated and described in Figure 10 and elsewhere herein.
  • the operation returns connection data sufficient to access the volume from another grid (e.g., a share ID, similar to the one used inside the grid, except it may expose the volume on the private (management) network).
  • share may be configured or designed to work by provisioning a small application, with two appliances: IN gateway and the block server; the latter may have mounted the volume locally (vol may need to be moved). This way all or selected this traffic may be routed through normal channels and sufficient resources may be allocated for the access.
  • ate may have various options: o create a new volume using a volume shared from another grid as one stream. This preferably creates a degraded volume, in which the only stream may be the external volume. Then the vol migrate command can be used to repair the volume and build local mirrors; the external mirror may be preferably treated as a disabled server (e.g., the volume may be in MIGRATE state even if it has sufficient mirrors). Once enough local mirrors are created, the external mirror may be preferably dropped, releasing the share.
  • This option may be preferably used in conjunction with the new volume share capability described immediately above; this may be done, for example, to assist low-downtime migration of applications between grids, such as illustrated and described in Figure 10 and elsewhere herein. For purposes of example, a low-downtime migration may be implemented by performing one or more of the following operations (or combinations thereof):
  • do data recovery e.g., to detach a mirror that may be marked as "out- of-sync," make a valid volume from it prior to trying a repair
  • this may also be useful when the server that holds the only good mirror becomes inoperational.
  • the various mirror tuning options of the RAID driver and its control utility can be reviewed and verified in order to use some of the newer stream flags in mdadm (like — writemostly) in order, for example, to reduce traffic to the external stream.
  • the existing volume when creating a new volume from a stream of an existing volume for the purpose of taking a snapshot, the existing volume may be prepared for taking the snapshot by creating an additional mirror (e.g., in excess of the number of mirrors normally assigned to that volume), so that when one mirror is taken away for the creation of the new volume, the old volume remains in healthy state, rather than degraded state.
  • an additional mirror e.g., in excess of the number of mirrors normally assigned to that volume
  • This interface may be kept for management purposes - to allow remote administration of grids by the service maintainer (not for regular users). In one embodiment, service users may not invoke this operation on specified grid(s).
  • This interface may be kept for management purposes - to allow remote administration of grids by the service maintainer (not for regular users), as well as for collecting logs and propagating them to the service logs.
  • service users may not invoke this operation on specified grid(s).
  • This interface may be kept for management purposes - to allow remote administration of grids by the service maintainer (not for regular users) In one embodiment, service users may not invoke this operation on specified grid(s). Note that any message changes preferably cause notifications to be sent to the service controller (e.g., if notifications are registered).
  • Cloudware controller and “service controller” may be used interchangeably.
  • SCHEDULING When operating large grids, it may be desirable to support additional resource scheduling algorithms in order to reduce fragmentation and more efficiently utilize the resources available on the grid. Additional algorithms that may be used in grids may be a pool- based scheduler and/or an optimizing scheduler.
  • the pool-based scheduler may designate server groups within the grid for different resource sizes. For example, servers 1-10 may be reserved for appliances that need up to 0.25 CPU cores, servers 11-20 for appliances that need up to 0.50 cores, servers 21-
  • pools may be defined based on any type of resource, including memory. Pool ranges can be dynamically assigned by the scheduler. In at least one embodiment, the scheduler can further use the "buddy" algorithm in order to define the pools and to use pools designated for larger components in order to run smaller components (e.g., one example of a well known "buddy algorithm" is used in the Linux kernel for memory allocation and is frequently described in various references, including
  • a scheduler may use optimization algorithms in order to optimally place appliance(s) in the available resources.
  • one or both algorithms may be used by the scheduler, together with the existing AppLogic scheduler algorithms (spread and pack).
  • the underlying AppLogic grid OS may be configured or designed to support live migration of components, preferably using the live migration capabilities of the underlying hypervisor. Such migration may be used by the scheduler and other components of the Cloudware network, for example, to reduce fragmentation and allow better utilization of resources. In addition to live migration, it may be possible to migrate appliances by stopping them and restarting them on other server(s) and/or other data center(s).
  • CLOUDWARE SERVICE API may be used by the scheduler and other components of the Cloudware network, for example, to reduce fragmentation and allow better utilization of resources.
  • it may be possible to migrate appliances by stopping them and restarting them on other server(s) and/or other data center(s).
  • the Cloudware Service API may provide one or more of the following objects and methods (or combinations thereof):
  • EXPLICIT LINKS Explicit links may be created as a result of a network search and adding a link by a human.
  • every entity type that can be linked to in addition to its normal constructor operations (create , de st roy), may also have a "link" constructors (l ink, unlink).
  • the link constructor operation may be preferably invoked with a global name of the entity to which the link may be established, as well as a local name under which that entity may be visible.
  • the constructor creates a local link object which appears equal to locally owned objects of the same type and may be listed together with those objects.
  • the local name will be MySQL and the remote name may be com.mysql.catalog4.mysql5.ver2.
  • Publisher's domain name may be used to ensure uniqueness of global names in a manner similar to the one used by Java classes (e.g., using the domain name of the publisher but in reverse order, starting with the Top Level Domain (TLD), for example com.3tera, uk.co.3tera, etc.)
  • TLD Top Level Domain
  • the API may verify whether the operation may be valid (allowed) through a link and forwards it or not, accordingly.
  • a subset of the object operations may be available over linked objects (for example, info may be available, destroy may not be).
  • a link attribute may be preferably defined on each entity; the attribute may identify the entity as a real entity or as a link. All or selected sub-entities of a linked entity (e.g., classes in a linked catalog) may be preferably treated as links. That may be, on traversing the path to an entity, if at least one element may be a link, then the entity may be considered as a link. Note that then we may have two types of explicit links - manual and inherited. In one embodiment, manual links may be unlinked; inherited links may not.
  • implicit links such as class-to-instances and application-to- grid, may be internal to the system and may not exposed through the public API .
  • reasons for this may include, for example: • a desire to leave this flexibility for the user
  • the billing may be based on a balanced CPU/mem combination.
  • the ratio may depend on the hardware configuration of the grid's servers (GB RAM per CPU core). This means that the price for running an app with certain resources may be different on different grids.
  • One preferred solution may be to show the price for running an app for an hour (or for a month) in the dashboard - in the application list and in the editor/configurator. This way the user may see what the application costs to run, regardless of how complex the calculation may be and what factors/pricing models are being used.
  • We may, of course, provide an explanation of the general principles somewhere in the docs, but it doesn't really matter, since users may be able to see the price both before starting the application and while the application is running. This achieves predictable and simple billing (e.g., the goal stated above).
  • computing cost may be expressed as $ per core and GB RAM.
  • computing cost may probably include some amount of storage and bandwidth xfer (per core) - see description of resource bundling herein

Abstract

La présente invention concerne diverses techniques destinées à être utilisées dans des réseaux informatiques tels que, par exemple, des réseaux d'informatique utilitaire et/ou de grille, à la demande. Des exemples d'au moins une partie des techniques (et/ou des caractéristiques, aspects et/ou avantages connexes) concernées par la présente comprennent : des techniques permettant de faire migrer des appareils virtuels d'une première grille de serveur à une seconde par le biais d'un réseau de communication; des techniques permettant de migrer des applications distribuées d'une première grille de serveur à une seconde par le biais d'un réseau de communication; des techniques permettant de fournir des logiciels pré-emballés dans des appareils virtuels à des systèmes informatiques pour être utilisés dans des applications de logiciels d'exploitation; des techniques permettant de gérer l'utilisation de ressources informatiques virtuelles implémentées dans un réseau informatique; des systèmes d'échange permettant de louer ou de prendre en location des ressources informatiques fournies sur un réseau informatique; des techniques permettant d'offrir, par le biais d'un réseau informatique, des ressources informatiques virtuelles destinées à être utilisées dans le déploiement d'une ou de plusieurs applications distribuées sur une ou plusieurs grilles de serveur d'un réseau informatique; des techniques permettant de proposer, par le biais d'un réseau informatique, des composants d'applications distribuées destinés à être utilisés dans le déploiement d'une ou de plusieurs applications distribuées sur une ou plusieurs grilles de serveur d'un réseau informatique; des techniques permettant d'implémenter l'échange de ressources informatiques entre des fournisseurs de ressources informatiques et des abonnés aux ressources informatiques d'un réseau informatique, et autres. Dans au moins un mode de réalisation, le réseau informatique peut comprendre plusieurs centres de données différents et/ou grilles de serveur différentes déployés dans différents sites géographiques. Dans au moins un mode de réalisation, au moins certaines des grilles de serveur permettent de fournir des ressources informatiques utilitaires et/ou de grille, à la demande, afin d'héberger divers types d'applications distribuées. Dans au moins un mode de réalisation, une application distribuée peut être caractérisée comme application constituée de différents composants (par exemple, des appareils virtuels, des machines virtuelles, des interfaces virtuelles, des volumes virtuels, des connexions de réseau virtuelles, etc.) dans des environnements d'exécution séparés. Dans au moins un mode de réalisation, certains des différents composants de l'application distribuée peuvent être hébergés ou déployés sur différentes plates-formes (par exemple, des serveurs différents) connectés par le biais d'un réseau. Dans certains modes de réalisation, une application distribuée peut être caractérisée comme application qui s'exécute sur au moins deux ordinateurs en réseau.
PCT/US2009/036579 2008-03-07 2009-03-09 Nuage d'informatique utilitaire distribué mondialement WO2009111799A2 (fr)

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
US6865908P 2008-03-07 2008-03-07
US61/068,659 2008-03-07
US12533408P 2008-04-23 2008-04-23
US61/125,334 2008-04-23

Publications (2)

Publication Number Publication Date
WO2009111799A2 true WO2009111799A2 (fr) 2009-09-11
WO2009111799A3 WO2009111799A3 (fr) 2009-12-17

Family

ID=41056698

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2009/036579 WO2009111799A2 (fr) 2008-03-07 2009-03-09 Nuage d'informatique utilitaire distribué mondialement

Country Status (1)

Country Link
WO (1) WO2009111799A2 (fr)

Cited By (35)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2011035443A1 (fr) * 2009-09-26 2011-03-31 Sharif-Ahmadi Seyed M Système et procédé de calcul informatisé en micronuage
WO2012089149A1 (fr) * 2010-12-30 2012-07-05 Huawei Technologies Co., Ltd. Procédé et système pour un service de listage et de commerce en ligne infonuagique pour des fournisseurs d'articles
EP2524321A2 (fr) * 2010-01-15 2012-11-21 Endurance International Group, Inc. Service d'hébergement de domaine web non-affilié basé sur un service commun
US8352611B2 (en) 2010-06-29 2013-01-08 International Business Machines Corporation Allocating computer resources in a cloud environment
US8370510B2 (en) 2009-12-18 2013-02-05 Microsoft Corporation Remote application presentation over a public network connection
US8407366B2 (en) 2010-05-14 2013-03-26 Microsoft Corporation Interconnecting members of a virtual network
US8560367B2 (en) 2012-02-09 2013-10-15 Mercury Holdings Llc Computer-implemented cloud-based litigation management system
US8589923B2 (en) 2010-10-14 2013-11-19 International Business Machines Corporation Preprovisioning virtual machines based on request frequency and current network configuration
US8621058B2 (en) 2010-10-28 2013-12-31 Hewlett-Packard Development Company, L.P. Providing cloud-based computing services
US8789164B2 (en) 2012-03-16 2014-07-22 International Business Machines Corporation Scalable virtual appliance cloud (SVAC) and devices usable in an SVAC
WO2014164521A1 (fr) 2013-03-11 2014-10-09 Amazon Technologies, Inc. Suivi d'utilisation d'application dans un environnement informatique
WO2014190703A1 (fr) * 2013-05-29 2014-12-04 国家电网公司 Système d'analyse de préavertissement en ligne basé sur une plate-forme de simulation de nuage
WO2015060833A1 (fr) * 2013-10-22 2015-04-30 Empire Technology Development, Llc Redirection de données d'application sandboxed vers des centres de données
AU2013200561B2 (en) * 2009-03-13 2015-11-19 Landmark Graphics Corporation Systems and methods for real time data management in a collaborative environment
US9277022B2 (en) 2010-01-15 2016-03-01 Endurance International Group, Inc. Guided workflows for establishing a web presence
WO2016087666A1 (fr) * 2014-12-05 2016-06-09 Hybrid Logic Ltd Contrôleur de stockage de données
WO2016173452A1 (fr) * 2015-04-29 2016-11-03 阿里巴巴集团控股有限公司 Procédé et appareil de traitement de tâche de résolution, et serveur
US9712599B2 (en) 2011-10-03 2017-07-18 International Business Machines Corporation Application peak load processing
WO2017221234A1 (fr) * 2016-06-23 2017-12-28 Elbit Systems Ltd. Représentation combinée de données tramées et vectorielles
US9883008B2 (en) 2010-01-15 2018-01-30 Endurance International Group, Inc. Virtualization of multiple distinct website hosting architectures
US9942262B1 (en) 2014-03-19 2018-04-10 University Of Virginia Patent Foundation Cyber-physical system defense
US10229005B2 (en) 2014-01-23 2019-03-12 Telefonaktiebolaget Lm Ericsson (Publ) Pattern based configuration method for minimizing the impact of component failures
CN110492967A (zh) * 2019-09-24 2019-11-22 瑞斯康达科技发展股份有限公司 一种时间同步方法、中继设备及装置
CN110928197A (zh) * 2019-11-28 2020-03-27 西门子交通技术(北京)有限公司 列车自动控制的仿真测试方法和系统
US10754699B2 (en) 2012-08-05 2020-08-25 International Business Machines Corporation Remote provisioning of virtual appliances for access to virtualized storage
CN113805874A (zh) * 2021-09-10 2021-12-17 上海得帆信息技术有限公司 适用于多框架语言的前端代码片段动态渲染方法和系统
EP4006728A1 (fr) * 2010-07-09 2022-06-01 State Street Corporation Systèmes et procédés pour une informatique en nuage privé
US11379843B2 (en) * 2020-03-31 2022-07-05 Paypal, Inc. Systems and methods for multi-domain application hosting platform migration
US11405415B2 (en) * 2019-12-06 2022-08-02 Tata Consultancy Services Limited System and method for selection of cloud service providers in a multi-cloud
WO2022189839A1 (fr) * 2021-03-12 2022-09-15 Kasten, Inc. Restauration inter-environnement natif en nuage
US11468134B2 (en) 2018-09-26 2022-10-11 International Business Machines Corporation Provisioning a customized software stack for network-based question and answer services
US20230205506A1 (en) * 2021-12-28 2023-06-29 Mastercard International Incorporated Semi-declarative method for infrastructure deployment and access control
CN116360711A (zh) * 2023-06-02 2023-06-30 杭州沃趣科技股份有限公司 一种分布式存储处理方法、装置、设备及介质
US11960496B2 (en) 2010-07-09 2024-04-16 State Street Corporation Systems and methods for data warehousing
CN113891466B (zh) * 2021-09-07 2024-04-26 武汉大学 一种面向边缘无线网络中udl任务的在线调度系统及方法

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9792140B2 (en) * 2015-03-04 2017-10-17 International Business Machines Corporation Maintenance of a software discovery catalog in a virtual machine environment
US9948786B2 (en) * 2015-04-17 2018-04-17 Cisco Technology, Inc. Handling conferences using highly-distributed agents

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6880157B1 (en) * 1999-04-05 2005-04-12 Gateway, Inc. System and method of providing a virtual appliance
US7143307B1 (en) * 2002-03-15 2006-11-28 Network Appliance, Inc. Remote disaster recovery and data migration using virtual appliance migration
US20070078988A1 (en) * 2005-09-15 2007-04-05 3Tera, Inc. Apparatus, method and system for rapid delivery of distributed applications
US20070162420A1 (en) * 2004-01-21 2007-07-12 Oracle International Corporation Techniques for automatically discovering a database device on a network

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6880157B1 (en) * 1999-04-05 2005-04-12 Gateway, Inc. System and method of providing a virtual appliance
US7143307B1 (en) * 2002-03-15 2006-11-28 Network Appliance, Inc. Remote disaster recovery and data migration using virtual appliance migration
US20070162420A1 (en) * 2004-01-21 2007-07-12 Oracle International Corporation Techniques for automatically discovering a database device on a network
US20070078988A1 (en) * 2005-09-15 2007-04-05 3Tera, Inc. Apparatus, method and system for rapid delivery of distributed applications

Cited By (69)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
AU2013200561B2 (en) * 2009-03-13 2015-11-19 Landmark Graphics Corporation Systems and methods for real time data management in a collaborative environment
US11089358B2 (en) 2009-09-26 2021-08-10 Mimik Technology Inc. Method of unscrambling television content on a bandwidth
US10341721B2 (en) 2009-09-26 2019-07-02 Mimik Technology Inc. Method and system for processing multi-media content
WO2011035443A1 (fr) * 2009-09-26 2011-03-31 Sharif-Ahmadi Seyed M Système et procédé de calcul informatisé en micronuage
US10433007B2 (en) 2009-09-26 2019-10-01 Mimik Technology Inc. Method of adapting a bit rate for a mobile device
US10440429B2 (en) 2009-09-26 2019-10-08 Mimik Technology Inc. Method of collecting usage information
US10477255B2 (en) 2009-09-26 2019-11-12 Mimik Technology Inc. Method of transitioning content on user devices
US10609447B2 (en) 2009-09-26 2020-03-31 Mimik Technology Inc. Method of unscrambling television content on a bandwidth
US10298967B2 (en) 2009-09-26 2019-05-21 Mimik Technology Inc. Method of unscrambling television content on a bandwidth
US10674202B2 (en) 2009-09-26 2020-06-02 Mimik Technology Inc. Method of using a mobile device with a television display
US10893322B2 (en) 2009-09-26 2021-01-12 Mimik Technology, Inc. Method of displaying multiple content streams on a user device
US8370510B2 (en) 2009-12-18 2013-02-05 Microsoft Corporation Remote application presentation over a public network connection
US9197517B2 (en) 2010-01-15 2015-11-24 Endurance International Group, Inc. Migrating a web hosting service via a virtual network from one architecture to another
US9883008B2 (en) 2010-01-15 2018-01-30 Endurance International Group, Inc. Virtualization of multiple distinct website hosting architectures
US8819121B2 (en) 2010-01-15 2014-08-26 Endurance International Group, Inc. Unaffiliated web domain hosting service based on service pools with flexible resource
US8819207B2 (en) 2010-01-15 2014-08-26 Endurance International Group, Inc. Unaffiliated web domain hosting service based on common service pools architecture
US8819122B2 (en) 2010-01-15 2014-08-26 Endurance International Group, Inc. Unaffiliated web domain common hosting service with service representative plug-in
US8825746B2 (en) 2010-01-15 2014-09-02 Endurance International Group, Inc. Unaffiliated web domain hosting service based on shared data structure
US8843571B2 (en) 2010-01-15 2014-09-23 Endurance International Group, Inc. Web hosting service based on a common service architecture and third party services
EP2524321A4 (fr) * 2010-01-15 2013-09-11 Endurance Int Group Inc Service d'hébergement de domaine web non-affilié basé sur un service commun
EP2524321A2 (fr) * 2010-01-15 2012-11-21 Endurance International Group, Inc. Service d'hébergement de domaine web non-affilié basé sur un service commun
US8935314B2 (en) 2010-01-15 2015-01-13 Endurance International Group, Inc. Common service web hosting architecture with CRM plus reporting
US9277022B2 (en) 2010-01-15 2016-03-01 Endurance International Group, Inc. Guided workflows for establishing a web presence
US10536544B2 (en) 2010-01-15 2020-01-14 Endurance International Group, Inc. Guided workflows for establishing a web presence
US9071552B2 (en) 2010-01-15 2015-06-30 Endurance International Group, Inc. Migrating a web hosting service between a one box per client architecture and a cloud computing architecture
US9071553B2 (en) 2010-01-15 2015-06-30 Endurance International Group, Inc. Migrating a web hosting service between a dedicated environment for each client and a shared environment for multiple clients
US8407366B2 (en) 2010-05-14 2013-03-26 Microsoft Corporation Interconnecting members of a virtual network
US8433803B2 (en) 2010-06-29 2013-04-30 International Business Machines Corporation Allocating computer resources in a cloud environment
US8352611B2 (en) 2010-06-29 2013-01-08 International Business Machines Corporation Allocating computer resources in a cloud environment
US11960496B2 (en) 2010-07-09 2024-04-16 State Street Corporation Systems and methods for data warehousing
EP4006728A1 (fr) * 2010-07-09 2022-06-01 State Street Corporation Systèmes et procédés pour une informatique en nuage privé
US8589923B2 (en) 2010-10-14 2013-11-19 International Business Machines Corporation Preprovisioning virtual machines based on request frequency and current network configuration
US8595722B2 (en) 2010-10-14 2013-11-26 International Business Machines Corporation Preprovisioning virtual machines based on request frequency and current network configuration
US8621058B2 (en) 2010-10-28 2013-12-31 Hewlett-Packard Development Company, L.P. Providing cloud-based computing services
CN103282897A (zh) * 2010-12-30 2013-09-04 华为技术有限公司 针对项目提供商的基于云在线商务和列表服务所用的方法和系统
WO2012089149A1 (fr) * 2010-12-30 2012-07-05 Huawei Technologies Co., Ltd. Procédé et système pour un service de listage et de commerce en ligne infonuagique pour des fournisseurs d'articles
US9712599B2 (en) 2011-10-03 2017-07-18 International Business Machines Corporation Application peak load processing
US9781191B2 (en) 2011-10-03 2017-10-03 International Business Machines Corporation Processing of application peak load
US8560367B2 (en) 2012-02-09 2013-10-15 Mercury Holdings Llc Computer-implemented cloud-based litigation management system
US9736163B2 (en) 2012-03-16 2017-08-15 International Business Machines Corporation Scalable virtual appliance cloud (SVAC) and methods usable in an SVAC
US8789164B2 (en) 2012-03-16 2014-07-22 International Business Machines Corporation Scalable virtual appliance cloud (SVAC) and devices usable in an SVAC
US9009831B2 (en) 2012-03-16 2015-04-14 International Business Machines Corporation Scalable virtual appliance cloud (SVAC) and methods usable in an SVAC
US10754699B2 (en) 2012-08-05 2020-08-25 International Business Machines Corporation Remote provisioning of virtual appliances for access to virtualized storage
US10440132B2 (en) 2013-03-11 2019-10-08 Amazon Technologies, Inc. Tracking application usage in a computing environment
EP2972728A4 (fr) * 2013-03-11 2016-12-28 Amazon Tech Inc Suivi d'utilisation d'application dans un environnement informatique
WO2014164521A1 (fr) 2013-03-11 2014-10-09 Amazon Technologies, Inc. Suivi d'utilisation d'application dans un environnement informatique
WO2014190703A1 (fr) * 2013-05-29 2014-12-04 国家电网公司 Système d'analyse de préavertissement en ligne basé sur une plate-forme de simulation de nuage
US9531813B2 (en) 2013-10-22 2016-12-27 Empire Technology Development Llc Sandboxed application data redirection to datacenters
CN105814579B (zh) * 2013-10-22 2019-01-08 英派尔科技开发有限公司 沙盒应用数据重定向至数据中心
CN105814579A (zh) * 2013-10-22 2016-07-27 英派尔科技开发有限公司 沙盒应用数据重定向至数据中心
WO2015060833A1 (fr) * 2013-10-22 2015-04-30 Empire Technology Development, Llc Redirection de données d'application sandboxed vers des centres de données
US10229005B2 (en) 2014-01-23 2019-03-12 Telefonaktiebolaget Lm Ericsson (Publ) Pattern based configuration method for minimizing the impact of component failures
US9942262B1 (en) 2014-03-19 2018-04-10 University Of Virginia Patent Foundation Cyber-physical system defense
WO2016087666A1 (fr) * 2014-12-05 2016-06-09 Hybrid Logic Ltd Contrôleur de stockage de données
WO2016173452A1 (fr) * 2015-04-29 2016-11-03 阿里巴巴集团控股有限公司 Procédé et appareil de traitement de tâche de résolution, et serveur
WO2017221234A1 (fr) * 2016-06-23 2017-12-28 Elbit Systems Ltd. Représentation combinée de données tramées et vectorielles
US10627243B2 (en) 2016-06-23 2020-04-21 Elbit Systems Ltd. Combined raster and vector data representation
US11468134B2 (en) 2018-09-26 2022-10-11 International Business Machines Corporation Provisioning a customized software stack for network-based question and answer services
CN110492967A (zh) * 2019-09-24 2019-11-22 瑞斯康达科技发展股份有限公司 一种时间同步方法、中继设备及装置
CN110928197A (zh) * 2019-11-28 2020-03-27 西门子交通技术(北京)有限公司 列车自动控制的仿真测试方法和系统
US11405415B2 (en) * 2019-12-06 2022-08-02 Tata Consultancy Services Limited System and method for selection of cloud service providers in a multi-cloud
US11379843B2 (en) * 2020-03-31 2022-07-05 Paypal, Inc. Systems and methods for multi-domain application hosting platform migration
WO2022189839A1 (fr) * 2021-03-12 2022-09-15 Kasten, Inc. Restauration inter-environnement natif en nuage
CN113891466B (zh) * 2021-09-07 2024-04-26 武汉大学 一种面向边缘无线网络中udl任务的在线调度系统及方法
CN113805874A (zh) * 2021-09-10 2021-12-17 上海得帆信息技术有限公司 适用于多框架语言的前端代码片段动态渲染方法和系统
US20230205506A1 (en) * 2021-12-28 2023-06-29 Mastercard International Incorporated Semi-declarative method for infrastructure deployment and access control
US11829744B2 (en) * 2021-12-28 2023-11-28 Mastercard International Incorporated Semi-declarative method for infrastructure deployment and access control
CN116360711A (zh) * 2023-06-02 2023-06-30 杭州沃趣科技股份有限公司 一种分布式存储处理方法、装置、设备及介质
CN116360711B (zh) * 2023-06-02 2023-08-11 杭州沃趣科技股份有限公司 一种分布式存储处理方法、装置、设备及介质

Also Published As

Publication number Publication date
WO2009111799A3 (fr) 2009-12-17

Similar Documents

Publication Publication Date Title
US9578088B2 (en) Globally distributed utility computing cloud
WO2009111799A2 (fr) Nuage d'informatique utilitaire distribué mondialement
Wittig et al. Amazon Web Services in Action: An in-depth guide to AWS
US11216756B2 (en) Mapping portal applications in multi-tenant environment
CN111279320B (zh) 实现微服务配置和管理的api储存库
US8612976B2 (en) Virtual parts having configuration points and virtual ports for virtual solution composition and deployment
Varia Best practices in architecting cloud applications in the AWS cloud
US7200530B2 (en) Architecture for distributed computing system and automated design, deployment, and management of distributed applications
EP3306473B1 (fr) Fédération de nuages en tant que service
Riti Pro DevOps with Google Cloud Platform: With Docker, Jenkins, and Kubernetes
Patterson Learn AWS Serverless Computing: A Beginner's Guide to Using AWS Lambda, Amazon API Gateway, and Services from Amazon Web Services
US10291488B1 (en) Workload management in multi cloud environment
Rosner Learning AWS OpsWorks
US20110258215A1 (en) Social network based information discovery about network data processing systems
Birkenheuer et al. Infrastructure Federation Through Virtualized Delegation of Resources and Services: DGSI: Adding Interoperability to DCI Meta Schedulers
Aguado et al. A practical approach to cloud IaaS with IBM SoftLayer: Presentations guide
MVP et al. Microsoft System Center 2012 R2 Operations Manager Cookbook
Vyas et al. Core Kubernetes
Ellis Microsoft azure iaas essentials
US11907731B1 (en) Configurable cloud development environments
Muñoz Exam Ref AZ-204 Developing Solutions for Microsoft Azure
Hirway Hybrid Cloud for Developers: Develop and deploy cost-effective applications on the AWS and OpenStack platforms with ease
Brandle et al. Cloud computing patterns of expertise
Polze A comparative analysis of cloud computing environments
Freato et al. Mastering Cloud Development using Microsoft Azure

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 09718180

Country of ref document: EP

Kind code of ref document: A2

NENP Non-entry into the national phase in:

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 09718180

Country of ref document: EP

Kind code of ref document: A2