Classifications

• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
  • G06F21/60—Protecting data
  • G06F21/62—Protecting access to data via a platform, e.g. using keys or access control rules
  • G06F21/6218—Protecting access to data via a platform, e.g. using keys or access control rules to a system of files or objects, e.g. local or distributed file system or database
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F9/00—Arrangements for program control, e.g. control units
  • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
  • G06F9/46—Multiprogramming arrangements
  • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
  • G06F9/5061—Partitioning or combining of resources
  • G06F9/5072—Grid computing
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
  • H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
  • H04L41/40—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks using virtualisation of network functions or resources, e.g. SDN or NFV entities
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
  • H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
  • H04L41/50—Network service management, e.g. ensuring proper service fulfilment according to agreements
  • H04L41/5041—Network service management, e.g. ensuring proper service fulfilment according to agreements characterised by the time relationship between creation and deployment of a service
  • H04L41/5048—Automatic or semi-automatic definitions, e.g. definition templates
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
  • H04L12/00—Data switching networks
  • H04L12/02—Details
  • H04L12/14—Charging, metering or billing arrangements for data wireline or wireless communications
  • H04L12/1453—Methods or systems for payment or settlement of the charges for data transmission involving significant interaction with the data transmission network
  • H04L12/1457—Methods or systems for payment or settlement of the charges for data transmission involving significant interaction with the data transmission network using an account

Definitions

• the present invention relates to cloud computing. More particularly, the present invention relates to the coordination of processes in cloud computing.
• Cloud computing is a way for users to store data and operate computational processes on infrastructure connected by a network.
• users instead of having to purchase physical infrastructure, users send processes and data out to be run and stored on infrastructure owned by other entities.
• the user only pays for the amount of data storage or processing capability that he or she desires. This allows the user to tap computing resources that would be impossible without owning actual, physical and vast computing resources.
• Cloud computing opens up great possibilities because of the many resources available. But, by its very nature, the cloud computing has so many tools and resources that it is difficult to organize efficiently.
• Systems and methods of managing computer cloud resources may comprise receiving, via at least one server acting as an orchestration site, at least one cloud resource management plan from at least one user and storing, via the at least one server, the at least one plan in at least one storage and retrieving, via the at least one server acting as an orchestration manager, the at least one plan from at the at least one database, and executing the plan, via the at least one server, with at least one site controller.
• Examples of these may also include where the orchestration site and the orchestration manager are further configured to communicate over a distributed messaging bus. Also, where the communication includes information regarding whether the at least one plan is ready for execution.
• the at least one plan includes at least one group of cloud resources. And where the at least one group includes at least one cloud resource and a policy regarding the at least one resource. [0006] Some examples may include where the plan includes information regarding the interaction of the at least one group with other groups. Also, where the plan includes a name of the plan. And where the at least one site controller includes a computer cloud resource. Yet others may include where the storage is at least one of a database and cloud storage.
• Examples may also include where the system and methods include at least one cloud resource management plan includes instructions including configuring a master orchestration to drive at least one sub orchestration. Also where a cloud resource includes at least one of, a third party provided object and third party provided cloud service.
• cloud resource is, or is a combination of, a virtual machine, a physical machine, an operating system, storage service, networking service, and an application service.
• Figure 1 is an overview of the steps required to create an orchestration according to some embodiments.
• Figures 2(a) to (c) illustrate the steps of Figure 1 schematically with reference to various objects in cloud computing according to some embodiments.
• Figure 3 is an expansion of Figure 2 showing how an orchestration can be joined to another orchestration according to some embodiments.
• Figure 4 is still a further expansion of Figures 2 and 3, showing multiple layers of orchestrations driving other orchestrations according to some embodiments.
• Figure 5 is a schematic overview of a system on which these orchestrations can run according to some embodiments.
• Figure 6 is a schematic illustrating an alternate structure of a cloud computing system, suitable for use with the orchestration techniques described herein according to some embodiments.
• This Orchestration' provides for the automated management of user- defined system components for high availability, monitoring, and persistence allowing users to troubleshoot systems, manage processes and to create and coordinate complex processes in a computing cloud, or over numerous clouds.
• Orchestration is a specification by which user can specify different objects or references to other orchestrations or objects, establish relationships between them and apply different kinds of policies on them. These objects include but are not limited to all first class features provided by the cloud and Cloud Services Extensions added to the cloud. Different kind of relationships can be specified, “dependency” is one example of a relationship. Different kinds of policies can be applied. “high-availability(HA) and autoscaling” are few examples of such policies.
• User can co-ordinate several processes on the same, multiple or public clouds. These processes may include any functionality/objects provided by the cloud. Example: security policies, storage co-ordination, networking, actual CRs (computational resources which includes but is not limited to a virtual machine, OS container or an actual physical machine), new cloud services/extensions enabled on the cloud, etc.
• Orchestration is the coordination of the objects.
• a single orchestration can even create and manage other orchestrations.
• a Master Orchestration can drive three children orchestrations, each of those three of their own children, and so on, creating a compounding or mushrooming effect.
• policies include but are not limited to High Availability (apply different policies if they go away e.g. - recreate them on same/another cloud etc.), Monitoring (monitor the state of the objects), Autoscaling (scale the objects up or down based on certain criteria). Cloud can provided its own implementation of these policies or a user /cloud-admin can create custom policies.
• orchestration ties together cloud computing components into a single, manageable collection for the user.
• a user can associate networks, network security, and storage with the instantiation of a Computational Resource, which includes, but is not limited to a Virtual Machine, OS containers or an actual physical machine. This instantiation can be restarted automatically if it terminates for some reason, it can be monitored, or it can be disabled.
• a user can specify dependencies to affect the sequence of how components are orchestrated.
• a user group 110 objects of similar type to create an orchestration plan or Op lan.' Thereafter, 112, the user adds one or more policies, for example 'high availability" (HA) to the specific oplan. Then, 114, the user adds a unique label to the oplan. Once that is done, the user adds additional groupings of similarly created bur differently functioning, oplans, 116. At this stage, 118, the user defines the relationships between the added oplans and thereafter, 120, has created an Orchestration.'
• an orchestration is a group of objects. As will be shown below, it can also have other orchestrations as objects.
• FIGs 2(a) to (c) illustrate these processes with reference to cloud computing objects shown schematically.
• a user (not shown) has grouped objects 212a, 212b, and 212c of similar type into an orchestration plan 214.
• the user has added one or more policies, shown schematically as "policies," and added a unique label A in this case to the oplan.
• the orchestration model therefore represents a grouping of objects, namely: oplans: is a fully listed sub collection of oplans described below; status: overall status for this
• the oplan represents an orchestration plan for a specific object and that it only exists as part of an orchestration. It may include a number of attributes, including: obj type: refers to the type of the object. It is nothing but the base path for other models in the system. Examples: launchplan, vservice/vdhcpservice etc.; objects: List of object dictionaries or names of obj type. See examples below; status: status for this oplan; and ha_policy: if a user wants an object to persist, for example with instances, launchplans, etc., the user can apply an ha policy here.
• obj type refers to the type of the object. It is nothing but the base path for other models in the system. Examples: launchplan, vservice/vdhcpservice etc.; objects: List of object dictionaries or names of obj type. See examples below; status: status for this oplan; and ha_policy: if a user wants an object to persist, for example with instances, launchplan
• this concept can be expanded even further so that a single orchestration, for example orchestration 224 created with reference to Figure 2; can have another orchestration 310 as an object.
• this added orchestration 310 will itself have a unique label "D" and policies at the orchestration level and will also have a defined relationship 312 with the orchestration 224 created with reference to Figure 2.
• a single orchestration can drive other orchestrations.
• a single master object 410 can drive an orchestration 412, which can itself drive other orchestrations 414a, 414b and 414c.
• These orchestrations 414a, 414b and 414c can themselves drive even other orchestrations 416a and 416b, etc, thus creating an ever expanding "mushroom effect.”
• the system 510 has two main components, i.e. site controller, orchestration site 512, which functions to expose the Web API (e.g., REST, SOAP, etc.) interface 514. It also adds/deletes an orchestration to/from the storage such as a database (DB) 516 and assigns it to one of the orchestration managers 524.
• the storage could also be cloud storage.
• a manager, orchestration manager 524 manages the actual orchestration by Restfully or otherwise managing 526 objects providing 'high availability' (HA), monitoring and other features.
• This figure also shows a plurality of controllers, site controllers 530. This publish subscribe mechanism can exist on the same cloud or across clouds.
• FIG. 6 is a schematic illustrating an alternate structure of a cloud computing system, suitable for use with the orchestration techniques described herein.
• the APIs, 610 are used to communicate to the cloud 620 through a distributing load balancer 630.
• the load balancer, 630 distributes services to different Orchestration Managers, 640, 642, which are each in communication with a distributed database system, distributed data store, 650.
• a Messaging Service 660 coordinates the communication among the distributed database system, Orchestration Managers 670, 672 and the Load Balancer 630 on the cloud 620 and among other possible clouds (not shown).
• this system allows a user to group together a plurality of objects and to orchestrate a number of common functions, such as
• add/get/delete/update monitor status
• provide High Availability specify relationships between different objects and auto-scale certain objects, for example, instances and cloud services.
• orchestration is the automated management of user-defined system components for high availability, monitoring, and persistence.
• Orchestrations can be available via a Web API/CLI/UI but they can be extended to other interfaces.
• orchestrations can be specified in any document /object specification language, for example: JSON/YAML/XML, etc.
• One example includes starting a virtual machine, where the name of the machine image list and the desired shape are known. From the command line, for example, an orchestration may be started as shown in the following example.
• nimbula-api orchestrate simple /acme/imagelists/lucdi64 medium
• Any object/functionality/feature provided by the cloud is supported. References to other orchestrations or objects are also supported. It also supports object/functionality which is dynamically added to a cloud.
• an orchestration could be (this as an example and an orchestration can reflect the following status, although it is not limited to only these):
• any single orchestration one can include many components.
• an orchestration can include references to other "nested" orchestrations so there is no effective limit on the number of components.
• Nesting For an example of nested orchestrations, see Nesting
• orchestrations One can specify the sequence in which the components in an orchestration start their dependencies on one another. For an example, see the section on Multiple objects with dependencies.
• the orchestration in this example is shown in Basic orchestration: configuring an instance and stored in a file called lpl.json.
• This example uses the -f json option to display the output fully.
• imagelist nimbula/public/lucid64
• imagelist nimbula/public/lucid64
• ip "10.33.1.90”
• imagelist 7nimbula/public/ec2image
• orchestrations in JSON format for instantiating virtual machines. Specifically, this section covers basic orchestration: configuring an instance and a complete annotated instance configuration. As will be shown later under a separate heading,
• orchestrations in JSON format as several other orchestrations for a variety of uses.
• This orchestration has a single object: a plan to instantiate a virtual machine from the default machine image stored in the image list /nimbula/public/lucid64 on a small shape.
• the high availability policy is set to active.
• An orchestration can describe the various instances to be launched, the relationships among them, and their networking, storage, and security characteristics. Below is an
• orchestration for a single instance of a primary and secondary web server that must run on different nodes and a primary and secondary database.
• the example is of a CR configuration on a specific cloud, but it is important to realize that orchestrations are not limited to any specific clouds. They span cloud implementations, multiple clouds, hybrid clouds, etc.
• imagelist "/acme/mary/webserver”,
• imagelist "/acme/mary/webserver”,
• imagelist "/acme/mary/dbserver"
• volume “/acme/mary/volume 1 "
• Cloud Resources are inclusive but not limited to a) virtual machine, physical machine or a OS container, b) any Storage services provided a private or public cloud, c) any Networking services provided by private or public cloud, and d) any feature that is provided by the private or public cloud.
• CR Computational Resource
• Relationships allows one to define relationships between various instances such as whether they should be launched on the same of a different node or the same or different cluster.
• Non-limiting examples of relationships element are: same node, different node, same cluster, and different cluster.
• Each type of instance can be separately defined in the launch plan.
• the following parameters can be specified:
• Shape A valid shape with the amount of RAM and the number of CPUs needed to run one instance.
• Version The version number identifying which machine image to run from the image list.
• tags (optional) A list of strings which will tag the instance for the end-user's uses.
• networking (optional)
• the networking elements allow the specification of three sub- elements related to supported The system network services:
• vEthernet - vEthernets are discussed in About virtual ethernets (vEthernets). As vEthernets are not supported with security lists and NAT, the vEthernet field should either be omitted or set to the default vEthernet /nimbula/public/default if that NIC also has security lists and/or NAT specified. Setting the vethernet sub element to the empty string, "", is not acceptable.
• Reference source not foundAn instance can belong to up to 64 security lists. For every customer, there is a default security list, /customer/default/default. If one launches a VM without a security list tag, it is assigned to the customer's default security list, nat - Network Address Translation is described in Using distributed NAT.
• the system's distributed NAT service provides public IP services to instances running in the system site.
• a launch plan can be used to:
• Imagelist The full path name of one's image list attributes (optional)
• Optional user-defined parameters that can be passed to an instance of this machine image when it is launched. See About user-defined attributes and parameters in orchestrations, images, and image lists.
• Label A label that can be used to identify this instance in the launch plan when defining relationships between launch plan elements.
• An account is a billing container associated with a customer.
• the customer administrator must create an account to be able to launch workloads in Amazon EC2 via the system Amazon EC2 proxy or update the default account to contain the relevant Amazon EC2 credentials.
• the default account can be used to launch workloads across The system sites and in this case does not need to be explicitly set.
• the account information is passed through to all calls to a remote site. A user needs use permission on an account that they use for federation purposes. Account information is passed through with any launch command: either the default account or an explicitly specified account is used. Whether or not an action is billable, irrespective of whether it is launched locally or on a remote The system site, will depend on how billing is set up within the site and the billing arrangements with other The system sites.
• This orchestration defines security lists (plan "A") and security rules (plan "B") for protecting a web server instance (plan “C”). First plan A is started, then plan B, and finally the web server in plan C is instantiated.
• the system not only allows users to orchestrate using their own created objects and processes, but third party created objects and processes as well.
• This allows for an extensible cloud where third party objects and processes are seamlessly integrated into the cloud and can show up in the API or CLI itself for complete availability.
• templates can also be created for these orchestrations and shared amongst community of users. This will enable sharing of ideas amongst a community to enable multi cloud application stacks .Users can refer other orchestrations in their templates thereby enabling richer collaboration.
• Orchestration also allows a developer to observe their entire use of the cloud at once.
• a master orchestration can display information about how all the interdependent orchestrations are running. If something isn't working correctly, the master orchestration can indicate so.
• the orchestration can also indicate if security rules have been changed. It will find faults throughout the cloud and report them. It will find faults on objects and orchestrations running on multiple clouds and report them as well.
• imagelist "/nimbula/public/tinycore"
• 'tinyplan which is a launchplan
• 'other-orchestration ' which points to another orchestration.
• 'adding' an orchestration is separated from 'starting' an orchestration. Since 'Start' is just changing the state of an object, the query arguments are provided as part of an update operation.
• the CLI command would be nimbula-api start orchestration
• Load Balancing is achieved via distributing orchestrations across different orchestration managers. This way a single orchestration can point to multiple orchestration objects and can drive a whole hierarchy (all assigned to different managers).
• features consistent with the present inventions may be implemented via computer-hardware, software and/or firmware.
• the systems and methods disclosed herein may be embodied in various forms including, for example, a data processor, such as a computer that also includes a database, digital electronic circuitry, firmware, software, computer networks, servers, or in combinations of them.
• a data processor such as a computer that also includes a database
• digital electronic circuitry such as a computer that also includes a database
• firmware firmware
• software computer networks, servers, or in combinations of them.
• the disclosed implementations describe specific hardware components, systems and methods consistent with the innovations herein may be implemented with any combination of hardware, software and/or firmware.
• the above-noted features and other aspects and principles of the innovations herein may be implemented in various environments.
• Such environments and related applications may be specially constructed for performing the various routines, processes and/or operations according to the invention or they may include a general-purpose computer or computing platform selectively activated or reconfigured by code to provide the necessary functionality.
• the processes disclosed herein are not inherently related to any particular computer, network, architecture, environment, or other apparatus, and may be implemented by a suitable combination of hardware, software, and/or firmware.
• various general- purpose machines may be used with programs written in accordance with teachings of the invention, or it may be more convenient to construct a specialized apparatus or system to perform the required methods and techniques.
• aspects of the method and system described herein, such as the logic may be implemented as functionality programmed into any of a variety of circuitry, including programmable logic devices ("PLDs”), such as field programmable gate arrays (“FPGAs”), programmable array logic (“PAL”) devices, electrically programmable logic and memory devices and standard cell-based devices, as well as application specific integrated circuits.
• PLDs programmable logic devices
• FPGAs field programmable gate arrays
• PAL programmable array logic
• electrically programmable logic and memory devices and standard cell-based devices as well as application specific integrated circuits.
• Some other possibilities for implementing aspects include: memory devices, microcontrollers with memory (such as EEPROM), embedded microprocessors, firmware, software, etc.
• aspects may be embodied in microprocessors having software-based circuit emulation, discrete logic (sequential and combinatorial), custom devices, fuzzy (neural) logic, quantum devices, and hybrids of any of the above device types.
• the underlying device technologies may be provided in a variety of component types, e.g., metal-oxide semiconductor field-effect transistor (“MOSFET”) technologies like complementary metal-oxide semiconductor (“CMOS”), bipolar technologies like emitter-coupled logic (“ECL”), polymer technologies (e.g., silicon-conjugated polymer and metal-conjugated polymer-metal structures), mixed analog and digital, and so on.
• MOSFET metal-oxide semiconductor field-effect transistor
• CMOS complementary metal-oxide semiconductor
• ECL emitter-coupled logic
• polymer technologies e.g., silicon-conjugated polymer and metal-conjugated polymer-metal structures
• mixed analog and digital and so on.
• Examples of transfers of such formatted data and/or instructions by carrier waves include, but are not limited to, transfers (uploads, downloads, e-mail, etc.) over the Internet and/or other computer networks via one or more data transfer protocols (e.g., HTTP, FTP, SMTP, and so on).
• transfers uploads, downloads, e-mail, etc.
• data transfer protocols e.g., HTTP, FTP, SMTP, and so on.

Landscapes

• Engineering & Computer Science (AREA)
• Theoretical Computer Science (AREA)
• Software Systems (AREA)
• Physics & Mathematics (AREA)
• Computer Networks & Wireless Communication (AREA)
• Signal Processing (AREA)
• General Physics & Mathematics (AREA)
• General Engineering & Computer Science (AREA)
• Bioethics (AREA)
• Computer Security & Cryptography (AREA)
• Computer Hardware Design (AREA)
• General Health & Medical Sciences (AREA)
• Health & Medical Sciences (AREA)
• Databases & Information Systems (AREA)
• Mathematical Physics (AREA)
• Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
• Information Transfer Between Computers (AREA)
• Stored Programmes (AREA)
• Processing Or Creating Images (AREA)
• Computer And Data Communications (AREA)

Priority Applications (4)

Applications Claiming Priority (2)

Related Parent Applications (1)

Related Child Applications (1)

Publications (1)

Family

ID=48984604

Family Applications (1)

Country Status (4)

Cited By (12)

Families Citing this family (4)

Citations (4)

• 2013
  • 2013-02-07 WO PCT/US2013/025211 patent/WO2013122815A1/en not_active Ceased
  • 2013-02-07 CN CN201380014746.1A patent/CN104335179B/zh active Active
  • 2013-02-07 JP JP2014556691A patent/JP6231020B2/ja active Active
  • 2013-02-07 EP EP13748986.0A patent/EP2815346B1/en active Active

Patent Citations (4)

Non-Patent Citations (2)

Also Published As

Similar Documents

Legal Events