WO2012162171A2 - Cross-cloud management and troubleshooting - Google Patents

Cross-cloud management and troubleshooting Download PDF

Info

Publication number
WO2012162171A2
WO2012162171A2 PCT/US2012/038647 US2012038647W WO2012162171A2 WO 2012162171 A2 WO2012162171 A2 WO 2012162171A2 US 2012038647 W US2012038647 W US 2012038647W WO 2012162171 A2 WO2012162171 A2 WO 2012162171A2
Authority
WO
WIPO (PCT)
Prior art keywords
application
management
data
request
cloud
Prior art date
Application number
PCT/US2012/038647
Other languages
French (fr)
Other versions
WO2012162171A3 (en
Inventor
Kannan C. Iyer
Eric B. Watson
Original Assignee
Microsoft Corporation
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Microsoft Corporation filed Critical Microsoft Corporation
Priority to JP2014512899A priority Critical patent/JP5980914B2/en
Priority to MX2017004805A priority patent/MX366620B/en
Priority to MX2013013577A priority patent/MX347110B/en
Priority to KR1020137030613A priority patent/KR101916847B1/en
Priority to CN201280024366.1A priority patent/CN103548009B/en
Priority to EP12789586.0A priority patent/EP2710484B1/en
Priority to AU2012259086A priority patent/AU2012259086B2/en
Priority to RU2013151607/08A priority patent/RU2604519C2/en
Priority to CA2835440A priority patent/CA2835440A1/en
Priority to BR112013029716-6A priority patent/BR112013029716B1/en
Publication of WO2012162171A2 publication Critical patent/WO2012162171A2/en
Publication of WO2012162171A3 publication Critical patent/WO2012162171A3/en

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00—Arrangements for monitoring or testing data switching networks
    • H04L43/04—Processing captured monitoring data, e.g. for logfile generation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/0703—Error or fault processing not based on redundancy, i.e. by taking additional measures to deal with the error or fault not making use of redundancy in operation, in hardware, or in data representation
    • G06F11/0706—Error or fault processing not based on redundancy, i.e. by taking additional measures to deal with the error or fault not making use of redundancy in operation, in hardware, or in data representation the processing taking place on a specific hardware platform or in a specific software environment
    • G06F11/0709—Error or fault processing not based on redundancy, i.e. by taking additional measures to deal with the error or fault not making use of redundancy in operation, in hardware, or in data representation the processing taking place on a specific hardware platform or in a specific software environment in a distributed system consisting of a plurality of standalone computer nodes, e.g. clusters, client-server systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/0703—Error or fault processing not based on redundancy, i.e. by taking additional measures to deal with the error or fault not making use of redundancy in operation, in hardware, or in data representation
    • G06F11/0751—Error or fault detection not based on redundancy
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/0703—Error or fault processing not based on redundancy, i.e. by taking additional measures to deal with the error or fault not making use of redundancy in operation, in hardware, or in data representation
    • G06F11/0766—Error or fault reporting or storing
    • G06F11/0772—Means for error signaling, e.g. using interrupts, exception flags, dedicated error registers
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/0703—Error or fault processing not based on redundancy, i.e. by taking additional measures to deal with the error or fault not making use of redundancy in operation, in hardware, or in data representation
    • G06F11/0766—Error or fault reporting or storing
    • G06F11/0784—Routing of error reports, e.g. with a specific transmission path or data flow
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/0703—Error or fault processing not based on redundancy, i.e. by taking additional measures to deal with the error or fault not making use of redundancy in operation, in hardware, or in data representation
    • G06F11/079—Root cause analysis, i.e. error or fault diagnosis
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/14—Error detection or correction of the data by redundancy in operation
    • G06F11/1479—Generic software techniques for error detection or fault masking
    • G06F11/1482—Generic software techniques for error detection or fault masking by means of middleware or OS functionality
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F15/00—Digital computers in general; Data processing equipment in general
    • G06F15/16—Combinations of two or more digital computers each having at least an arithmetic unit, a program unit and a register, e.g. for a simultaneous processing of several programs
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5061—Partitioning or combining of resources
    • G06F9/5072—Grid computing
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00—Arrangements for monitoring or testing data switching networks
    • H04L43/06—Generation of reports
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/16—Error detection or correction of the data by redundancy in hardware
    • G06F11/20—Error detection or correction of the data by redundancy in hardware using active fault-masking, e.g. by switching out faulty elements or by switching in spare elements
    • G06F11/202—Error detection or correction of the data by redundancy in hardware using active fault-masking, e.g. by switching out faulty elements or by switching in spare elements where processing functionality is redundant
    • G06F11/2023—Failover techniques
    • G06F11/203—Failover techniques using migration

Definitions

  • Datacenters provide servers for running large applications. Enterprises often use datacenters to run core business functions such as sales, marketing, human resources, billing, product catalogs, and so forth. Datacenters may also run customer-facing applications, such as web sites, web services, email hosts, databases, and many other applications. Datacenters are typically built by determining an expected peak load and providing servers, network infrastructure, cooling, and other resources to handle the peak load level. Datacenters are known for being very expensive and for being underutilized at non-peak times. They also involve a relatively high management expense in terms of both equipment and personnel for monitoring and performing maintenance on the datacenter. Because almost every enterprise uses a datacenter of some sort, there are many redundant functions performed by organizations across the world.
  • Cloud computing has emerged as one optimization of the traditional datacenter.
  • a cloud is defined as a set of resources (e.g., processing, storage, or other resources) available through a network that can serve at least some traditional datacenter functions for an enterprise.
  • a cloud often involves a layer of abstraction such that the applications and users of the cloud may not know the specific hardware that the applications are running on, where the hardware is located, and so forth. This allows the cloud operator some additional freedom in terms of rotating resources into and out of service, maintenance, and so on.
  • Clouds may include public clouds, such as MICROSOFT TM Azure, Amazon Web Services, and others, as well as private clouds, such as those provided by Eucalyptus Systems, MICROSOFT TM, and others. Companies have begun offering appliances (e.g., the MICROSOFT TM Azure Appliance) that enterprises can place in their own datacenters to connect the datacenter with varying levels of cloud functionality.
  • appliances e.g., the MICROSOFT TM Azure Appliance
  • a cloud management system is described herein that provides the ability for an application to span two or more clouds (which may be across great distances), while allowing operation, management, and troubleshooting of the distributed application as a single application.
  • the system provides infrastructure that communicates across datacenters for execution and for centralizing knowledge of instances of an application that are running at different locations.
  • the system provides a computing appliance that an enterprise can place in its own private datacenter that allows an administrator to distribute at least some application loads to a public cloud or other separate locations, while providing unified management via the computing appliance.
  • the infrastructure provided by the system monitors both the application and connections between the clouds, with intelligence to know if issues are within the application or because of the connection between the clouds.
  • the system coordinates management functions across multiple cloud platforms/locations.
  • the system allows live debugging at the correct location through a seamless, unified interface.
  • the cloud management system creates a single monitoring and troubleshooting interface and knowledge and execution "fabric" across multiple clouds so that applications spread across multiple clouds can be monitored, managed, and debugged more easily.
  • Figure 1 illustrates an application running in two clouds with associated management infrastructure, in one embodiment.
  • Figure 2 is a block diagram that illustrates components of the cloud management system, in one embodiment.
  • Figure 3 is a flow diagram that illustrates processing of the cloud management system to handle a request from a management tool to access data from distributed application instances, in one embodiment.
  • Figure 4 is a flow diagram that illustrates processing of the cloud management system to report data back from and handle troubleshooting requests at a location of a remote application instance, in one embodiment.
  • a cloud management system is described herein that provides the ability for an application to span two or more clouds (which may be across great distances), while allowing operation, management, and troubleshooting of the distributed application as a single application.
  • the system provides infrastructure that communicates across datacenters for execution and for centralizing knowledge of instances of an application that are running at different locations. For example, the system may centralize logging, performance tracking, and other management functions, regardless of where the application is running.
  • the system provides a computing appliance that an enterprise can place in its own private datacenter that allows an administrator to distribute at least some application loads to a public cloud or other separate locations, while providing unified management via the computing appliance.
  • the infrastructure provided by the cloud management system monitors both the application and connections between the clouds, with intelligence to know if issues are within the application or because of the connection between the clouds.
  • the system coordinates management functions across multiple cloud platforms/locations (from one cloud's infrastructure, tasks are coordinated to execute across two or more clouds). If an administrator wants to debug the application, the system allows live debugging at the correct location through a seamless, unified interface.
  • the cloud management system creates a single monitoring and troubleshooting interface and knowledge and execution "fabric" across multiple clouds so that applications spread across multiple clouds can be monitored, managed, and debugged more easily.
  • Figure 1 illustrates an application running in two clouds with associated management infrastructure, in one embodiment.
  • the cloud management system involves the application (and/or the administrator) using
  • Cloud 110 includes a MICROSOFT TM Azure appliance instance 120 that includes
  • the appliance instance 120 includes application instance 125 that is running role 140 and role 145.
  • a second cloud 150 includes application instance 155 that is running role 160and role 170.
  • the second cloud 150 also includes infrastructure 180.
  • the appliance instance 120 knows about each of the roles and that they are part of the same application.
  • the infrastructure plumbing at each location allows the appliance instance 120 to retrieve information about role 160 and role 170 executing at the second cloud 150.
  • the system can distribute either individual roles, whole applications, or both. With all of the management data (e.g., logs from the applications, machines, and infrastructure), the system can assess the health of the application just as if all of the roles were local by applying pre-defined health rules. The system can also see the infrastructure health across both locations as well as the connection 190 in between to assess if a problem is occurring with the application or the infrastructure/network.
  • the infrastructure 130 in cloud 110 can coordinate with the infrastructure 180 in cloud 150 to provide troubleshooting and debugging support.
  • the system fabric can reach across locations to execute an application wide update, shutdown, and so forth.
  • infrastructure 130 may directly control infrastructure 180, infrastructure 130 may request infrastructure 180 to execute on infrastructure 130's behalf, and so forth.
  • operator/administrator troubleshooting tools e.g., monitoring visualization, alerting, log and configuration data viewing, and so on
  • the location of the applications and infrastructure is available and logically displayed, but does not involve separate tools and mental gymnastics from the administrator to put together.
  • the system connects the administrator 105 directly, regardless of at which location the role is residing.
  • the design of the cloud management system provides simplified and consistent running of a service across multiple clouds/location.
  • the system moves the definition of "a computing resource” from a server, beyond a datacenter to a portion of the internet (the datacenters and the connection between them).
  • This allows service level agreements (SLAs) to be defined, monitored, and managed at the service level - which is what service owners often care most about.
  • SLAs service level agreements
  • the cloud management system works in cooperation with a cloud migration system that seamlessly migrates applications from one location to another as needed, called bursting.
  • the cloud migration system provides capacity management and disaster recovery by detecting peak load conditions and automatically moving computing to another source (and back) and by providing computing across two or more clouds and moving completely to one in the case of a disaster at one site.
  • the cloud migration system monitors loads within a datacenter and detects a threshold that indicates that the current load is nearing the datacenter' s capacity. For example, the system may monitor central processing unit (CPU) usage, memory usage, storage usage, network bandwidth, and other metrics to determine how well the datacenter is handling the current load. The system may also observe trends (e.g., a rate of acceleration of resource usage) to determine whether the threshold has been or will soon be reached.
  • CPU central processing unit
  • the cloud migration system facilitates an orderly move of at least some datacenter load to another datacenter or cloud- based resources. For example, the system may migrate some peak load to a public cloud. Because cloud pricing models may vary, the system may factor cost into the decision. For example, the system may prefer to host as much load as possible at the enterprise datacenter to reduce cost, while leveraging cloud resources only to the extent needed to satisfy client requests.
  • the system may also provide management and monitoring tools that provide a consistent experience for information technology (IT) personnel regardless of where particular loads are run (e.g., locally within the enterprise or publicly using a cloud). The system may also provide planning tools to help decide appropriate workloads or applications for moving to other resources during high loads.
  • IT information technology
  • applications may have various compliance/regulatory or networking/design limitations that make them more or less suitable for migration.
  • the system can also be used as a disaster recovery architecture at a datacenter/network level to manage fast workload transition in case of disaster. If a datacenter resource permanently fails, the system can quickly and efficiently migrate additional load to the cloud or other resources so that clients of the datacenter are unaffected or are less affected by the failure.
  • the cloud migration system allows enterprises to build smaller and more efficient datacenters that leverage other resources for rare extra loads.
  • the cloud management system works with the cloud migration system to provide seamless management and troubleshooting as applications are migrated from one location to another.
  • the cloud migration system may move resources between a datacenter and the cloud on a temporary (i.e., bursting) or permanent (i.e., disaster recovery) basis.
  • Temporary movements include bursting an application or other load for a short time period to handle a peak or other high load that exceeds the datacenter' s capacity.
  • a temporary movement may include bursting an entire application or splitting the application's load across two or more locations.
  • Permanent movements include longer-term migration of loads due to a failure of hardware in the datacenter, a more sustained increase in capacity needs, a desire to globally distribute an application with dynamic load balancing, and so forth. Following are several example scenarios in which the system may be used by an enterprise.
  • an enterprise bursts application load to a public cloud to manage capacity.
  • the business decision maker i.e., CEO, CFO, or VP Marketing/sales
  • datacenter systems administrator decide it would be more cost effective and provide a better customer experience to burst some work to the public cloud at their top three peak day level of use/traffic per year, and maintain their own datacenter (potentially with a cloud appliance) at their peak monthly usage level. They sign business agreements with the cloud provider to burst work into the cloud and project estimates of when and how much work that would be. Their account is set up and the information entered into the cloud appliance.
  • the administrator runs a test with a test application from the cloud provider that ensures the connection is working properly.
  • the administrator then sets the capacity values (e.g., threshold) for initiating bursting of applications that keep capacity at the specified level in a capacity management tool.
  • the administrator goes into the tool to further specify the applications that are eligible to move in this situation (e.g., no regulatory issues with temporary movement, good technical fit).
  • an enterprise splits applications across clouds to manage capacity.
  • This scenario is similar to the scenario above except for the type of application moved is more complex, so is split out to prioritize differently.
  • the company decides to have a relationship with the cloud provider for splitting applications into the cloud (a form of bursting). In this large application was pre-identified as a bursting candidate.
  • a cloud provider decides to burst from one cluster to another.
  • the public cloud capacity planning team decides that a cluster in the Chicago datacenter is critically full but wants to maintain high utilization. They set up bursting to an
  • the day comes when the usage of the Chicago cluster reaches the threshold and the system automatically moves the selected applications (e.g., 10% of the cluster's apps) to the West coast datacenter for one day. As usage returns below the threshold, the system moves the applications back to Chicago. The system notifies a designated monitoring team proactively of the burst to be able to answer customer questions.
  • the selected applications e.g. 10% of the cluster's apps
  • the system is used for cross-cloud portfolio management.
  • An enterprise decides that to manage capacity efficiently on their cloud appliance, they want to put all variable demand apps in a public cloud and their constant demand applications in the appliance or local datacenter resources (and thus be able to run the appliance at higher utilization). While they want their computing resources split, they still want a global view across all of their application's health, to have their application developers manage applications the same way, and to maintain a single view of departmental billing across both (e.g., what costs to allocate to the consumer sales groups, internal IT, B2B sales, and so forth). The enterprise is able to set up the aggregation accounts with the public cloud with the same groups as the appliance and get billing data to integrate on their side.
  • API application-programming interface
  • NOC network operation center
  • an enterprise sets up a globally distributed application with dynamic load balancing.
  • An enterprise customer wants to manage capacity across two or more cloud instances and has a significant amount of their load in independent but geographically distributed instances (e.g., Bing search with a US and UK datacenter that both serve German queries).
  • a global traffic manager sends 50% traffic to each location.
  • load gets high at the primary location the system instructs the load balancer to send 75% of the traffic to the UK system, thus freeing up capacity from the US cloud instance, bringing it to acceptable levels.
  • capacity returns to normal the system tells the load balancer to return to the 50/50 split.
  • a variation of this is for the public cloud to be used as a secondary datacenter (with say 1% of load, the customer's site with the appliance to be the other 99%). In the case of a disaster or other reason to move load from the customer's site, 100% of traffic is shifted to the public cloud.
  • an enterprise has reached its datacenter' s capacity and needs extra computing resources, but does not yet have the available capital to expend to expand the datacenter.
  • the company can use a public cloud for spillover until they can get the hardware purchase completed.
  • FIG. 2 is a block diagram that illustrates components of the cloud management system, in one embodiment.
  • the system 200 includes a location management component 210, location data store 220, tool interface component 230, one or more management tools 240, a data migration component 250, a troubleshooting component 260, and a billing component 270. Each of these components is described in further detail herein.
  • the location management component 210 manages information about multiple datacenter locations at which instances of an application are running.
  • the component 210 includes information describing how to reach each location, connections available for retrieving management information, user accounts to use for each location with associated security credentials, application and datacenter components from which to gather troubleshooting information and send troubleshooting commands, and so forth.
  • the location management component 210 receives information describing any migration of application loads or bursting from one datacenter/cloud to another and updates the managed information so that the component 210 has a complete picture of all of the locations where the application is running. This allows the system 200 to present the complete picture and to make management of applications uniform, no matter where or at how many locations the applications are running. As conditions change and applications are distributed, the location management component 210 can present management tools with a comprehensive set of management data.
  • the location data store 220 stores information that describes locations at which instances of the application are running.
  • the data store 220 may include one or more files, file system, hard drives, databases, cloud-based storage services, or other facilities for persisting information between sessions with the system 200.
  • the stored information may include connection information, user roles, sources of management data, available log files, and any other information related to management or troubleshooting of applications distributed to multiple locations.
  • the tool interface component 230 provides an interface to the system 200 through which one or more tools can access management and troubleshooting information for the application.
  • the interface may include one or more web pages, web services, application-programming interfaces (APIs), or other interfaces through which an administrator or tools can directly or programmatically access management and
  • the tool interface component 230 provides an initial connection point for tools to access information related to the application at a cloud-computing appliance located within an enterprise's private datacenter.
  • the appliance may manage migration and distribution of application loads to a public cloud or other datacenter, and provides a central point of contact for tools that gather management information or provide application troubleshooting.
  • One or more management tools 240 connect to the tool interface component 230 to access management information or perform application troubleshooting.
  • the tools may include log viewers, reporting tools, debugging tools, or other tools that display
  • the management tools 240 may include tools designed to work with a local application, and the system 200 provides the tools with information describing a distributed application running at multiple locations without the tool's knowledge. This allows existing tools that administrators rely upon to be used even as automatic application load migration is introduced into a datacenter or cloud. In other cases, tools may be specifically written to understand distributed applications and to provide specific management information or troubleshooting related to multiple locations.
  • the tool interface component 230 may provide multiple interfaces through which management tools 240 connect to the system 200 using paradigms that are understood by each tool.
  • the data migration component 250 migrates management information at one or more remote locations where the application is running back to a home location of the application.
  • the home location may include a private datacenter, location or a cloud- computing appliance, or other location where the application normally runs under steady conditions.
  • the application may migrate some load to one or more other datacenters or public clouds to help satisfy client requests.
  • These other locations generate management data, such as log files, transaction data, and so on, just like the home location, and the data migration component 250 migrates this data back to the home location or provides access to the data from the home location, so that the management tools 240 can provide a comprehensive picture of the application's activity to administrators.
  • the troubleshooting component 260 performs troubleshooting tasks on the application at one or more locations. Troubleshooting may include debugging, processing test data, or other forms of determining whether an application is operating correctly. Troubleshooting is generally well understood at the home location, but becomes more complex as an application begins to span multiple datacenters or clouds.
  • the cloud management system 200 isolates management tools 240 and administrators from this complexity by providing a uniform interface through which tools and administrators access management information and perform troubleshooting at multiple locations. Thus, if a management tool allows an administrator to place a breakpoint at or receive trace information from a particular piece of application code at the home location, then the troubleshooting component 260 makes it just as easy to do so at a remote cloud-based instance of the application. The tools and administrator may even be unaware of all of the locations where the application is running, but can still perform management tasks as if the application were only executing at the home location.
  • the billing component 270 reports billing information related to one or more locations where an application is running.
  • One common management task is to manage computing costs, and public clouds often charge based on metrics related to workload (e.g., computing time, storage space used, and so forth). It may be useful for an administrator to gather a picture of the costs that application instances are incurring at various locations, and the cloud management system 200 can optionally provide the billing component 270 to gather this type of information so that the information can be reported through management tools and reports.
  • the computing device on which the cloud management system is implemented may include a central processing unit, memory, input devices (e.g., keyboard and pointing devices), output devices (e.g., display devices), and storage devices (e.g., disk drives or other non-volatile storage media).
  • the memory and storage devices are computer- readable storage media that may be encoded with computer-executable instructions (e.g., software) that implement or enable the system.
  • the data structures and message structures may be stored or transmitted via a data transmission medium, such as a signal on a communication link.
  • Various communication links may be used, such as the Internet, a local area network, a wide area network, a point-to-point dial-up connection, a cell phone network, and so on.
  • Embodiments of the system may be implemented in various operating environments that include personal computers, server computers, handheld or laptop devices, multiprocessor systems, microprocessor-based systems, programmable consumer electronics, digital cameras, network PCs, minicomputers, mainframe computers, distributed computing environments that include any of the above systems or devices, set top boxes, systems on a chip (SOCs), and so on.
  • the computer systems may be cell phones, personal digital assistants, smart phones, personal computers, programmable consumer electronics, digital cameras, and so on.
  • the system may be described in the general context of computer-executable instructions, such as program modules, executed by one or more computers or other devices.
  • program modules include routines, programs, objects, components, data structures, and so on that perform particular tasks or implement particular abstract data types.
  • functionality of the program modules may be combined or distributed as desired in various embodiments.
  • FIG. 3 is a flow diagram that illustrates processing of the cloud management system to handle a request from a management tool to access data from distributed application instances, in one embodiment.
  • the system receives from a management tool a request to access management data related to an application running instances in one or more datacenters.
  • a performance-monitoring tool may request status information describing how many client requests the application is handling, resource usage of the application, or other information from the application.
  • the system may receive the tool request through an API the system exposes to tools for requesting management data.
  • the API may comprise a uniform interface for accessing management data irrespective of where or at how many locations application instances are running.
  • the system identifies one or more types of management data that satisfy the received request. For example, the system may determine that the request asks for log information that is produced by each instance of the application.
  • Identifying the requested data allows the system to determine which information to gather from each application instance or whether the data is already gathered locally from data pushed to a central location by each application instance.
  • the system determines a distribution of the application that includes two or more instances of the application.
  • the distribution determines where the application is running and where the system will find management data to satisfy the request.
  • the system may include a data store that tracks information describing each burst or other migration of application load to and from other datacenters, so that the system is aware of each location where application instances are running. Upon receiving the management tool request, this information allows the system to determine from where to gather management data.
  • the system gathers management data to satisfy the request from each distributed application instance.
  • the instances may include an instance in a local private datacenter, a remote private datacenter, a private cloud computing facility, a public cloud computing facility, spare resources offered by other private datacenters, and so on.
  • the system contacts each instance of the application or accesses previously sent information from each instance that contains information (such as performance data, failures, and so forth) to satisfy the received management tool request.
  • the system optionally sends one or more
  • troubleshooting commands to one or more remote application instances. For example, if one location is experiencing failures, the administrator may use a management tool to request additional trace information, to send one or more test requests, or to perform other types of debugging.
  • the remote application instances carry out the troubleshooting commands and report requested data back to a central location where the management tool can access the information.
  • the system unifies gathered data to provide a uniform response to the received management tool request.
  • management tools need not be written to include an understanding of the various potential distributions of applications managed by the tools.
  • the system can thus freely migrate the application from location to location or to multiple locations as needed to handle application loads, while still providing administrators with a straightforward management and
  • management data in response to the received management tool request.
  • the system may send the data through the interface on which the request was received or through a notification interface or other facility for providing data to the tool.
  • FIG. 4 is a flow diagram that illustrates processing of the cloud management system to report data back from and handle troubleshooting requests at a location of a remote application instance, in one embodiment.
  • the system receives management data at a remote application instance handling a portion of load generated by requests from clients of the application.
  • the management data may include performance data, log information, error details, statistical information, sales history, or other indications of application operation useful for management the application.
  • the system determines a home location of the application where an administrator can access management data reported by multiple instances of the application running at distributed remote locations.
  • the application instance may receive configuration information from the home location upon creation of the instance that specifies where the home location can be contacted and that the application instance is a remote instance of the application.
  • the system may migrate applications to multiple locations to handle peak loads, perform low priority tasks at locations where processing is off-peak and thus cheaper, or for other reasons determined by an administrator.
  • the application may have a home location that is where the application normally runs and may handle peak or other loads at one or more distributed remote locations.
  • the system sends the received management data from the remote application instance to the determined home location of the application.
  • the system may periodically migrate data generated at distributed instances back to the home location so that management data is available in one place at the home location for the convenience of administrators and management tools.
  • the system may also migrate data on demand or as requested by various tools (see, e.g., Figure 3).
  • the system may burst application loads to remote locations for short durations and then collect information related to the application's execution when the loads are migrated back to the home location and the remote instances are finished.
  • the system optionally receives a troubleshooting request from a management tool run at the home location to troubleshoot the remote application instance.
  • the troubleshooting requests may include debug breakpoints, a request for detailed trace information, or other commands or requests to perform troubleshooting actions.
  • the system performs one or more troubleshooting actions in response to the received troubleshooting request.
  • the action may include setting a debug breakpoint, turning up a logging level, sending test data to the application, or performing any other action specified by the request for determining whether the application is operating properly.
  • the system sends a troubleshooting result to the home location in response to the received troubleshooting request.
  • the system allows a troubleshooting tool operating at the home location to troubleshoot application instances no matter where the instances are running, and allows the system to seamlessly migrate instances of the application to various locations without interrupting the ability of an administrator to manage and troubleshoot the application.
  • the cloud management system migrates application load by modifying domain name service (DNS) records.
  • DNS domain name service
  • the system may modify a DNS server to point incoming client requests to one or more new destination Internet Protocol (IP) addresses to direct loads away from a source datacenter over to a target datacenter/cloud.
  • IP Internet Protocol
  • a global traffic manager (GTM) often points clients to the nearest server for handling their requests, and these solutions can be modified to redirect traffic based on load or other conditions.
  • GTM global traffic manager
  • the system may inform the GTM to direct at least some client requests to a new location that can handle the excess load.
  • the system may provide a DNS or other address to which management tools can address management requests and be connected to application instances no matter where they reside.
  • the cloud management system migrates log and other data back from the target computing resources after migration conditions have eased. For example, following a period of peak load, the system may migrate all application loads back to the original datacenter, and may pull information generated at the target datacenter, such as application logs, back to the original datacenter for later analysis. For some applications, tracking client requests may be a matter of regulatory compliance or simply useful for debugging and reporting. In either case, consolidating the logs at the source location may be part of a successful migration back to the source location.
  • the cloud management system allocates a dynamically variable amount of application load between a source computing resource and one or more target computing resources. For example, the system may dynamically route requests to keep the source computing resource at or near full capacity and only send requests out to external computing resources that the source computing resource could not successfully handle. Such decisions may be a matter of cost, data safety, or other considerations to migrate out as little application load as needed or to place application loads where they can be performed cheapest or most efficiently. In some cases, the decisions may be based on regulatory requirements of applications. For example, applications subject to healthcare or other recordkeeping laws may have restrictions about the datacenters/clouds in which they can operate.
  • the cloud management system provides various options for disaster recovery.
  • the system may enlist resources at an external datacenter to monitor a main datacenter for outages. If the external datacenter becomes unable to reach the main datacenter, then the external datacenter may determine that a disaster has occurred and move application loads to the external datacenter.
  • the organization can maintain a lower amount of available capacity at a second location (e.g., 10%) and can rapidly request more as needed in the event of a failure.
  • the likelihood of all clients of a cloud provider failing at the same time and requesting a high capacity spare is low, such that multiple clients can share a set of redundant secondary resources to be used in the event of failure of primary resources.
  • the system may also re-home management tools and troubleshooting resources to point to the new location following disaster recovery so that management continues uninterrupted.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Quality & Reliability (AREA)
  • Software Systems (AREA)
  • Computer Hardware Design (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Mathematical Physics (AREA)
  • Signal Processing (AREA)
  • Data Mining & Analysis (AREA)
  • Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Debugging And Monitoring (AREA)
  • Computer And Data Communications (AREA)

Abstract

A cloud management system is described herein that provides the ability for an application to span two or more clouds while allowing operation, management, and troubleshooting of the distributed application as a single application. The system provides infrastructure that communicates across datacenters for execution and for centralizing knowledge of instances of an application that are running at different locations. The infrastructure provided by the system monitors both the application and connections between the clouds, with intelligence to know if issues are within the application or because of the connection between the clouds. The system coordinates management functions across multiple cloud platforms/locations. Thus, the cloud management system creates a single monitoring and troubleshooting interface and knowledge and execution fabric across multiple clouds so that applications spread across multiple clouds can be monitored, managed, and debugged more easily.

Description

CROSS-CLOUD MANAGEMENT AND TROUBLESHOOTING
BACKGROUND
[0001] Datacenters provide servers for running large applications. Enterprises often use datacenters to run core business functions such as sales, marketing, human resources, billing, product catalogs, and so forth. Datacenters may also run customer-facing applications, such as web sites, web services, email hosts, databases, and many other applications. Datacenters are typically built by determining an expected peak load and providing servers, network infrastructure, cooling, and other resources to handle the peak load level. Datacenters are known for being very expensive and for being underutilized at non-peak times. They also involve a relatively high management expense in terms of both equipment and personnel for monitoring and performing maintenance on the datacenter. Because almost every enterprise uses a datacenter of some sort, there are many redundant functions performed by organizations across the world.
[0002] Cloud computing has emerged as one optimization of the traditional datacenter. A cloud is defined as a set of resources (e.g., processing, storage, or other resources) available through a network that can serve at least some traditional datacenter functions for an enterprise. A cloud often involves a layer of abstraction such that the applications and users of the cloud may not know the specific hardware that the applications are running on, where the hardware is located, and so forth. This allows the cloud operator some additional freedom in terms of rotating resources into and out of service, maintenance, and so on. Clouds may include public clouds, such as MICROSOFT TM Azure, Amazon Web Services, and others, as well as private clouds, such as those provided by Eucalyptus Systems, MICROSOFT TM, and others. Companies have begun offering appliances (e.g., the MICROSOFT TM Azure Appliance) that enterprises can place in their own datacenters to connect the datacenter with varying levels of cloud functionality.
[0003] Enterprises with datacenters incur substantial costs building out large datacenters, even when cloud-based resources are leveraged. Enterprises often still planned for "worst-case" peak scenarios and thus include an amount of hardware at least some of which is rarely used or underutilized in terms of extra processing capacity, extra storage space, and so forth. This extra amount of resources incurs a high cost for little return. Customers using cloud based computing on premise expect to be able to use capacity in another compatible cloud (e.g., a second instance of their own in another location, Microsoft's public cloud, and so forth) for peak capacity times, for disaster recover scenarios, or just for capacity management. Doing so is much less expensive than building out for the worst-case scenario and then doubling for redundancy. In addition, they expect to be able to manage (e.g., troubleshoot, operate) applications split across multiple clouds. Today, applications, cloud management, and troubleshooting do not operate across clouds or other datacenters.
SUMMARY
[0004] A cloud management system is described herein that provides the ability for an application to span two or more clouds (which may be across great distances), while allowing operation, management, and troubleshooting of the distributed application as a single application. The system provides infrastructure that communicates across datacenters for execution and for centralizing knowledge of instances of an application that are running at different locations. In some cases, the system provides a computing appliance that an enterprise can place in its own private datacenter that allows an administrator to distribute at least some application loads to a public cloud or other separate locations, while providing unified management via the computing appliance. The infrastructure provided by the system monitors both the application and connections between the clouds, with intelligence to know if issues are within the application or because of the connection between the clouds. The system coordinates management functions across multiple cloud platforms/locations. If an administrator wants to debug the application, the system allows live debugging at the correct location through a seamless, unified interface. Thus, the cloud management system creates a single monitoring and troubleshooting interface and knowledge and execution "fabric" across multiple clouds so that applications spread across multiple clouds can be monitored, managed, and debugged more easily.
[0005] This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.
BRIEF DESCRIPTION OF THE DRAWINGS
[0006] Figure 1 illustrates an application running in two clouds with associated management infrastructure, in one embodiment.
[0007] Figure 2 is a block diagram that illustrates components of the cloud management system, in one embodiment. [0008] Figure 3 is a flow diagram that illustrates processing of the cloud management system to handle a request from a management tool to access data from distributed application instances, in one embodiment.
[0009] Figure 4 is a flow diagram that illustrates processing of the cloud management system to report data back from and handle troubleshooting requests at a location of a remote application instance, in one embodiment.
DETAILED DESCRIPTION
[0010] A cloud management system is described herein that provides the ability for an application to span two or more clouds (which may be across great distances), while allowing operation, management, and troubleshooting of the distributed application as a single application. The system provides infrastructure that communicates across datacenters for execution and for centralizing knowledge of instances of an application that are running at different locations. For example, the system may centralize logging, performance tracking, and other management functions, regardless of where the application is running. In some cases, the system provides a computing appliance that an enterprise can place in its own private datacenter that allows an administrator to distribute at least some application loads to a public cloud or other separate locations, while providing unified management via the computing appliance.
[0011] The infrastructure provided by the cloud management system monitors both the application and connections between the clouds, with intelligence to know if issues are within the application or because of the connection between the clouds. The system coordinates management functions across multiple cloud platforms/locations (from one cloud's infrastructure, tasks are coordinated to execute across two or more clouds). If an administrator wants to debug the application, the system allows live debugging at the correct location through a seamless, unified interface. Thus, the cloud management system creates a single monitoring and troubleshooting interface and knowledge and execution "fabric" across multiple clouds so that applications spread across multiple clouds can be monitored, managed, and debugged more easily.
[0012] Figure 1 illustrates an application running in two clouds with associated management infrastructure, in one embodiment. In some embodiments, the cloud management system involves the application (and/or the administrator) using
infrastructure in one cloud that has the data/access to the data at all locations to be able to fully monitor/troubleshoot the application. As an example, consider an application with instances running in two clouds, cloud 110 and cloud 150 as shown in Figure 1. Cloud 110 includes a MICROSOFT TM Azure appliance instance 120 that includes
infrastructure 130. The appliance instance 120 includes application instance 125 that is running role 140 and role 145. A second cloud 150 includes application instance 155 that is running role 160and role 170. The second cloud 150 also includes infrastructure 180. The appliance instance 120 knows about each of the roles and that they are part of the same application. The infrastructure plumbing at each location allows the appliance instance 120 to retrieve information about role 160 and role 170 executing at the second cloud 150. The system can distribute either individual roles, whole applications, or both. With all of the management data (e.g., logs from the applications, machines, and infrastructure), the system can assess the health of the application just as if all of the roles were local by applying pre-defined health rules. The system can also see the infrastructure health across both locations as well as the connection 190 in between to assess if a problem is occurring with the application or the infrastructure/network.
[0013] Similarly, when automated or manual troubleshooting or remediation steps are needed, the infrastructure 130 in cloud 110 can coordinate with the infrastructure 180 in cloud 150 to provide troubleshooting and debugging support. For example, the system fabric can reach across locations to execute an application wide update, shutdown, and so forth. Those of ordinary skill in the art will recognize numerous ways to perform cross- location control. For example, infrastructure 130 may directly control infrastructure 180, infrastructure 130 may request infrastructure 180 to execute on infrastructure 130's behalf, and so forth. Likewise, with operator/administrator troubleshooting tools (e.g., monitoring visualization, alerting, log and configuration data viewing, and so on), the location of the applications and infrastructure is available and logically displayed, but does not involve separate tools and mental gymnastics from the administrator to put together. For instance, when troubleshooting and viewing data on all roles, if the administrator 105's next step is using one or more tools 195 to view the application's logs or start a remote session to the role instance, the system connects the administrator 105 directly, regardless of at which location the role is residing.
[0014] The design of the cloud management system provides simplified and consistent running of a service across multiple clouds/location. The system moves the definition of "a computing resource" from a server, beyond a datacenter to a portion of the internet (the datacenters and the connection between them). This allows service level agreements (SLAs) to be defined, monitored, and managed at the service level - which is what service owners often care most about. [0015] In some embodiments, the cloud management system works in cooperation with a cloud migration system that seamlessly migrates applications from one location to another as needed, called bursting. The cloud migration system provides capacity management and disaster recovery by detecting peak load conditions and automatically moving computing to another source (and back) and by providing computing across two or more clouds and moving completely to one in the case of a disaster at one site. This allows enterprises to plan for local resources for a sustained level of load and to leverage cloud-based resources for peak or other unusual loads. In many cases, an enterprise's business is such that a particular time of year is busier, and extra resources may only be needed during those times. For example, tax-planning enterprises are particular busy in mid- April, e-commerce sites experience holiday rushes around Thanksgiving and
Christmas, and so on. The cloud migration system monitors loads within a datacenter and detects a threshold that indicates that the current load is nearing the datacenter' s capacity. For example, the system may monitor central processing unit (CPU) usage, memory usage, storage usage, network bandwidth, and other metrics to determine how well the datacenter is handling the current load. The system may also observe trends (e.g., a rate of acceleration of resource usage) to determine whether the threshold has been or will soon be reached.
[0016] Upon detecting that the threshold will be reached, the cloud migration system facilitates an orderly move of at least some datacenter load to another datacenter or cloud- based resources. For example, the system may migrate some peak load to a public cloud. Because cloud pricing models may vary, the system may factor cost into the decision. For example, the system may prefer to host as much load as possible at the enterprise datacenter to reduce cost, while leveraging cloud resources only to the extent needed to satisfy client requests. The system may also provide management and monitoring tools that provide a consistent experience for information technology (IT) personnel regardless of where particular loads are run (e.g., locally within the enterprise or publicly using a cloud). The system may also provide planning tools to help decide appropriate workloads or applications for moving to other resources during high loads. For example, applications may have various compliance/regulatory or networking/design limitations that make them more or less suitable for migration. The system can also be used as a disaster recovery architecture at a datacenter/network level to manage fast workload transition in case of disaster. If a datacenter resource permanently fails, the system can quickly and efficiently migrate additional load to the cloud or other resources so that clients of the datacenter are unaffected or are less affected by the failure. Thus, the cloud migration system allows enterprises to build smaller and more efficient datacenters that leverage other resources for rare extra loads.
[0017] The cloud management system works with the cloud migration system to provide seamless management and troubleshooting as applications are migrated from one location to another. As described above, the cloud migration system may move resources between a datacenter and the cloud on a temporary (i.e., bursting) or permanent (i.e., disaster recovery) basis. Temporary movements include bursting an application or other load for a short time period to handle a peak or other high load that exceeds the datacenter' s capacity. A temporary movement may include bursting an entire application or splitting the application's load across two or more locations. Permanent movements include longer-term migration of loads due to a failure of hardware in the datacenter, a more sustained increase in capacity needs, a desire to globally distribute an application with dynamic load balancing, and so forth. Following are several example scenarios in which the system may be used by an enterprise.
[0018] In the first example, an enterprise bursts application load to a public cloud to manage capacity. The business decision maker (i.e., CEO, CFO, or VP Marketing/sales) and datacenter systems administrator decide it would be more cost effective and provide a better customer experience to burst some work to the public cloud at their top three peak day level of use/traffic per year, and maintain their own datacenter (potentially with a cloud appliance) at their peak monthly usage level. They sign business agreements with the cloud provider to burst work into the cloud and project estimates of when and how much work that would be. Their account is set up and the information entered into the cloud appliance. During a planning phase, the administrator runs a test with a test application from the cloud provider that ensures the connection is working properly. The administrator then sets the capacity values (e.g., threshold) for initiating bursting of applications that keep capacity at the specified level in a capacity management tool. The administrator goes into the tool to further specify the applications that are eligible to move in this situation (e.g., no regulatory issues with temporary movement, good technical fit).
[0019] The day comes when use exceeds the limits and the system automatically moves applications to the public cloud. Alerts are thrown in the monitoring/usage systems when capacity is within 5% of the bursting being initiated, when the system does burst, what the system bursts, and when the system brings the applications back. An explicit log is kept of all compute resources and/or storage moved and the administrator is alerted to go to their public cloud account for billing. A review of the bursting parameters and
applications tagged as movable is reviewed in the regular capacity planning meetings with in the enterprise datacenter group and management.
[0020] In a second example, an enterprise splits applications across clouds to manage capacity. This scenario is similar to the scenario above except for the type of application moved is more complex, so is split out to prioritize differently. The company decides to have a relationship with the cloud provider for splitting applications into the cloud (a form of bursting). In this large application was pre-identified as a bursting candidate.
When the capacity reaches the threshold, 50 of the 100 worker instances are automatically moved to the public cloud. The application is now split across two appliance instances or cloud instances, with all monitoring and billing data being sent to the starting instance so it can be centrally managed. A cloud appliance in the enterprise's own datacenter has troubleshooting tools to help debug possible issues of split applications (e.g., networking issues, network bandwidth/latency issues, fabric communication, and so on). When the capacity situation has subsided on the appliance, the 50 worker instances are moved back to the appliance and it is a normally functioning application again.
[0021] In another example, a cloud provider decides to burst from one cluster to another. The public cloud capacity planning team decides that a cluster in the Chicago datacenter is critically full but wants to maintain high utilization. They set up bursting to an
underutilized cluster in a West coast datacenter when utilization gets to 90%. The administrator goes into the capacity management tool and chooses appropriate
customers/applications (e.g., with low data usage) to be move candidates. The day comes when the usage of the Chicago cluster reaches the threshold and the system automatically moves the selected applications (e.g., 10% of the cluster's apps) to the West coast datacenter for one day. As usage returns below the threshold, the system moves the applications back to Chicago. The system notifies a designated monitoring team proactively of the burst to be able to answer customer questions.
[0022] In another example, the system is used for cross-cloud portfolio management. An enterprise decides that to manage capacity efficiently on their cloud appliance, they want to put all variable demand apps in a public cloud and their constant demand applications in the appliance or local datacenter resources (and thus be able to run the appliance at higher utilization). While they want their computing resources split, they still want a global view across all of their application's health, to have their application developers manage applications the same way, and to maintain a single view of departmental billing across both (e.g., what costs to allocate to the consumer sales groups, internal IT, B2B sales, and so forth). The enterprise is able to set up the aggregation accounts with the public cloud with the same groups as the appliance and get billing data to integrate on their side. Similarly, they are able to get application-programming interface (API) access to the public cloud monitoring data for the platform their apps are running on as well as application level monitoring, so their network operation center (NOC) has a complete and consistent view of the state of the computing activity of the enterprise.
[0023] In another example, an enterprise sets up a globally distributed application with dynamic load balancing. An enterprise customer wants to manage capacity across two or more cloud instances and has a significant amount of their load in independent but geographically distributed instances (e.g., Bing search with a US and UK datacenter that both serve German queries). Under normal circumstances, a global traffic manager sends 50% traffic to each location. When load gets high at the primary location, the system instructs the load balancer to send 75% of the traffic to the UK system, thus freeing up capacity from the US cloud instance, bringing it to acceptable levels. When capacity returns to normal, the system tells the load balancer to return to the 50/50 split. A variation of this is for the public cloud to be used as a secondary datacenter (with say 1% of load, the customer's site with the appliance to be the other 99%). In the case of a disaster or other reason to move load from the customer's site, 100% of traffic is shifted to the public cloud.
[0024] In another example, an enterprise has reached its datacenter' s capacity and needs extra computing resources, but does not yet have the available capital to expend to expand the datacenter. In this case, the company can use a public cloud for spillover until they can get the hardware purchase completed.
[0025] Figure 2 is a block diagram that illustrates components of the cloud management system, in one embodiment. The system 200 includes a location management component 210, location data store 220, tool interface component 230, one or more management tools 240, a data migration component 250, a troubleshooting component 260, and a billing component 270. Each of these components is described in further detail herein.
[0026] The location management component 210 manages information about multiple datacenter locations at which instances of an application are running. The component 210 includes information describing how to reach each location, connections available for retrieving management information, user accounts to use for each location with associated security credentials, application and datacenter components from which to gather troubleshooting information and send troubleshooting commands, and so forth. The location management component 210 receives information describing any migration of application loads or bursting from one datacenter/cloud to another and updates the managed information so that the component 210 has a complete picture of all of the locations where the application is running. This allows the system 200 to present the complete picture and to make management of applications uniform, no matter where or at how many locations the applications are running. As conditions change and applications are distributed, the location management component 210 can present management tools with a comprehensive set of management data.
[0027] The location data store 220 stores information that describes locations at which instances of the application are running. The data store 220 may include one or more files, file system, hard drives, databases, cloud-based storage services, or other facilities for persisting information between sessions with the system 200. The stored information may include connection information, user roles, sources of management data, available log files, and any other information related to management or troubleshooting of applications distributed to multiple locations.
[0028] The tool interface component 230 provides an interface to the system 200 through which one or more tools can access management and troubleshooting information for the application. The interface may include one or more web pages, web services, application-programming interfaces (APIs), or other interfaces through which an administrator or tools can directly or programmatically access management and
troubleshooting information of the system 200. In some embodiments, the tool interface component 230 provides an initial connection point for tools to access information related to the application at a cloud-computing appliance located within an enterprise's private datacenter. The appliance may manage migration and distribution of application loads to a public cloud or other datacenter, and provides a central point of contact for tools that gather management information or provide application troubleshooting.
[0029] One or more management tools 240 connect to the tool interface component 230 to access management information or perform application troubleshooting. The tools may include log viewers, reporting tools, debugging tools, or other tools that display
information about or assist in resolving problems with a running application. The management tools 240 may include tools designed to work with a local application, and the system 200 provides the tools with information describing a distributed application running at multiple locations without the tool's knowledge. This allows existing tools that administrators rely upon to be used even as automatic application load migration is introduced into a datacenter or cloud. In other cases, tools may be specifically written to understand distributed applications and to provide specific management information or troubleshooting related to multiple locations. The tool interface component 230 may provide multiple interfaces through which management tools 240 connect to the system 200 using paradigms that are understood by each tool.
[0030] The data migration component 250 migrates management information at one or more remote locations where the application is running back to a home location of the application. The home location may include a private datacenter, location or a cloud- computing appliance, or other location where the application normally runs under steady conditions. Upon hitting a certain level of load (e.g., peak or periodic bursts), the application may migrate some load to one or more other datacenters or public clouds to help satisfy client requests. These other locations generate management data, such as log files, transaction data, and so on, just like the home location, and the data migration component 250 migrates this data back to the home location or provides access to the data from the home location, so that the management tools 240 can provide a comprehensive picture of the application's activity to administrators.
[0031] The troubleshooting component 260 performs troubleshooting tasks on the application at one or more locations. Troubleshooting may include debugging, processing test data, or other forms of determining whether an application is operating correctly. Troubleshooting is generally well understood at the home location, but becomes more complex as an application begins to span multiple datacenters or clouds. The cloud management system 200 isolates management tools 240 and administrators from this complexity by providing a uniform interface through which tools and administrators access management information and perform troubleshooting at multiple locations. Thus, if a management tool allows an administrator to place a breakpoint at or receive trace information from a particular piece of application code at the home location, then the troubleshooting component 260 makes it just as easy to do so at a remote cloud-based instance of the application. The tools and administrator may even be unaware of all of the locations where the application is running, but can still perform management tasks as if the application were only executing at the home location.
[0032] The billing component 270 reports billing information related to one or more locations where an application is running. One common management task is to manage computing costs, and public clouds often charge based on metrics related to workload (e.g., computing time, storage space used, and so forth). It may be useful for an administrator to gather a picture of the costs that application instances are incurring at various locations, and the cloud management system 200 can optionally provide the billing component 270 to gather this type of information so that the information can be reported through management tools and reports.
[0033] The computing device on which the cloud management system is implemented may include a central processing unit, memory, input devices (e.g., keyboard and pointing devices), output devices (e.g., display devices), and storage devices (e.g., disk drives or other non-volatile storage media). The memory and storage devices are computer- readable storage media that may be encoded with computer-executable instructions (e.g., software) that implement or enable the system. In addition, the data structures and message structures may be stored or transmitted via a data transmission medium, such as a signal on a communication link. Various communication links may be used, such as the Internet, a local area network, a wide area network, a point-to-point dial-up connection, a cell phone network, and so on.
[0034] Embodiments of the system may be implemented in various operating environments that include personal computers, server computers, handheld or laptop devices, multiprocessor systems, microprocessor-based systems, programmable consumer electronics, digital cameras, network PCs, minicomputers, mainframe computers, distributed computing environments that include any of the above systems or devices, set top boxes, systems on a chip (SOCs), and so on. The computer systems may be cell phones, personal digital assistants, smart phones, personal computers, programmable consumer electronics, digital cameras, and so on.
[0035] The system may be described in the general context of computer-executable instructions, such as program modules, executed by one or more computers or other devices. Generally, program modules include routines, programs, objects, components, data structures, and so on that perform particular tasks or implement particular abstract data types. Typically, the functionality of the program modules may be combined or distributed as desired in various embodiments.
[0036] Figure 3 is a flow diagram that illustrates processing of the cloud management system to handle a request from a management tool to access data from distributed application instances, in one embodiment. Beginning in block 310, the system receives from a management tool a request to access management data related to an application running instances in one or more datacenters. For example, a performance-monitoring tool may request status information describing how many client requests the application is handling, resource usage of the application, or other information from the application. The system may receive the tool request through an API the system exposes to tools for requesting management data. The API may comprise a uniform interface for accessing management data irrespective of where or at how many locations application instances are running.
[0037] Continuing in block 320, the system identifies one or more types of management data that satisfy the received request. For example, the system may determine that the request asks for log information that is produced by each instance of the application.
Identifying the requested data allows the system to determine which information to gather from each application instance or whether the data is already gathered locally from data pushed to a central location by each application instance.
[0038] Continuing in block 330, the system determines a distribution of the application that includes two or more instances of the application. The distribution determines where the application is running and where the system will find management data to satisfy the request. The system may include a data store that tracks information describing each burst or other migration of application load to and from other datacenters, so that the system is aware of each location where application instances are running. Upon receiving the management tool request, this information allows the system to determine from where to gather management data.
[0039] Continuing in block 340, the system gathers management data to satisfy the request from each distributed application instance. The instances may include an instance in a local private datacenter, a remote private datacenter, a private cloud computing facility, a public cloud computing facility, spare resources offered by other private datacenters, and so on. The system contacts each instance of the application or accesses previously sent information from each instance that contains information (such as performance data, failures, and so forth) to satisfy the received management tool request.
[0040] Continuing in block 350, the system optionally sends one or more
troubleshooting commands to one or more remote application instances. For example, if one location is experiencing failures, the administrator may use a management tool to request additional trace information, to send one or more test requests, or to perform other types of debugging. The remote application instances carry out the troubleshooting commands and report requested data back to a central location where the management tool can access the information.
[0041] Continuing in block 360, the system unifies gathered data to provide a uniform response to the received management tool request. In this way, management tools need not be written to include an understanding of the various potential distributions of applications managed by the tools. The system can thus freely migrate the application from location to location or to multiple locations as needed to handle application loads, while still providing administrators with a straightforward management and
troubleshooting experience.
[0042] Continuing in block 370, the system reports the gathered and unified
management data in response to the received management tool request. The system may send the data through the interface on which the request was received or through a notification interface or other facility for providing data to the tool. After block 370, these steps conclude.
[0043] Figure 4 is a flow diagram that illustrates processing of the cloud management system to report data back from and handle troubleshooting requests at a location of a remote application instance, in one embodiment. Beginning in block 410, the system receives management data at a remote application instance handling a portion of load generated by requests from clients of the application. The management data may include performance data, log information, error details, statistical information, sales history, or other indications of application operation useful for management the application.
[0044] Continuing in block 420, the system determines a home location of the application where an administrator can access management data reported by multiple instances of the application running at distributed remote locations. The application instance may receive configuration information from the home location upon creation of the instance that specifies where the home location can be contacted and that the application instance is a remote instance of the application. The system may migrate applications to multiple locations to handle peak loads, perform low priority tasks at locations where processing is off-peak and thus cheaper, or for other reasons determined by an administrator. The application may have a home location that is where the application normally runs and may handle peak or other loads at one or more distributed remote locations.
[0045] Continuing in block 430, the system sends the received management data from the remote application instance to the determined home location of the application. The system may periodically migrate data generated at distributed instances back to the home location so that management data is available in one place at the home location for the convenience of administrators and management tools. The system may also migrate data on demand or as requested by various tools (see, e.g., Figure 3). In some cases, the system may burst application loads to remote locations for short durations and then collect information related to the application's execution when the loads are migrated back to the home location and the remote instances are finished.
[0046] Continuing in block 440, the system optionally receives a troubleshooting request from a management tool run at the home location to troubleshoot the remote application instance. The troubleshooting requests may include debug breakpoints, a request for detailed trace information, or other commands or requests to perform troubleshooting actions.
[0047] Continuing in block 450, the system performs one or more troubleshooting actions in response to the received troubleshooting request. The action may include setting a debug breakpoint, turning up a logging level, sending test data to the application, or performing any other action specified by the request for determining whether the application is operating properly.
[0048] Continuing in block 460, the system sends a troubleshooting result to the home location in response to the received troubleshooting request. By providing a facility for executing troubleshooting commands remotely, the system allows a troubleshooting tool operating at the home location to troubleshoot application instances no matter where the instances are running, and allows the system to seamlessly migrate instances of the application to various locations without interrupting the ability of an administrator to manage and troubleshoot the application. After block 460, these steps conclude.
[0049] In some embodiments, the cloud management system migrates application load by modifying domain name service (DNS) records. The system may modify a DNS server to point incoming client requests to one or more new destination Internet Protocol (IP) addresses to direct loads away from a source datacenter over to a target datacenter/cloud. A global traffic manager (GTM) often points clients to the nearest server for handling their requests, and these solutions can be modified to redirect traffic based on load or other conditions. Thus, when one datacenter becomes overloaded or near capacity, the system may inform the GTM to direct at least some client requests to a new location that can handle the excess load. Similarly, the system may provide a DNS or other address to which management tools can address management requests and be connected to application instances no matter where they reside.
[0050] In some embodiments, the cloud management system migrates log and other data back from the target computing resources after migration conditions have eased. For example, following a period of peak load, the system may migrate all application loads back to the original datacenter, and may pull information generated at the target datacenter, such as application logs, back to the original datacenter for later analysis. For some applications, tracking client requests may be a matter of regulatory compliance or simply useful for debugging and reporting. In either case, consolidating the logs at the source location may be part of a successful migration back to the source location.
[0051] In some embodiments, the cloud management system allocates a dynamically variable amount of application load between a source computing resource and one or more target computing resources. For example, the system may dynamically route requests to keep the source computing resource at or near full capacity and only send requests out to external computing resources that the source computing resource could not successfully handle. Such decisions may be a matter of cost, data safety, or other considerations to migrate out as little application load as needed or to place application loads where they can be performed cheapest or most efficiently. In some cases, the decisions may be based on regulatory requirements of applications. For example, applications subject to healthcare or other recordkeeping laws may have restrictions about the datacenters/clouds in which they can operate.
[0052] In some embodiments, the cloud management system provides various options for disaster recovery. In some cases, the system may enlist resources at an external datacenter to monitor a main datacenter for outages. If the external datacenter becomes unable to reach the main datacenter, then the external datacenter may determine that a disaster has occurred and move application loads to the external datacenter. In past systems, it was typical for an organization to maintain 200% of needed capacity (at substantial expense) in order to successfully handle disasters. With the cloud management system, the organization can maintain a lower amount of available capacity at a second location (e.g., 10%) and can rapidly request more as needed in the event of a failure.
Much like insurance, the likelihood of all clients of a cloud provider failing at the same time and requesting a high capacity spare is low, such that multiple clients can share a set of redundant secondary resources to be used in the event of failure of primary resources. The system may also re-home management tools and troubleshooting resources to point to the new location following disaster recovery so that management continues uninterrupted.
[0053] From the foregoing, it will be appreciated that specific embodiments of the cloud management system have been described herein for purposes of illustration, but that various modifications may be made without deviating from the spirit and scope of the inventon. Accordingly, the invention is not limited except as by the appended claims.

Claims

1. A computer-implemented method to handle a request from a management tool to access application management data from distributed application instances, the method comprising:
receiving from an application management tool a request to access management data related to an application running instances in one or more datacenters;
identifying one or more types of management data that satisfy the received request; determining a distribution of the application that includes two or more instances of the application;
gathering management data to satisfy the request from each distributed application instance;
unifying gathered data to provide a uniform response to the received management tool request; and
reporting the gathered and unified management data in response to the received management tool request,
wherein the preceding steps are performed by at least one processor.
2. The method of claim 1 wherein receiving the management request comprises receiving the tool request through a programmatic application programming interface (API) exposed to tools for requesting management data.
3. The method of claim 1 wherein the API comprises a uniform interface for accessing management data without requiring tools to understand where or at how many locations application instances are running.
4. The method of claim 1 wherein receiving the management request comprises receiving a request from a performance monitoring tool to access status information describing operation of one or more application instances.
5. The method of claim 1 wherein identifying types of management data comprises determining that the request asks for information that is produced by each instance of the application.
6. The method of claim 1 wherein identifying types of management data comprises determining which information to gather from each application instance and whether the data is already gathered locally from data pushed to a central location by each application instance.
7. The method of claim 1 wherein determining the distribution of the application comprises determining where the application is running and where the system will find management data to satisfy the request.
8. The method of claim 1 wherein gathering management data comprises accessing at least one instance in a private datacenter and at least one instance in a cloud computing facility.
9. The method of claim 1 wherein gathering management data comprises contacting each instance of the application to satisfy the received management tool request.
10. The method of claim 1 further comprising sending one or more troubleshooting commands to one or more remote application instances, wherein the remote application instances carry out the troubleshooting commands and report requested data back to a central location where the management tool can access the information.
11. The method of claim 1 wherein unifying gathered data comprises formatting data so that management tools need not be written to include an understanding of the various potential distributions of applications managed by the tools.
12. The method of claim 1 wherein unifying gathered data comprises formatting data so that the system can freely migrate the application from location to location or to multiple locations as needed to handle application loads, while still providing
administrators with a consistent management and troubleshooting experience.
13. The method of claim 1 wherein reporting the data comprises sending the data to the management tool through an interface on which the request was received.
14. A computer system for cross-cloud computing for cross-cloud management and troubleshooting, the system comprising:
a processor and memory configured to execute software instructions embodied within the following components;
a location management component that manages information about multiple datacenter locations at which instances of an application are running;
a location data store that stores information that describes locations at which instances of the application are running;
a tool interface component that provides an interface to the system through which one or more tools can access management and troubleshooting information for the application;
one or more management tools that connect to the tool interface component to access management information or perform application troubleshooting; a data migration component that migrates management information at one or more remote locations where the application is running back to a home location of the application; and
a troubleshooting component that performs one or more troubleshooting tasks on the application at one or more locations.
15. The system of claim 14 wherein the tool interface component provides an initial connection point for tools to access information related to the application at a cloud- computing appliance located within an enterprise's private datacenter.
PCT/US2012/038647 2011-05-20 2012-05-18 Cross-cloud management and troubleshooting WO2012162171A2 (en)

Priority Applications (10)

Application Number Priority Date Filing Date Title
JP2014512899A JP5980914B2 (en) 2011-05-20 2012-05-18 Mutual cloud management and fault diagnosis
MX2017004805A MX366620B (en) 2011-05-20 2012-05-18 Cross-cloud management and troubleshooting.
MX2013013577A MX347110B (en) 2011-05-20 2012-05-18 Cross-cloud management and troubleshooting.
KR1020137030613A KR101916847B1 (en) 2011-05-20 2012-05-18 Cross-cloud management and troubleshooting
CN201280024366.1A CN103548009B (en) 2011-05-20 2012-05-18 Method and system for cross-cloud management and troubleshooting
EP12789586.0A EP2710484B1 (en) 2011-05-20 2012-05-18 Cross-cloud management and troubleshooting
AU2012259086A AU2012259086B2 (en) 2011-05-20 2012-05-18 Cross-cloud management and troubleshooting
RU2013151607/08A RU2604519C2 (en) 2011-05-20 2012-05-18 Cross-cloud management and troubleshooting
CA2835440A CA2835440A1 (en) 2011-05-20 2012-05-18 Cross-cloud management and troubleshooting
BR112013029716-6A BR112013029716B1 (en) 2011-05-20 2012-05-18 COMPUTER IMPLEMENTED METHOD TO HANDLE A REQUEST FOR A COMPUTER MANAGEMENT TOOL AND COMPUTER SYSTEM TO ACCESS APPLICATION MANAGEMENT DATA FROM DISTRIBUTED APPLICATIONS INSTANCES

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US13/111,956 US9223632B2 (en) 2011-05-20 2011-05-20 Cross-cloud management and troubleshooting
US13/111,956 2011-05-20

Publications (2)

Publication Number Publication Date
WO2012162171A2 true WO2012162171A2 (en) 2012-11-29
WO2012162171A3 WO2012162171A3 (en) 2013-02-21

Family

ID=47175779

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2012/038647 WO2012162171A2 (en) 2011-05-20 2012-05-18 Cross-cloud management and troubleshooting

Country Status (11)

Country Link
US (2) US9223632B2 (en)
EP (1) EP2710484B1 (en)
JP (1) JP5980914B2 (en)
KR (1) KR101916847B1 (en)
CN (1) CN103548009B (en)
AU (1) AU2012259086B2 (en)
BR (1) BR112013029716B1 (en)
CA (1) CA2835440A1 (en)
MX (2) MX347110B (en)
RU (1) RU2604519C2 (en)
WO (1) WO2012162171A2 (en)

Families Citing this family (120)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9223632B2 (en) * 2011-05-20 2015-12-29 Microsoft Technology Licensing, Llc Cross-cloud management and troubleshooting
US10353563B2 (en) * 2011-06-08 2019-07-16 Citrix Systems, Inc. Methods and system for locally generated gesture and transition graphics interaction with terminal control services
WO2013028193A1 (en) * 2011-08-25 2013-02-28 Empire Technology Development, Llc Quality of service aware captive aggregation with true datacenter testing
US9026837B2 (en) * 2011-09-09 2015-05-05 Microsoft Technology Licensing, Llc Resource aware placement of applications in clusters
US20130151688A1 (en) * 2011-12-07 2013-06-13 Alcatel-Lucent Usa Inc. Optimization mechanisms for latency reduction and elasticity improvement in geographically distributed data centers
US10503615B2 (en) * 2011-12-16 2019-12-10 Basen Corporation Spime™ host system, process, object, self-determination apparatus, and host device
US9590876B2 (en) * 2012-03-02 2017-03-07 Payoda Inc. Centralized dashboard for monitoring and controlling various application specific network components across data centers
US20130290406A1 (en) * 2012-04-26 2013-10-31 Salesforce.Com, Inc. Mechanism for providing a cloud platform for facilitating and supporting user-controlled development and management of user products
US9264289B2 (en) 2013-06-27 2016-02-16 Microsoft Technology Licensing, Llc Endpoint data centers of different tenancy sets
US9996562B2 (en) 2013-07-09 2018-06-12 Oracle International Corporation Automated database migration architecture
US10776244B2 (en) 2013-07-09 2020-09-15 Oracle International Corporation Consolidation planning services for systems migration
US9805070B2 (en) 2013-07-09 2017-10-31 Oracle International Corporation Dynamic migration script management
US9967154B2 (en) 2013-07-09 2018-05-08 Oracle International Corporation Advanced customer support services—advanced support cloud portal
US9792321B2 (en) 2013-07-09 2017-10-17 Oracle International Corporation Online database migration
US11157664B2 (en) 2013-07-09 2021-10-26 Oracle International Corporation Database modeling and analysis
US9491072B2 (en) * 2013-07-09 2016-11-08 Oracle International Corporation Cloud services load testing and analysis
US9747311B2 (en) 2013-07-09 2017-08-29 Oracle International Corporation Solution to generate a scriptset for an automated database migration
US9762461B2 (en) * 2013-07-09 2017-09-12 Oracle International Corporation Cloud services performance tuning and benchmarking
US9509759B2 (en) 2013-09-09 2016-11-29 International Business Machines Corporation Service agreement performance validation in a cloud hosted environment
EP3053052B1 (en) 2013-10-03 2019-08-14 Hewlett Packard Enterprise Development LP Managing a number of secondary clouds by a master cloud service manager
WO2015054832A1 (en) * 2013-10-16 2015-04-23 Empire Technology Development Llc Two-level cloud system migration
US9401954B2 (en) * 2013-11-06 2016-07-26 International Business Machines Corporation Scaling a trusted computing model in a globally distributed cloud environment
US9791485B2 (en) * 2014-03-10 2017-10-17 Silver Spring Networks, Inc. Determining electric grid topology via a zero crossing technique
US10157100B2 (en) * 2014-04-30 2018-12-18 Hewlett Packard Enterprise Development Lp Support action based self learning and analytics for datacenter device hardware/firmare fault management
US9811365B2 (en) * 2014-05-09 2017-11-07 Amazon Technologies, Inc. Migration of applications between an enterprise-based network and a multi-tenant network
WO2015184158A1 (en) * 2014-05-28 2015-12-03 New Media Solutions, Inc. Generation and management of computing infrastructure instances
WO2015191394A1 (en) * 2014-06-09 2015-12-17 Northrop Grumman Systems Corporation System and method for real-time detection of anomalies in database usage
US10922666B1 (en) * 2014-06-23 2021-02-16 Amazon Technologies, Inc. Resource management for logical and physical availability zones of a provider network
US9606826B2 (en) * 2014-08-21 2017-03-28 International Business Machines Corporation Selecting virtual machines to be migrated to public cloud during cloud bursting based on resource usage and scaling policies
US9680920B2 (en) * 2014-09-08 2017-06-13 International Business Machines Corporation Anticipatory resource allocation/activation and lazy de-allocation/deactivation
CN106464541B (en) * 2015-03-19 2019-09-20 华为技术有限公司 Fault handling method and equipment based on network function virtualization
US10129157B2 (en) * 2015-03-31 2018-11-13 At&T Intellectual Property I, L.P. Multiple feedback instance inter-coordination to determine optimal actions
US9524200B2 (en) 2015-03-31 2016-12-20 At&T Intellectual Property I, L.P. Consultation among feedback instances
US9992277B2 (en) 2015-03-31 2018-06-05 At&T Intellectual Property I, L.P. Ephemeral feedback instances
US10277666B2 (en) * 2015-03-31 2019-04-30 At&T Intellectual Property I, L.P. Escalation of feedback instances
US10129156B2 (en) 2015-03-31 2018-11-13 At&T Intellectual Property I, L.P. Dynamic creation and management of ephemeral coordinated feedback instances
US9769206B2 (en) 2015-03-31 2017-09-19 At&T Intellectual Property I, L.P. Modes of policy participation for feedback instances
US10728092B2 (en) 2015-05-01 2020-07-28 Microsoft Technology Licensing, Llc Cloud-mastered settings
US10581670B2 (en) 2015-10-02 2020-03-03 Microsoft Technology Licensing, Llc Cross-data center interoperation and communication
US10425386B2 (en) 2016-05-11 2019-09-24 Oracle International Corporation Policy enforcement point for a multi-tenant identity and data security management cloud service
US9838377B1 (en) 2016-05-11 2017-12-05 Oracle International Corporation Task segregation in a multi-tenant identity and data security management cloud service
US9838376B1 (en) 2016-05-11 2017-12-05 Oracle International Corporation Microservices based multi-tenant identity and data security management cloud service
US10581820B2 (en) 2016-05-11 2020-03-03 Oracle International Corporation Key generation and rollover
US10454940B2 (en) 2016-05-11 2019-10-22 Oracle International Corporation Identity cloud service authorization model
US9781122B1 (en) 2016-05-11 2017-10-03 Oracle International Corporation Multi-tenant identity and data security management cloud service
US10341410B2 (en) 2016-05-11 2019-07-02 Oracle International Corporation Security tokens for a multi-tenant identity and data security management cloud service
US10878079B2 (en) 2016-05-11 2020-12-29 Oracle International Corporation Identity cloud service authorization model with dynamic roles and scopes
US10148740B2 (en) 2016-06-03 2018-12-04 Microsoft Technology Licensing, Llc Multi-service application fabric architecture
US11036696B2 (en) 2016-06-07 2021-06-15 Oracle International Corporation Resource allocation for database provisioning
US10834226B2 (en) * 2016-07-15 2020-11-10 International Business Machines Corporation Live migration of containers based on geo-location
US10585682B2 (en) * 2016-08-05 2020-03-10 Oracle International Corporation Tenant self-service troubleshooting for a multi-tenant identity and data security management cloud service
US10735394B2 (en) 2016-08-05 2020-08-04 Oracle International Corporation Caching framework for a multi-tenant identity and data security management cloud service
US10263947B2 (en) 2016-08-05 2019-04-16 Oracle International Corporation LDAP to SCIM proxy service
US10255061B2 (en) 2016-08-05 2019-04-09 Oracle International Corporation Zero down time upgrade for a multi-tenant identity and data security management cloud service
US10530578B2 (en) 2016-08-05 2020-01-07 Oracle International Corporation Key store service
US10516672B2 (en) 2016-08-05 2019-12-24 Oracle International Corporation Service discovery for a multi-tenant identity and data security management cloud service
US10721237B2 (en) 2016-08-05 2020-07-21 Oracle International Corporation Hierarchical processing for a virtual directory system for LDAP to SCIM proxy service
US10484382B2 (en) 2016-08-31 2019-11-19 Oracle International Corporation Data management for a multi-tenant identity cloud service
US10594684B2 (en) 2016-09-14 2020-03-17 Oracle International Corporation Generating derived credentials for a multi-tenant identity cloud service
US10511589B2 (en) 2016-09-14 2019-12-17 Oracle International Corporation Single logout functionality for a multi-tenant identity and data security management cloud service
US10846390B2 (en) 2016-09-14 2020-11-24 Oracle International Corporation Single sign-on functionality for a multi-tenant identity and data security management cloud service
US11973758B2 (en) * 2016-09-14 2024-04-30 Microsoft Technology Licensing, Llc Self-serve appliances for cloud services platform
EP3513542B1 (en) 2016-09-16 2021-05-19 Oracle International Corporation Tenant and service management for a multi-tenant identity and data security management cloud service
US10791087B2 (en) 2016-09-16 2020-09-29 Oracle International Corporation SCIM to LDAP mapping using subtype attributes
US10341354B2 (en) 2016-09-16 2019-07-02 Oracle International Corporation Distributed high availability agent architecture
US10567364B2 (en) 2016-09-16 2020-02-18 Oracle International Corporation Preserving LDAP hierarchy in a SCIM directory using special marker groups
US10484243B2 (en) 2016-09-16 2019-11-19 Oracle International Corporation Application management for a multi-tenant identity cloud service
US10445395B2 (en) 2016-09-16 2019-10-15 Oracle International Corporation Cookie based state propagation for a multi-tenant identity cloud service
US10904074B2 (en) 2016-09-17 2021-01-26 Oracle International Corporation Composite event handler for a multi-tenant identity cloud service
JP2018060332A (en) * 2016-10-04 2018-04-12 富士通株式会社 Incident analysis program, incident analysis method, information processing device, service specification program, service specification method and service specification device
US10261836B2 (en) 2017-03-21 2019-04-16 Oracle International Corporation Dynamic dispatching of workloads spanning heterogeneous services
US10454915B2 (en) 2017-05-18 2019-10-22 Oracle International Corporation User authentication using kerberos with identity cloud service
US10635433B2 (en) * 2017-08-24 2020-04-28 General Electric Company Cross application behavior customization
US10348858B2 (en) 2017-09-15 2019-07-09 Oracle International Corporation Dynamic message queues for a microservice based cloud service
US10831789B2 (en) 2017-09-27 2020-11-10 Oracle International Corporation Reference attribute query processing for a multi-tenant cloud service
US11271969B2 (en) 2017-09-28 2022-03-08 Oracle International Corporation Rest-based declarative policy management
US10834137B2 (en) 2017-09-28 2020-11-10 Oracle International Corporation Rest-based declarative policy management
US10705823B2 (en) 2017-09-29 2020-07-07 Oracle International Corporation Application templates and upgrade framework for a multi-tenant identity cloud service
US10447536B2 (en) 2017-10-20 2019-10-15 Vmware, Inc. Managing cross-cloud distributed application
US10481970B2 (en) 2017-11-28 2019-11-19 Bank Of America Corporation Dynamic cloud deployment and calibration tool
US10609131B2 (en) 2018-01-12 2020-03-31 Citrix Systems, Inc. Non-disruptive enablement of highly available cloud computing services
US10715564B2 (en) 2018-01-29 2020-07-14 Oracle International Corporation Dynamic client registration for an identity cloud service
US10931656B2 (en) 2018-03-27 2021-02-23 Oracle International Corporation Cross-region trust for a multi-tenant identity cloud service
US11165634B2 (en) 2018-04-02 2021-11-02 Oracle International Corporation Data replication conflict detection and resolution for a multi-tenant identity cloud service
US10798165B2 (en) 2018-04-02 2020-10-06 Oracle International Corporation Tenant data comparison for a multi-tenant identity cloud service
US11258775B2 (en) 2018-04-04 2022-02-22 Oracle International Corporation Local write for a multi-tenant identity cloud service
US11012444B2 (en) 2018-06-25 2021-05-18 Oracle International Corporation Declarative third party identity provider integration for a multi-tenant identity cloud service
US10764273B2 (en) 2018-06-28 2020-09-01 Oracle International Corporation Session synchronization across multiple devices in an identity cloud service
US10846070B2 (en) 2018-07-05 2020-11-24 At&T Intellectual Property I, L.P. Facilitating cloud native edge computing via behavioral intelligence
US11070613B2 (en) * 2018-08-16 2021-07-20 Microsoft Technology Licensing, Llc Automatic application scaling between private and public cloud platforms
US20200065149A1 (en) 2018-08-24 2020-02-27 Embotics Corporation Method and system for selection of cloud-computing services
CN109240837B (en) * 2018-09-11 2020-09-29 四川虹微技术有限公司 Construction method of universal cloud storage service API
US11693835B2 (en) 2018-10-17 2023-07-04 Oracle International Corporation Dynamic database schema allocation on tenant onboarding for a multi-tenant identity cloud service
US11321187B2 (en) 2018-10-19 2022-05-03 Oracle International Corporation Assured lazy rollback for a multi-tenant identity cloud service
DE102018009906A1 (en) 2018-12-20 2020-06-25 Volkswagen Aktiengesellschaft Process for the management of computer capacities in a network with mobile participants
US11113385B2 (en) * 2018-12-21 2021-09-07 Paypal, Inc. Communicating trace information between security zones
US10776158B2 (en) * 2019-01-31 2020-09-15 Lockheed Martin Corporation Management of application deployment across multiple provisioning layers
US11651357B2 (en) 2019-02-01 2023-05-16 Oracle International Corporation Multifactor authentication without a user footprint
US11061929B2 (en) 2019-02-08 2021-07-13 Oracle International Corporation Replication of resource type and schema metadata for a multi-tenant identity cloud service
US11321343B2 (en) 2019-02-19 2022-05-03 Oracle International Corporation Tenant replication bootstrap for a multi-tenant identity cloud service
US11669321B2 (en) 2019-02-20 2023-06-06 Oracle International Corporation Automated database upgrade for a multi-tenant identity cloud service
US11423111B2 (en) 2019-02-25 2022-08-23 Oracle International Corporation Client API for rest based endpoints for a multi-tenant identify cloud service
US11792226B2 (en) 2019-02-25 2023-10-17 Oracle International Corporation Automatic api document generation from scim metadata
US11010228B2 (en) * 2019-03-01 2021-05-18 International Business Machines Corporation Apparatus, systems, and methods for identifying distributed objects subject to service
JP7193732B2 (en) * 2019-04-08 2022-12-21 富士通株式会社 Management device, information processing system and management program
US11528194B2 (en) * 2019-09-06 2022-12-13 Jpmorgan Chase Bank, N.A. Enterprise control plane for data streaming service
US11687378B2 (en) 2019-09-13 2023-06-27 Oracle International Corporation Multi-tenant identity cloud service with on-premise authentication integration and bridge high availability
US11256671B2 (en) 2019-09-13 2022-02-22 Oracle International Corporation Integrated transition control center
US11870770B2 (en) 2019-09-13 2024-01-09 Oracle International Corporation Multi-tenant identity cloud service with on-premise authentication integration
US11611548B2 (en) 2019-11-22 2023-03-21 Oracle International Corporation Bulk multifactor authentication enrollment
US11868622B2 (en) 2020-02-25 2024-01-09 Pure Storage, Inc. Application recovery across storage systems
US11637896B1 (en) 2020-02-25 2023-04-25 Pure Storage, Inc. Migrating applications to a cloud-computing environment
US11283671B2 (en) 2020-02-28 2022-03-22 Hewlett Packard Enterprise Development Lp Handling issues reported by network devices
US20210382798A1 (en) * 2020-06-04 2021-12-09 Servicenow, Inc. Optimizing configuration of cloud instances
US11968210B2 (en) * 2021-05-19 2024-04-23 International Business Machines Corporation Management of access control in multi-cloud environments
US11368539B1 (en) 2021-05-27 2022-06-21 International Business Machines Corporation Application deployment in a multi-cluster environment
US20230128370A1 (en) * 2021-10-21 2023-04-27 EMC IP Holding Company LLC Data Center Restoration and Migration
US11979456B2 (en) * 2022-01-14 2024-05-07 Dell Products, L.P. Cloud provisioning readiness verification
US11973843B2 (en) 2022-06-22 2024-04-30 Cisco Technology, Inc. On demand end user monitoring for automated help desk support
CN116795821A (en) * 2023-07-11 2023-09-22 新疆联佳网络有限公司 Method, system, platform and data management method for constructing metadata management system

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110055377A1 (en) 2009-08-31 2011-03-03 Dehaan Michael Paul Methods and systems for automated migration of cloud processes to external clouds
US20110078303A1 (en) 2009-09-30 2011-03-31 Alcatel-Lucent Usa Inc. Dynamic load balancing and scaling of allocated cloud resources in an enterprise network

Family Cites Families (56)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6862736B2 (en) * 1998-02-06 2005-03-01 Microsoft Corporation Object manager for common information model
US20020120741A1 (en) * 2000-03-03 2002-08-29 Webb Theodore S. Systems and methods for using distributed interconnects in information management enviroments
US6901530B2 (en) * 2000-08-01 2005-05-31 Qwest Communications International, Inc. Proactive repair process in the xDSL network (with a VDSL focus)
CA2319918A1 (en) * 2000-09-18 2002-03-18 Linmor Technologies Inc. High performance relational database management system
JP4542253B2 (en) * 2000-11-06 2010-09-08 株式会社日本コンラックス Promotion system
US7600014B2 (en) * 2000-11-16 2009-10-06 Symantec Corporation Method and system for monitoring the performance of a distributed application
KR100346185B1 (en) * 2000-12-01 2002-07-26 삼성전자 주식회사 System and method for managing alarm in network management system
US20020103886A1 (en) * 2000-12-04 2002-08-01 International Business Machines Corporation Non-local aggregation of system management data
US7337473B2 (en) * 2000-12-15 2008-02-26 International Business Machines Corporation Method and system for network management with adaptive monitoring and discovery of computer systems based on user login
US7305461B2 (en) * 2000-12-15 2007-12-04 International Business Machines Corporation Method and system for network management with backup status gathering
US7430594B2 (en) * 2001-01-26 2008-09-30 Computer Associates Think, Inc. Method and apparatus for distributed systems management
JP2002300308A (en) * 2001-03-30 2002-10-11 Ricoh Co Ltd Customer support system, office system, customer support center, supply center and customer support method
US7010593B2 (en) * 2001-04-30 2006-03-07 Hewlett-Packard Development Company, L.P. Dynamic generation of context-sensitive data and instructions for troubleshooting problem events in a computing environment
US20020161876A1 (en) * 2001-04-30 2002-10-31 Robert Raymond System and method for managing data miner modules in an information network system
JP2002366454A (en) * 2001-06-11 2002-12-20 Fujitsu Ltd Network managing method and its device
JP2003101586A (en) * 2001-09-25 2003-04-04 Hitachi Ltd Network management support method
US20050262229A1 (en) * 2004-04-16 2005-11-24 Samsung Electronics Co., Ltd. Object conduit MIB for communicating over SNMP between distributed objects
US20060031446A1 (en) * 2004-06-17 2006-02-09 Mohamed Hamedi Gathering network management data using a command line function
WO2006009210A1 (en) * 2004-07-21 2006-01-26 Sony Corporation Content processing device, content processing method, and computer program
JP2006344025A (en) * 2005-06-09 2006-12-21 Hitachi Ltd Operation performance data acquisition method, performance monitor server, work server, computer, and computing system
US7356590B2 (en) * 2005-07-12 2008-04-08 Visible Measures Corp. Distributed capture and aggregation of dynamic application usage information
US20070064714A1 (en) * 2005-09-16 2007-03-22 Sbc Knowledge Ventures, L.P. Wireless based troubleshooting of customer premise equipment installation
JP4738144B2 (en) * 2005-11-28 2011-08-03 株式会社日立製作所 Information monitoring method, system and program
US20070198554A1 (en) * 2006-02-10 2007-08-23 Sun Microsystems, Inc. Apparatus for business service oriented management infrastructure
US8190682B2 (en) * 2006-03-31 2012-05-29 Amazon Technologies, Inc. Managing execution of programs by multiple computing systems
US7676475B2 (en) * 2006-06-22 2010-03-09 Sun Microsystems, Inc. System and method for efficient meta-data driven instrumentation
US7814114B2 (en) * 2006-09-05 2010-10-12 Oracle International Corporation Tree-based information query model
JP2008217735A (en) * 2007-03-08 2008-09-18 Nec Corp Fault analysis system, method and program
US10127059B2 (en) * 2008-05-02 2018-11-13 Skytap Multitenant hosted virtual machine infrastructure
US7886038B2 (en) * 2008-05-27 2011-02-08 Red Hat, Inc. Methods and systems for user identity management in cloud-based networks
US20090300423A1 (en) * 2008-05-28 2009-12-03 James Michael Ferris Systems and methods for software test management in cloud-based network
US8239509B2 (en) 2008-05-28 2012-08-07 Red Hat, Inc. Systems and methods for management of virtual appliances in cloud-based network
US8250215B2 (en) 2008-08-12 2012-08-21 Sap Ag Method and system for intelligently leveraging cloud computing resources
US7894334B2 (en) * 2008-08-15 2011-02-22 Telefonaktiebolaget L M Ericsson Hierarchical redundancy for a distributed control plane
US8271974B2 (en) 2008-10-08 2012-09-18 Kaavo Inc. Cloud computing lifecycle management for N-tier applications
US9037692B2 (en) 2008-11-26 2015-05-19 Red Hat, Inc. Multiple cloud marketplace aggregation
US7996525B2 (en) * 2008-12-31 2011-08-09 Sap Ag Systems and methods for dynamically provisioning cloud computing resources
US8977750B2 (en) * 2009-02-24 2015-03-10 Red Hat, Inc. Extending security platforms to cloud-based networks
US20100220622A1 (en) 2009-02-27 2010-09-02 Yottaa Inc Adaptive network with automatic scaling
US8751627B2 (en) * 2009-05-05 2014-06-10 Accenture Global Services Limited Method and system for application migration in a cloud
US8290998B2 (en) * 2009-05-20 2012-10-16 Sap Ag Systems and methods for generating cloud computing landscapes
US9104407B2 (en) * 2009-05-28 2015-08-11 Red Hat, Inc. Flexible cloud management with power management support
US20100306767A1 (en) * 2009-05-29 2010-12-02 Dehaan Michael Paul Methods and systems for automated scaling of cloud computing systems
US9325802B2 (en) * 2009-07-16 2016-04-26 Microsoft Technology Licensing, Llc Hierarchical scale unit values for storing instances of data among nodes of a distributed store
US9329951B2 (en) * 2009-07-31 2016-05-03 Paypal, Inc. System and method to uniformly manage operational life cycles and service levels
US9558441B2 (en) * 2009-08-28 2017-01-31 Pneuron Corp. Legacy application migration to real time, parallel performance cloud
US8769083B2 (en) * 2009-08-31 2014-07-01 Red Hat, Inc. Metering software infrastructure in a cloud computing environment
JP5471198B2 (en) * 2009-09-03 2014-04-16 株式会社リコー Integrated management apparatus, integrated management system, integrated management method, integrated management program, and recording medium recording the program
JP2011090429A (en) * 2009-10-21 2011-05-06 Nomura Research Institute Ltd Integrated monitoring system
US10402544B2 (en) * 2009-11-30 2019-09-03 Red Hat, Inc. Generating a software license knowledge base for verifying software license compliance in cloud computing environments
US8745397B2 (en) * 2010-01-04 2014-06-03 Microsoft Corporation Monitoring federation for cloud based services and applications
US8346935B2 (en) * 2010-01-15 2013-01-01 Joyent, Inc. Managing hardware resources by sending messages amongst servers in a data center
US20120011077A1 (en) * 2010-07-12 2012-01-12 Bhagat Bhavesh C Cloud Computing Governance, Cyber Security, Risk, and Compliance Business Rules System and Method
US8769534B2 (en) * 2010-09-23 2014-07-01 Accenture Global Services Limited Measuring CPU utilization in a cloud computing infrastructure by artificially executing a bursting application on a virtual machine
US8977754B2 (en) * 2011-05-09 2015-03-10 Metacloud Inc. Composite public cloud, method and system
US9223632B2 (en) * 2011-05-20 2015-12-29 Microsoft Technology Licensing, Llc Cross-cloud management and troubleshooting

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110055377A1 (en) 2009-08-31 2011-03-03 Dehaan Michael Paul Methods and systems for automated migration of cloud processes to external clouds
US20110078303A1 (en) 2009-09-30 2011-03-31 Alcatel-Lucent Usa Inc. Dynamic load balancing and scaling of allocated cloud resources in an enterprise network

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP2710484A4

Also Published As

Publication number Publication date
MX347110B (en) 2017-04-12
BR112013029716B1 (en) 2021-08-17
KR101916847B1 (en) 2019-01-24
EP2710484A4 (en) 2016-04-06
AU2012259086B2 (en) 2016-09-22
RU2013151607A (en) 2015-05-27
US20160119202A1 (en) 2016-04-28
KR20140026503A (en) 2014-03-05
US20120297016A1 (en) 2012-11-22
MX2013013577A (en) 2014-07-30
CA2835440A1 (en) 2012-11-29
MX366620B (en) 2019-07-16
JP2014515522A (en) 2014-06-30
EP2710484A2 (en) 2014-03-26
EP2710484B1 (en) 2020-02-26
US10009238B2 (en) 2018-06-26
BR112013029716A2 (en) 2017-01-24
CN103548009B (en) 2017-02-08
US9223632B2 (en) 2015-12-29
CN103548009A (en) 2014-01-29
WO2012162171A3 (en) 2013-02-21
RU2604519C2 (en) 2016-12-10
JP5980914B2 (en) 2016-08-31

Similar Documents

Publication Publication Date Title
US10009238B2 (en) Cross-cloud management and troubleshooting
AU2012259086A1 (en) Cross-cloud management and troubleshooting
US8719627B2 (en) Cross-cloud computing for capacity management and disaster recovery
US11249815B2 (en) Maintaining two-site configuration for workload availability between sites at unlimited distances for products and services
US10044550B2 (en) Secure cloud management agent
US10084858B2 (en) Managing continuous priority workload availability and general workload availability between sites at unlimited distances for products and services
US10303678B2 (en) Application resiliency management using a database driver

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 12789586

Country of ref document: EP

Kind code of ref document: A2

ENP Entry into the national phase

Ref document number: 2835440

Country of ref document: CA

WWE Wipo information: entry into national phase

Ref document number: 2012789586

Country of ref document: EP

ENP Entry into the national phase

Ref document number: 2013151607

Country of ref document: RU

Kind code of ref document: A

Ref document number: 20137030613

Country of ref document: KR

Kind code of ref document: A

ENP Entry into the national phase

Ref document number: 2014512899

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

WWE Wipo information: entry into national phase

Ref document number: MX/A/2013/013577

Country of ref document: MX

ENP Entry into the national phase

Ref document number: 2012259086

Country of ref document: AU

Date of ref document: 20120518

Kind code of ref document: A

REG Reference to national code

Ref country code: BR

Ref legal event code: B01A

Ref document number: 112013029716

Country of ref document: BR

ENP Entry into the national phase

Ref document number: 112013029716

Country of ref document: BR

Kind code of ref document: A2

Effective date: 20131119