WO2023136755A1 - Method and apparatus for tailored data monitoring of microservice executions in mobile edge clouds - Google Patents

Method and apparatus for tailored data monitoring of microservice executions in mobile edge clouds Download PDF

Info

Publication number
WO2023136755A1
WO2023136755A1 PCT/SE2022/050023 SE2022050023W WO2023136755A1 WO 2023136755 A1 WO2023136755 A1 WO 2023136755A1 SE 2022050023 W SE2022050023 W SE 2022050023W WO 2023136755 A1 WO2023136755 A1 WO 2023136755A1
Authority
WO
WIPO (PCT)
Prior art keywords
microservice
data
monitoring data
critical
monitoring
Prior art date
Application number
PCT/SE2022/050023
Other languages
French (fr)
Inventor
Nanjangud Chandrasekhara Swamy NARENDRA
Chunyan Fu
Original Assignee
Telefonaktiebolaget Lm Ericsson (Publ)
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Telefonaktiebolaget Lm Ericsson (Publ) filed Critical Telefonaktiebolaget Lm Ericsson (Publ)
Priority to PCT/SE2022/050023 priority Critical patent/WO2023136755A1/en
Publication of WO2023136755A1 publication Critical patent/WO2023136755A1/en

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/48Program initiating; Program switching, e.g. by interrupt
    • G06F9/4806Task transfer initiation or dispatching
    • G06F9/4843Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
    • G06F9/485Task life-cycle, e.g. stopping, restarting, resuming execution
    • G06F9/4856Task life-cycle, e.g. stopping, restarting, resuming execution resumption being on a different machine, e.g. task migration, virtual machine migration
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/12Discovery or management of network topologies
    • H04L41/122Discovery or management of network topologies of virtualised topologies, e.g. software-defined networks [SDN] or network function virtualisation [NFV]
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/14Network analysis or design
    • H04L41/145Network analysis or design involving simulating, designing, planning or modelling of a network
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/50Network services
    • H04L67/51Discovery or management thereof, e.g. service location protocol [SLP] or web services
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F2209/00Indexing scheme relating to G06F9/00
    • G06F2209/50Indexing scheme relating to G06F9/50
    • G06F2209/508Monitor
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/06Management of faults, events, alarms or notifications
    • H04L41/0631Management of faults, events, alarms or notifications using root cause analysis; using analysis of correlation between notifications, alarms or events based on decision criteria, e.g. hierarchy, tree or time analysis
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/08Configuration management of networks or network elements
    • H04L41/0803Configuration setting
    • H04L41/0806Configuration setting for initial configuration or provisioning, e.g. plug-and-play
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/08Configuration management of networks or network elements
    • H04L41/0894Policy-based network configuration management
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/08Configuration management of networks or network elements
    • H04L41/0895Configuration of virtualised networks or elements, e.g. virtualised network function or OpenFlow elements
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/08Configuration management of networks or network elements
    • H04L41/0896Bandwidth or capacity management, i.e. automatically increasing or decreasing capacities
    • H04L41/0897Bandwidth or capacity management, i.e. automatically increasing or decreasing capacities by horizontal or vertical scaling of resources, or by migrating entities, e.g. virtual resources or entities
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00Arrangements for monitoring or testing data switching networks
    • H04L43/08Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00Arrangements for monitoring or testing data switching networks
    • H04L43/20Arrangements for monitoring or testing data switching networks the monitoring system or the monitored elements being virtualised, abstracted or software-defined entities, e.g. SDN or NFV

Definitions

  • Embodiments disclosed herein relate to the field of distributed mobile edge clouds, and more specifically to methods and apparatus for tailored monitoring of microservice chain execution in a distributed mobile edge cloud.
  • microservices are also migrated across edge sites in order to ensure uninterrupted access to end users, who may also be moving around.
  • Such a volatile and dynamic infrastructure requires a highly distributed and efficient mechanism of monitoring application execution to enable service assurance.
  • data monitoring that is, monitoring of the application data that is flowing through the system and which is being used by microservices during their execution. Such monitoring may help determine whether the application is performing as designed.
  • Data handling is the most important aspect of microservice chain execution in distributed mobile edge clouds.
  • domains such as video streaming, transportation/logistics, vehicle to everything (V2X), and gaming
  • data migration may become a major stumbling block leading to user dissatisfaction if not handled properly.
  • monitoring and governance of the data migration to identify problems and issues, if any, and correct them, is of utmost importance.
  • monitoring data also referred to as “monitoring data,” and also sometimes referred to, for ease of explanation, as “metrics” or “metrics data”
  • metrics data metrics data
  • An example embodiment is a method performed by one or more network devices to provide tailored monitoring of microservice chain execution in a distributed mobile edge cloud.
  • the method includes determining a microservice topology of a microservice chain, determining which microservices in the microservice chain are critical microservices based on the microservice topology, determining which types of monitoring data to collect for each of one or more microservices in the microservice chain based on whether the microservice was determined to be a critical microservice, generating an integrated microservice chain model for the microservice chain that comprises an indication of the microservice topology, an indication of the critical microservices, and an indication of the types of monitoring data to collect for each of the one or more microservices, and providing the integrated microservice chain model to a microservice tracker, wherein the microservice tracker is configured to collect, from the distributed mobile edge cloud while the microservice chain is being executed in the distributed mobile edge cloud, monitoring data for each of the one or more microservices corresponding to the types of monitoring data to collect for the microservice as indicated in the integrated microservice
  • An example embodiment is a set of non-transitory machine-readable media having computer code stored therein, which when executed by a set of one or more processors of one or more network devices, causes the one or more network devices to perform operations for providing tailored monitoring of microservice chain execution in a distributed mobile edge cloud.
  • the operations include determining a microservice topology of a microservice chain, determining which microservices in the microservice chain are critical microservices based on the microservice topology, determining which types of monitoring data to collect for each of one or more microservices in the microservice chain based on whether the microservice was determined to be a critical microservice, generating an integrated microservice chain model for the microservice chain that comprises an indication of the microservice topology, an indication of the critical microservices, and an indication of the types of monitoring data to collect for each of the one or more microservices, and providing the integrated microservice chain model to a microservice tracker, wherein the microservice tracker is configured to collect, from the distributed mobile edge cloud while the microservice chain is being executed in the distributed mobile edge cloud, monitoring data for each of the one or more microservices corresponding to the types of monitoring data to collect for the microservice as indicated in the integrated microservice chain model and provide the collected monitoring data to a monitoring data analyzer that is to analyze the collected monitoring data for the one or more microservices, together with
  • An example embodiment is a method performed by one or more network devices to provide tailored monitoring of microservice chain execution in a distributed mobile edge cloud.
  • the method includes obtaining an integrated microservice chain model for a microservice chain, wherein the integrated microservice chain model comprises an indication of a microservice topology of the microservice chain, an indication of which microservices in the microservice chain are critical microservices, and an indication of which types of monitoring data to collect for each of one or more microservices in the microservice chain, collecting, from the distributed mobile edge cloud while the microservice chain is being executed in in the distributed mobile edge cloud, monitoring data for each of the one or more microservices corresponding to the types of monitoring data to collect for the microservice as indicated in the integrated microservice chain model, and providing the collected monitoring data for the one or more microservices to a monitoring data analyzer, wherein the monitoring data analyzer is to analyze the collected monitoring data for the one or more microservices, together with UE monitoring data and network monitoring data, to generate an analysis result.
  • An example embodiment is a set of non-transitory machine-readable media having computer code stored therein, which when executed by a set of one or more processors of one or more network devices, causes the one or more network devices to perform operations for providing tailored monitoring of microservice chain execution in a distributed mobile edge cloud.
  • the operations include obtaining an integrated microservice chain model for a microservice chain, wherein the integrated microservice chain model comprises an indication of a microservice topology of the microservice chain, an indication of which microservices in the microservice chain are critical microservices, and an indication of which types of monitoring data to collect for each of one or more microservices in the microservice chain, collecting, from the distributed mobile edge cloud while the microservice chain is being executed in the distributed mobile edge cloud, monitoring data for each of the one or more microservices corresponding to the types of monitoring data to collect for the microservice as indicated in the integrated microservice chain model, and providing the collected monitoring data for the one or more microservices to a monitoring data analyzer, wherein the monitoring data analyzer is to analyze the collected monitoring data for the one or more microservices, together with UE monitoring data and network monitoring data, to generate an analysis result.
  • the integrated microservice chain model comprises an indication of a microservice topology of the microservice chain, an indication of which microservices in the microservice chain are critical microservices, and an indication of which
  • An advantage of embodiments disclosed herein is that they can monitor microservice chain execution in a tailored manner that allows for gaining valuable insights into the operation of the microservice chain execution while reducing the amount of monitoring data that is collected. Thereby, the embodiments herein help reduce the burden on the underlying infrastructure that has to collect the data and reduce the amount of data that has to be analyzed, thereby resulting in more efficient monitoring and analysis.
  • Figure 1 is a diagram showing an environment in which tailored monitoring of microservice chain execution may be performed, according to some embodiments.
  • Figure 2 is a diagram showing component interactions for providing tailored monitoring of microservice chain execution in a distributed mobile edge cloud, according to some embodiments.
  • Figure 3 is a flow diagram showing a method for tailored monitoring of microservice execution in a distributed mobile edge cloud, according to some embodiments.
  • Figure 4 is a flow diagram showing a method for determining data item criticality, according to some embodiments.
  • Figure 5 is a diagram showing a microservice topology for a movie delivery service microservice chain, according to some embodiments.
  • Figure 6 is a diagram showing a microservice topology for a drone swarm microservice chain, according to some embodiments.
  • Figure 7 is a flow diagram showing a method for generating an integrated microservice chain model, according to some embodiments.
  • Figure 8 is a flow diagram showing a method for collecting monitoring data in a tailored manner, according to some embodiments.
  • Figure 9 is a flow diagram showing a method for analyzing monitoring data, according to some embodiments.
  • Figure 10A illustrates connectivity between network devices within an exemplary network, as well as three exemplary implementations of the network devices, according to some embodiments.
  • Figure 10B illustrates an exemplary way to implement a special-purpose network device, according to some embodiments.
  • references in the specification to “one embodiment,” “an embodiment,” “an example embodiment,” etc., indicate that the embodiment described may include a particular feature, structure, or characteristic, but every embodiment may not necessarily include the particular feature, structure, or characteristic. Moreover, such phrases are not necessarily referring to the same embodiment. Further, when a particular feature, structure, or characteristic is described in connection with an embodiment, it is submitted that it is within the knowledge of one skilled in the art to affect such feature, structure, or characteristic in connection with other embodiments whether or not explicitly described.
  • Bracketed text and blocks with dashed borders may be used herein to illustrate optional operations that add additional features to embodiments. However, such notation should not be taken to mean that these are the only options or optional operations, and/or that blocks with solid borders are not optional in certain embodiments.
  • Coupled is used to indicate that two or more elements, which may or may not be in direct physical or electrical contact with each other, co-operate or interact with each other.
  • Connected is used to indicate the establishment of communication between two or more elements that are coupled with each other.
  • An electronic device stores and transmits (internally and/or with other electronic devices over a network) code (which is composed of software instructions and which is sometimes referred to as computer program code or a computer program) and/or data using machine-readable media (also called computer-readable media), such as machine-readable storage media (e.g., magnetic disks, optical disks, solid state drives, read only memory (ROM), flash memory devices, phase change memory) and machine-readable transmission media (also called a carrier) (e.g., electrical, optical, radio, acoustical or other form of propagated signals - such as carrier waves, infrared signals).
  • machine-readable media also called computer-readable media
  • machine-readable storage media e.g., magnetic disks, optical disks, solid state drives, read only memory (ROM), flash memory devices, phase change memory
  • machine-readable transmission media also called a carrier
  • carrier e.g., electrical, optical, radio, acoustical or other form of propagated signals - such as carrier waves, inf
  • an electronic device e.g., a computer
  • hardware and software such as a set of one or more processors (e.g., wherein a processor is a microprocessor, controller, microcontroller, central processing unit, digital signal processor, application-specific integrated circuit, field-programmable gate array, other electronic circuitry, a combination of one or more of the preceding) coupled to one or more machine-readable storage media to store code for execution on the set of processors and/or to store data.
  • processors e.g., wherein a processor is a microprocessor, controller, microcontroller, central processing unit, digital signal processor, application-specific integrated circuit, field-programmable gate array, other electronic circuitry, a combination of one or more of the preceding
  • an electronic device may include nonvolatile memory containing the code since the non-volatile memory can persist code/data even when the electronic device is turned off (when power is removed), and while the electronic device is turned on that part of the code that is to be executed by the processor(s) of that electronic device is typically copied from the slower non-volatile memory into volatile memory (e.g., dynamic random access memory (DRAM), static random access memory (SRAM)) of that electronic device.
  • Typical electronic devices also include a set of one or more physical network interface(s) (NI(s)) to establish network connections (to transmit and/or receive code and/or data using propagating signals) with other electronic devices.
  • NI(s) physical network interface
  • a physical NI may comprise radio circuitry capable of receiving data from other electronic devices over a wireless connection and/or sending data out to other devices via a wireless connection.
  • This radio circuitry may include transmitter(s), receiver(s), and/or trans DC(s) suitable for radiofrequency communication.
  • the radio circuitry may convert digital data into a radio signal with appropriate parameters (e.g., frequency, timing, channel, bandwidth, etc.). The radio signal may then be transmitted via antennas to the appropriate recipient(s).
  • the set of physical NI(s) may comprise network interface controller(s) (NICs), also known as a network interface card, network adapter, or local area network (LAN) adapter.
  • NICs network interface controller
  • the NIC(s) may facilitate in connecting the electronic device to other electronic devices allowing them to communicate via wire through plugging in a cable to a physical port connected to a NIC.
  • One or more parts of an embodiment may be implemented using different combinations of software, firmware, and/or hardware.
  • a network device is an electronic device that communicatively interconnects other electronic devices on the network (e.g., other network devices, end-user devices).
  • Some network devices are “multiple services network devices” that provide support for multiple networking functions (e.g., routing, bridging, switching, Layer 2 aggregation, session border control, Quality of Service, and/or subscriber management), and/or provide support for multiple application services (e.g., data, voice, and video).
  • microservice chain for a movie delivery service comprising the following microservices: movie selection, movie encoding, movie streaming, view reviews, and enter review.
  • the most critical microservices in this microservice chain are movie selection, movie encoding, and movie streaming. As such, it is important that these microservices be specifically targeted for closer monitoring, especially since movie streaming is a highly data-intensive microservice and can have a significant impact on the end-user experience.
  • the monitoring data collected for such monitoring purposes would provide useful information regarding how well (or not) the microservice is performing, and could help identify where something may be going wrong. Identifying the critical microservices and focusing the monitoring primarily on them (and collecting less monitoring data for the non-critical microservices) may help reduce the overall amount of monitoring data collected, which may help reduce the burden on the distributed mobile edge cloud operators whose task is to analyze the collected monitoring data and draw inferences from it.
  • monitoring data may provide a good indication of where the movie stream was interrupted or was buffering, thereby causing delays to the viewer.
  • monitoring data indicative of the underlying network conditions that monitor network congestion, if any
  • monitoring data indicative of user equipment (UE) mobility between edge clouds the UE may move between the edge clouds, possibly triggering migration of the movie streaming microservice, resulting in possible interruption of service
  • monitoring data indicative of faults/failures/maintenance of the underlying infrastructure which may have caused a temporary interruption of service.
  • the aforementioned monitoring data may provide the operator with a clearer picture of how efficiently or inefficiently (or not) the microservice chain is performing.
  • the monitoring data together with the system helps determine the reasons that caused quality degradation. For example, the difference between the actual length of time that the movie streamed for and the one- hour movie length may be correlated against the other collected monitoring data to arrive at a determination of what may have gone wrong and where.
  • Such an approach may be implemented offline (e.g., based on analyzing logs) or online (e.g., by an alerting system that generates alerts when it detects that something may be going wrong). Offline analysis may be particularly useful when there is a need to analyze the overall system as a whole (e.g., analyze multiple instances of the movie delivery service microservice chain).
  • edge sites are typically more resource-constrained. That is, they typically have limited network, compute, and storage capacities. Hence there is an even greater need to reduce the amount of monitoring data collected for microservices executing in a distributed mobile edge cloud to ensure that the amount of collected monitoring data does not overwhelm the edge sites.
  • edge-based microservices are mobile (they can be migrated between edge sites - they are designed to be “edge-native” to allow for easy migration between edge sites). As such, the monitoring infrastructure applicable to traditional cloud-based applications may not apply to edge-based microservices.
  • Embodiments disclosed herein provide tailored monitoring of microservice chain execution in a distributed mobile edge cloud.
  • An embodiment is a method performed by one or more network devices to provide tailored monitoring of microservice chain execution in a distributed mobile edge cloud. The method includes determining a microservice topology of a microservice chain, determining which microservices in the microservice chain are critical microservices based on the microservice topology, determining which types of monitoring data to collect for each of one or more microservices in the microservice chain based on whether the microservice was determined to be a critical microservice, generating an integrated microservice chain model for the microservice chain that comprises an indication of the microservice topology, an indication of the critical microservices, and an indication of the types of monitoring data to collect for each of the one or more microservices, and providing the integrated microservice chain model to a microservice tracker, wherein the microservice tracker is configured to collect, from the distributed mobile edge cloud while the microservice chain is being executed in the distributed mobile edge cloud, monitoring data for each of the one or more microservice
  • FIG. 1 is a diagram showing an environment in which tailored monitoring of microservice chain execution may be performed, according to some embodiments.
  • the environment includes an operator 120, a tailored monitoring system 100, and a mobile network 110.
  • the mobile network 110 may be a communication network where the link to and from the end nodes (e.g., UEs) is wireless.
  • the mobile network 110 may be a 5G mobile network or similar network.
  • the mobile network 110 may implement a type of mobile edge computing (also referred to as multi-access edge computing) that provides execution resources (e.g., compute and storage resources) for applications with networking close to the end users.
  • execution resources e.g., compute and storage resources
  • the mobile network 110 may include a distributed mobile edge cloud (comprising multiple edge clouds distributed across multiple geographically dispersed edge sites), which may execute microservice chains.
  • the mobile network 110 may facilitate communication between UEs and microservice chains being executed in the distributed mobile edge cloud. While a mobile network 110 is shown in the diagram to help illustrate an embodiment, embodiments may also work with other types of (non-mobile) networks that are able to execute microservice chains across multiple edge sites.
  • the tailored monitoring system 100 includes a microservice chain modeler 130, a microservice tracker 140, a UE tracker 160, a network tracker 170, and a monitoring data analyzer 180.
  • the tailored monitoring system 100 also includes a microservice instance manager 150.
  • the components of the tailored monitoring system 100 may be implemented by one or more network devices.
  • the diagram shows a single tailored monitoring system 100 (single monolithic architecture), this is only for ease of explanation. It should be understood that there can be multiple installations of the tailored monitoring system 100 (e.g., that are geographically distributed), through which monitoring data can be collected and provided to the operator 120.
  • the operator 120 may be responsible for operating and managing the mobile network 110.
  • the operator 120 may be implemented by one or more network devices that are being controlled by a human user (e.g., a human user responsible for managing the mobile network 110 or aspects thereol).
  • the operator 120 may use the microservice chain modeler 130 to create an integrated microservice chain model.
  • a microservice chain is a composition of microservices, which can execute sequentially and/or in parallel.
  • An integrated microservice chain model may augment a conventional microservice chain model (e.g., which typically only includes basic information about the microservice chain such as microservice topology information) with various other information regarding a microservice chain that can be used by the tailored monitoring system 100 to perform tailored monitoring of microservice chain execution.
  • the integrated microservice chain model for a microservice chain may include an indication of the microservice topology of the microservice chain, an indication of which microservices in the microservice chain are critical microservices, an indication of the types of monitoring data to collect for different microservices in the microservice chain, an indication of the input and output data sets of different microservices in the microservice chain, an indication of which data items used by the microservices in the microservice chain are critical data items, an indication of data item dependencies between data items used by microservices in the microservice chain, and/or an indication of the monitoring intervals for different data items used by microservices in the microservice chain.
  • Some or all of the information included in the integrated microservice chain model may be provided/generated by the operator 120 and/or the microservice chain modeler 130.
  • the input data set of a microservice may be the set of data items that are provided as an input to the microservice, while the output data set of a microservice may be the set of data items that are provided as an output of the microservice.
  • a data item may be a piece of data that is used by microservices (e.g., as an input or an output of a microservice).
  • the critical microservices are the microservices that are deemed more important to monitor more closely.
  • the critical data items are the data items that are deemed more important to monitor more closely.
  • the data item dependencies indicate dependency relationships between different data items.
  • a data item dependency may be expressed in the form Di -> Dk, which indicates that the data item Di (which could be included in an input data set or output data set of a microservice) is dependent upon the data item Dk (which could also be included in an input data set or output data set of a microservice).
  • a group of data item dependencies may form a data item dependency chain/graph.
  • the types of monitoring data to collect for a microservice may include one or more of: the time at which the microservice began execution, the time at which the microservice completed execution, the edge site location at which the microservice began execution, the times at which the microservice changed state, the edge site locations at which the microservice changed state, the times at which the microservice resumed execution, the edge site locations at which the microservice resumed execution, and the edge site location at which the microservice completed execution.
  • the monitoring intervals for a data item indicate the points at which monitoring data for the data item is to be collected.
  • the monitoring intervals for a data item include one or more of: the times at which the data item is taken as an input of a microservice, the times at which the data item is provided as an output of a microservice, and the times at which the data item is updated by a microservice (e.g., after being taken as an input of a microservice and before being provided as an output of the microservice).
  • monitoring data for data items that are determined to be critical is collected with more granularity (collected more frequently and/or at more points of operation compared to other data items).
  • Microservice Chain C Topology based on Microservice Tuple: ⁇ Si ⁇ /* each Si is a microservice; S(i+ 1) is a successor of Si */
  • each Sj is a successor of Si - please note that any Si can have more than one successor */
  • Si: ⁇ Din, Dout ⁇ /* Din and Dout are the input and output data sets, respectively - this is the actual application data needed for the microservice to function */
  • Dj Boolean /* Dj is a data item that belongs to Din or Dout.
  • Lj is a boolean that is either “true” or “false”, depending on whether the data is considered critical or not; hence non-critical data need not be monitored */
  • the example representation provided above indicates the microservice topology of the microservice chain C (in set notation), the types of monitoring data to collect for each microservice Si (e.g., edge site location, start time, end time, interruption times, and resumption times), the input data set (Din) and output data set (Dout) of each microservice Si, the criticality of each data item (Dj), and the data item dependencies (expressed in the form Dj -> Dk).
  • the types of monitoring data to collect for each microservice Si e.g., edge site location, start time, end time, interruption times, and resumption times
  • Din input data set
  • Dout output data set
  • Dj criticality of each data item
  • Dj data item dependencies
  • the determination of which microservices in the microservice chain are critical is made based on identifying the critical path in the microservice chain. Any microservices that are on the critical path may then be labeled as critical microservices.
  • any data items that are in the input data sets or the output data sets of one of the critical microservices are determined to be critical data items.
  • data items that the critical data items are dependent upon are determined based on the data item dependency information.
  • critical microservices and critical data items, as well as any data items that the critical data items are dependent upon may be more closely monitored than other microservices and data items.
  • the types of monitoring data to collect for a microservice may be determined based on a variety of factors including whether the microservice has determined to be critical or not.
  • the types of monitoring data to collect for a critical microservice includes more types compared to that of other (non-critical) microservices.
  • the types of monitoring data to collect for a non-critical microservice may only include the time at which the microservice began execution and the time at which the microservice completed execution.
  • the types of monitoring data to collect for a critical microservice may include the above types and also include the edge site location at which the microservice began execution, the time instance at which the microservice changed state, the edge site locations at which the microservice changed state, the time instance at which the microservice resumed execution, the edge site locations at which the microservice resumed execution, and/or the edge site location at which the microservice completed execution.
  • the monitoring intervals for a data item may be determined based on a variety of factors including whether the data item has been determined to be critical or not and whether the data item is one that a critical data item is dependent upon.
  • the monitoring intervals for a critical data item or a data item that a critical data item is dependent upon may be more granular compared to the monitoring intervals for other data items.
  • the monitoring intervals for a data item that has determined not to be critical and not to be a data item that a critical data item is dependent upon may only include the times at which the data item is taken as an input of a microservice and the times at which the data item is provided as an output of a microservice.
  • the monitoring intervals for a critical data item or a data item that a critical data item is dependent upon may include the above monitoring intervals and also include the times at which the data item is updated by a microservice.
  • Selecting monitoring intervals that are too granular may flood the system with too much monitoring data, while selecting monitoring intervals that are too coarse may result in not collecting enough monitoring data to perform a useful analysis. If the operator 120 wishes to only know whether a microservice is working as designed or not, then it may be sufficient to only monitor when a data item is taken as input to the microservice and when a data item is provided as an output of the microservice. Otherwise, if the operator 120 wishes to know how well the microservice is performing, then it may be useful to monitor whenever a data item gets transformed within the microservice.
  • the granularity of monitoring intervals may be configurable depending on the requirements of the operator 120 in terms of how much monitoring data they wish to collect in order to analyze how well their microservice chain is performing. In other words, the operator 120 may determine the monitoring intervals based on achieving a balance between monitoring accuracy and the amount of monitoring data being collected.
  • the operator 120 and the microservice chain modeler 130 may work in conjunction to generate an integrated microservice chain model for a particular microservice chain.
  • the microservice chain modeler 130 sends the integrated microservice chain model to the microservice instance manager 150, which may instantiate the microservice chain in the distributed mobile edge cloud of the mobile network 110 based on the integrated microservice chain model.
  • the microservice chain modeler 130 may send the integrated microservice chain model to the microservice tracker 140.
  • the microservice tracker 140 may collect (from the distributed mobile edge cloud) monitoring data for each of one or more microservices in the microservice chain corresponding to the types of monitoring data to collect for the microservice as indicated in the integrated microservice chain model. This may involve, for example, collecting more types of monitoring data for critical microservices compared to other non-critical microservices.
  • the microservice tracker 140 also collects (from the distributed mobile edge cloud) monitoring data for each of one or more data items used by microservices in the microservice chain according to the monitoring intervals for the data item as indicated in the integrated microservice chain model.
  • microservice monitoring data may be collected from the distributed mobile edge cloud with the help of mobile network 110.
  • the microservice tracker 140 may send the collected microservice monitoring data to the monitoring data analyzer 180.
  • the microservice chain modeler 130 may send instantiated microservice chain information to the UE tracker 160.
  • the instantiated microservice chain information may include information about the instances of a microservice chain that are being executed in the distributed mobile edge cloud of the mobile network. This information may allow the UE tracker 160 to be able to track any UEs that are using an instance of the microservice chain.
  • the UE tracker 160 may track any UEs that are using an instance of the microservice chain and collect monitoring data for those UEs. For example, the UE tracker 160 may collect monitoring data related to the location and movement of any UEs using an instance of the microservice chain. The monitoring data collected by the UE tracker 160 may be referred to herein as UE monitoring data. The UE tracker 160 may collect the UE monitoring data with the help of components of the mobile network 110. The UE tracker 160 may send the collected UE monitoring data to the monitoring data analyzer 180.
  • the network tracker 170 may collect monitoring data related to the mobile network 110.
  • the network tracker 170 may collect monitoring data related to latency of communication between a UE and a microservice in the mobile network 110, latency of communication between microservices executing in the mobile network 110, and/or faults occurring at edge sites or other parts of the mobile network 110.
  • the monitoring data collected by the network tracker 170 may be referred to herein as network monitoring data.
  • the network tracker 170 may collect the network monitoring data with the help of components of the mobile network 110 such as a network exposure function (NEF) of the mobile network 110 (e.g., which can provide a means to securely expose the services and capabilities provided by network functions).
  • NEF network exposure function
  • a network repository function (e.g., which can maintain a record of 5G elements that are available in the mobile network 110 and their supported services; it may allow other network function instances to subscribe and be notified of registrations from network function instances of a given type), a network slice selection function (NSSF) (e.g., which can select network slices within which microservices can be implemented), unified data management (UDM) and unified data repository (UDR) (e.g., which can manage network user data in a single/centralized element; with the help of UDR, it may store data such as customer profile information, customer authentication information, and encryption of keys for the information), and/or a policy control function (PCF) (e.g., which can support a unified policy framework within the 5G infrastructure for governing network behavior; the PCF may access the subscription information required to make policy decisions from the UDM and provide the appropriate policy rules to the control plane).
  • the network tracker e.g., which can maintain a record of 5G elements that are available in the mobile network 110 and their
  • the monitoring data analyzer 180 may obtain microservice monitoring data from the microservice tracker 140, UE monitoring data from UE tracker 160, and network monitoring data from network tracker 170.
  • the monitoring data analyzer 180 may analyze the obtained data to generate an analysis result.
  • the monitoring data analyzer 180 may use known data analysis techniques to analyze the obtained monitoring data. For example, the monitoring data analyzer 180 may use machine learning algorithms to determine correlations between monitored data and identify potential problems/issues with the microservice chain execution (e.g., performance issues) and possibly identify root causes of those problems/issues.
  • the analysis can be performed in real-time (e.g., as monitoring data is being collected) and/or offline (e.g., after the monitoring data has been collected and stored in a storage medium (e.g., a log)) depending on the needs of the operator 120.
  • the monitoring data analyzer 180 may send the analysis result to the operator 120.
  • the monitoring data analyzer 180 may be implemented centrally or in a distributed manner. [0060] Embodiments may thus monitor microservice chain execution in a tailored manner that allows for gaining valuable insights into the operation of the microservice chain execution, while reducing the amount of monitoring data that is collected, which helps reduce the burden on the underlying infrastructure that has to collect the data and reduces the amount of data that has to be analyzed, thereby resulting in more efficient monitoring and analysis.
  • Figure 2 is a diagram showing component interactions for providing tailored monitoring of microservice chain execution in a distributed mobile edge cloud, according to some embodiments.
  • the operator 120 may send a request to the microservice chain modeler 130 to create an integrated microservice chain model for a specified microservice chain. As part of this request or in conjunction with this request, the operator 120 may also send a request to the microservice chain modeler 130 to determine the critical microservices in the microservice chain and/or determine the critical data items used by the microservices in the microservice chain, as well as data items that the critical data items are dependent upon. The microservice chain modeler 130 may send corresponding responses to the requests to the operator 120.
  • the microservice chain modeler 130 may then send a request to the microservice instance manager 150 to instantiate the microservice in the distributed mobile edge cloud.
  • the microservice instance manager 150 may instantiate the microservice in the distributed mobile edge cloud and send a corresponding response to the microservice chain modeler 130 (e.g., indicating that the microservice chain was successfully instantiated).
  • the microservice chain modeler 130 may send information about the instantiated microservice chain (and information about UEs using the instantiated microservice chain in some embodiments) to the UE tracker 160 and send the integrated microservice chain model to the microservice tracker 140.
  • the UE tracker 160 may collect UE monitoring data from the mobile network 110 (e.g., monitoring data related to the locations of UEs that use an instance of the microservice chain) and send the collected UE monitoring data to the monitoring data analyzer 180.
  • UE monitoring data from the mobile network 110 (e.g., monitoring data related to the locations of UEs that use an instance of the microservice chain) and send the collected UE monitoring data to the monitoring data analyzer 180.
  • the microservice tracker 140 may collect microservice monitoring data from the distributed mobile edge cloud of the mobile network 110 (in a tailored manner in accordance with the integrated microservice chain model) and send the collected microservice monitoring data to the monitoring data analyzer 180.
  • the network tracker 170 may collect network monitoring data from the mobile network 110 (e.g., monitoring data related to network performance related to the microservice chain execution) and send the collected network monitoring data to the monitoring data analyzer 180.
  • network monitoring data e.g., monitoring data related to network performance related to the microservice chain execution
  • the monitoring data analyzer 180 may analyze the microservice monitoring data, the UE monitoring data, and the network monitoring data to generate an analysis result (e.g., identifying potential problems/issues in the microservice chain execution and possible root causes). The monitoring data analyzer 180 may then send the analysis result to the operator 120.
  • an analysis result e.g., identifying potential problems/issues in the microservice chain execution and possible root causes.
  • the operator 120 manages and uses the tailored monitoring system 100.
  • the input of the operator 120 may include instructions from end users regarding which microservice chain(s) to monitor.
  • the output of the operator 120 may include a conventional (non-integrated) microservice chain model, an indication of the critical microservices in the microservice chain, and an indication of the critical data items used by the microservices in the microservice chain, along with dependencies thereof.
  • the actions of the operator 120 may include sending messages to the microservice chain modeler 130 asking it to create and maintain the integrated microservice chain model (including an indication of the determined critical microservices) and their data including dependencies thereof. It should be noted that some of the operations of the operator 120 may be performed by a human or by an automated system that is controlled by a human user (e.g., the operator 120 may be an automated agent that is being controlled by a human user).
  • the microservice chain modeler 130 is responsible for creating and maintaining the integrated microservice chain model.
  • the input of the microservice chain modeler 130 may include the microservice chain model, an indication of the critical microservices in the microservice chain, and an indication of the critical data items used by the microservices in the microservice chain along with dependencies thereof.
  • the output of the microservice chain modeler 130 may include a microservice instantiation request that is to be sent to the microservice instance manager 150 and integrated UE and microservice instance information that is to be sent to the UE tracker 160 (this may help the UE tracker 160 track UEs that are using an instance of the microservice chain (it should be noted that a UE may be mobile and not fixed to a particular location)).
  • the actions of the microservice chain modeler 130 may include creating the integrated microservice chain model and sending a request to the microservice instance manager 150 to instantiate the microservice chain.
  • the actions of the microservice chain modeler 130 may also include sending the microservice instance information to the UE tracker 160 so that the UE tracker 160 can track the UEs that are using an instance of the microservice chain.
  • the microservice instance manager 150 is responsible for instantiating the microservice chain per requests/instructions received from the microservice chain modeler 130 and ensuring that it is running correctly in the distributed mobile edge cloud of the mobile network 110.
  • the input of the microservice instance manager 150 may include the instantiation request received from the microservice chain modeler 130.
  • the output of the microservice instance manager 150 may include an instantiation confirmation response to be sent to the microservice chain modeler 130.
  • the actions of the microservice instance manager 150 may include instantiating the microservice chain and sending the instantiation confirmation response to the microservice chain modeler 130.
  • the mobile network 110 may be responsible for providing various monitoring data related to microservice chain execution to the microservice tracker 140, the UE tracker 160, and the network tracker 170.
  • the input of the mobile network 110 may include monitoring data received from underlying network components.
  • the output of the mobile network 110 may include (tailored) microservice monitoring data, UE monitoring data, and network monitoring data.
  • the actions of the mobile network 110 may include providing the microservice monitoring data, UE monitoring data, and network monitoring data to the microservice tracker 140, the UE tracker 160, and the network tracker 170, respectively.
  • the microservice tracker 140 is responsible for collecting microservice monitoring data.
  • the input of the microservice tracker 140 may include microservice monitoring data collected from the mobile network 110 according to the integrated microservice chain model.
  • the output of the microservice tracker 140 may include the collected microservice monitoring data.
  • the actions of the microservice tracker 140 may include tracking microservice chain execution (e.g., tracking microservice instances being executed in the distributed mobile edge cloud with the help of mobile network components), collecting relevant microservice monitoring data according to the integrated microservice chain model, and sending collected microservice monitoring data to the monitoring data analyzer 180.
  • the UE tracker 160 is responsible for collecting UE monitoring data for UEs that use an instance of the microservice chain.
  • the input of the UE tracker 160 may include UE monitoring data collected from the mobile network 110.
  • the output of the UE tracker 160 may include UE monitoring data to be sent to the monitoring data analyzer 180.
  • the actions of the UE tracker 160 may include tracking relevant UE actions/movements with the help of mobile network components, collecting relevant UE monitoring data, and sending collected UE monitoring data to the monitoring data analyzer 180.
  • the network tracker 170 is responsible for collecting network monitoring data.
  • the input of the network tracker 170 may include network monitoring data collected from the mobile network 110.
  • the output of the network tracker 170 may include network monitoring data to be sent to the monitoring data analyzer 180.
  • the actions of the network tracker 170 may include collecting relevant network monitoring data and sending it to the monitoring data analyzer 180.
  • the monitoring data analyzer 180 is responsible for analyzing various monitoring data to generate an analysis result.
  • the input of the monitoring data analyzer 180 may include microservice monitoring data, UE monitoring data, and network monitoring data.
  • the output of the monitoring data analyzer may include an analysis result (e.g., synthesized/analyzed monitoring information) to be sent to the operator 120.
  • the actions of the monitoring data analyzer 180 may include obtaining microservice monitoring data, UE monitoring data, and network monitoring data and synthesizing/analyzing the obtained monitoring data to generate an analysis result and sending the analysis result to the operator 120.
  • Figure 3 is a flow diagram showing a method for tailored monitoring of microservice execution in a distributed mobile edge cloud, according to some embodiments.
  • the method may be performed by one or more network devices (e.g., implementing a tailored monitoring system).
  • network devices e.g., implementing a tailored monitoring system.
  • the operations in the flow diagrams will be described with reference to the exemplary embodiments of the other figures. However, it should be understood that the operations of the flow diagrams can be performed by embodiments other than those discussed with reference to the other figures, and the embodiments discussed with reference to these other figures can perform operations different than those discussed with reference to the flow diagrams.
  • the one or more network devices generate an integrated microservice chain model for a microservice chain.
  • the one or more network devices determine which microservices in the microservice chain are critical microservices.
  • the one or more network devices determine which data items used by the microservices in the microservice chain are critical data items or data items that the critical items are dependent upon. [0084] At block 340, the one or more network devices collect more types of monitoring data for the critical microservices (compared to non-critical microservices) and collect monitoring data for critical data items and data items that the critical items are dependent upon with more granularity (compared to other data items).
  • the one or more network devices analyze the collected monitoring data (e.g., offline or online (in real-time)) to generate an analysis result.
  • the collected monitoring data e.g., offline or online (in real-time)
  • Figure 4 is a flow diagram showing a method for determining data item criticality, according to some embodiments.
  • the method may be performed by one or more network devices (e.g., implementing an operator 120 and/or a microservice chain modeler 130).
  • network devices e.g., implementing an operator 120 and/or a microservice chain modeler 130.
  • the one or more network devices model a microservice chain as a workflow.
  • the one or more network devices label all input and output data items for each microservice in the microservice chain as being non-critical data items (this is an initialization step).
  • the one or more network devices identify a critical path of the workflow.
  • Critical path determination can be performed using any suitable technique.
  • the critical path is the longest path of the workflow between the start activity and the end activity (e.g., the longest path between the beginning of the microservice chain and the end of the microservice chain).
  • the one or more network devices determine which microservices are on the critical path.
  • the one or more network devices label any data items that are in the input data set or output data set of a microservice on the critical path as being critical data items.
  • FIG. 5 is a diagram showing a microservice topology for a movie delivery service microservice chain, according to some embodiments.
  • the movie delivery service microservice chain includes a movie selection microservice (SI) 510, a movie encoding microservice (S2) 520, a movie streaming microservice (S3) 530, a view reviews microservice (S4) 540, and an enter review microservice (S5) 550.
  • SI movie selection microservice
  • S2 movie encoding microservice
  • S3 movie streaming microservice
  • S4 movie streaming microservice
  • S5 enter review microservice
  • the microservice topology of the movie delivery service microservice chain may be represented as: ⁇ SI, ⁇ S2, S4, S5 ⁇ , S3 ⁇ .
  • the input data set and output data set of each microservice may be as follows:
  • a critical path identification method may determine that the movie selection 510, movie encoding 520 and movie streaming 530 microservices are critical microservices since they are on the critical path (the longest path in the microservice chain between “begin” and “end” (the path SI -> S2 -> S3)).
  • monitoring data may be collected for these microservices: starting time (the time at which the microservice began execution), the edge site location at which the microservice began execution, the times at which the microservice changed state (e.g., it was migrated or suspended), the edge site locations at which the microservice changed state, the times at which the microservice resumed execution, the edge site locations at which the microservice resumed execution, ending time (the time at which the microservice completed execution), and the edge site location at which the microservice completed execution.
  • monitoring data For the other two non-critical microservices (view reviews 540 and enter review 550 microservices), only the following types of monitoring data may be collected: starting time and ending time.
  • the microservice tracker 140 may collect the above types of monitoring data for the respective microservices from the distributed mobile edge cloud.
  • the relevant network monitoring data and UE monitoring data may be collected for the three critical microservices.
  • This monitoring data may be collected in an automated manner by the mobile network 110 (e.g., using known monitoring techniques that are used by mobile network operators to monitor the operations of their networks at the network layer).
  • the network monitoring data (which may be collected by the network tracker 170) may include network latency related data associated with the critical microservices (e.g., at different points of operation mentioned above such as when the microservice began execution, when the microservice changed state, when the microservice resumed execution, and when the microservice completed execution).
  • the network latency related data may include the latency of interaction of a UE with each critical microservice and the latency of inter-microservice interactions (e.g., latency between the movie selection microservice 510 and the movie encoding microservice 520 and latency between the movie encoding microservice 520 and the movie streaming microservice 530).
  • the UE monitoring data (which may be collected by the UE tracker 160) may include the UE location at different points of operation mentioned above (e.g., to which edge site location the UE is connected and/or to which user plane function (UPF) of the mobile network 110 the UE is connected).
  • UPF user plane function
  • the network monitoring data and UE monitoring data become dependent data for microservice execution time, as collecting just the microservice monitoring data alone may be insufficient without also collecting the network latency-related data and UE location related data. For example, a network latency longer than an expected length may be correlated against the overall duration of the movie stream to determine how much this extra network latency contributed to the overall time to stream the movie.
  • the above collected monitoring data may be used to analyze the performance of the one-hour movie. This analysis may be performed by the monitoring data analyzer 180.
  • the microservice monitoring data shows that it took one hour and five minutes to stream the movie instead of just one hour, then various monitoring data may be correlated, and some conclusions can be reached. For example, if it assumed that the movie streaming was interrupted due to UE migration, this can be determined by correlating the times at which the microservice changed state and the UE locations at that time. If the correlation shows that the state/session change - interruption and resumption - coincide with UE location changes, then this related delay may be calculated. A variant of this could be autoscaling (where a new instance of the microservice is created at the new edge site location to which the UE has moved) but that may also be modeled as a combination of interruption and resumption.
  • the above analysis may also be done in real-time by the monitoring data analyzer 180 while the microservice chain is being executed (e.g., while the movie is still streaming). For example, if the movie has reached the time of 45:53 (45 minutes and 53 seconds) but 48 minutes have actually elapsed, a real-time analysis of the collected monitoring data may help to provide indications of where most of the extra two minutes and seven seconds may have gone, and help operators readjust the underlying infrastructure (whether network or UE migration related or edge cloud related) so that further delays can be avoided.
  • Figure 6 is a diagram showing a microservice topology for a drone swarm microservice chain, according to some embodiments.
  • This example features a collection (or swarm) of drones traveling together through a set of edge zones each managed by an edge site (represented as an edge router in the diagram).
  • the edge routers are in turn connected to a central cloud.
  • a critical path identification method may determine that image recognition (IR) and obstacle avoidance (OA) microservices belong to the critical path.
  • IR image recognition
  • OA obstacle avoidance
  • these microservices may invoke other microservices (e.g., camera, location, speed, etc.).
  • the actual paths of the drones may be determined via the ConstructRoute microservice running in the cloud.
  • IR and OA microservices are therefore offloaded from the drone to the nearest edge router (edge site) where they are executed. As the drone moves between edge sites, these microservices would also need to be migrated to the edge site under whose jurisdiction the drone will move.
  • the UEs in this case are the drones (can be assumed to be equipped with subscriber identification module (SIM) cards to help them communicate with edge sites), accessing the latency critical IR and OA microservices. Since the drones need to move to reach their destinations, this makes UE mobility across edge sites a given. There may be network congestion due to large numbers of drones accessing an edge site. As such, even if a drone is under the jurisdiction of, say, edge site El, its IR and/or OA microservices may have to be migrated to another edge site, say, E2, or to the cloud itself.
  • SIM subscriber identification module
  • Edge sites can fail at any time. As such, if El mentioned above fails - or shows indications that it may fail soon as per data from suitable monitoring tools - then the IR and OA microservices running on it may have to be migrated to other edge sites. Not all of them can be migrated to the same edge site, hence they may have to be distributed among multiple edge sites and even the cloud until El starts functioning properly again.
  • the following types of monitoring data may be collected for the critical microservices IR and OA: starting time (the time at which the microservice began execution), the edge site location at which the microservice began execution, the times at which the microservice changed state (e.g., it was migrated or suspended), the times at which the microservice resumed execution, and ending time (the time at which the microservice completed execution).
  • monitoring data For the other non-critical microservices, only the following types of monitoring data may be collected: starting time and ending time.
  • the microservice tracker 140 may collect the above types of monitoring data for the respective microservices from the distributed mobile edge cloud.
  • the relevant network monitoring data and UE monitoring data may be collected for the three critical microservices.
  • the network monitoring data (which may be collected by the network tracker 170) may include network latency related data associated with the critical microservices (e.g., at different points of operation mentioned above such as when the microservice began execution, when the microservice changed state, when the microservice resumed execution, and when the microservice completed execution).
  • the network monitoring data may also include data related to infrastructure faults/failure at any edge sites (if any) such as the time of fault, the time when microservices executed on the failed edge site had to be migrated to another edge site, and the time at which microservice execution was resumed.
  • the UE monitoring data (which may be collected by the UE tracker 160) may include the UE (e.g., drone) location at different points of operation mentioned above.
  • this data may be needed in order to complete the monitoring of the two critical microservices.
  • IR and OA microservices are supposed to complete execution within a particular time (e.g., 50 milliseconds each)
  • the actual time taken for each of these microservices to complete execution may then be correlated against both 50 milliseconds as well as the combination of network latency, UE location, and faults/failures if any. Similar to the movie delivery service example, this analysis can be performed by the monitoring data analyzer 180.
  • This tailored approach to collecting monitoring data may provide the appropriate insights into the operation of the microservice chain execution, while reducing the amount of monitoring data that needs to be collected.
  • Figure 7 is a flow diagram showing a method for generating an integrated microservice chain model, according to some embodiments.
  • the method may be implemented by one or more network devices (e.g., implementing the operator 120 and/or the microservice chain modeler 130).
  • the one or more network devices determine a microservice topology of a microservice chain.
  • the one or more network devices determine which microservices in the microservice chain are critical microservices based on the microservice topology.
  • the determination of which microservices in the microservice chain are critical microservices is based on identifying a critical path in the microservice chain and identifying microservices on the critical path as being critical microservices.
  • the one or more network devices determine which types of monitoring data to collect for each of one or more microservices in the microservice chain based on whether the microservice was determined to be a critical microservice.
  • the types of monitoring data to collect for a critical microservice comprises more types of monitoring data compared to the types of monitoring data to collect for a non-critical microservice.
  • the types of monitoring data to collect for the critical microservice and the types of monitoring data to collect for the non-critical microservice both comprise a time at which the microservice began execution and a time at which the microservice completed execution, wherein the types of monitoring data to collect for the critical microservice further comprises one or more other types of monitoring data that are not included in the types of monitoring data to collect for the non-critical microservice.
  • the one or more other types of monitoring data comprises one or more of: edge site location at which the microservice began execution, times at which the microservice changed state, edge site locations at which the microservice changed state, times at which the microservice resumed execution, edge site locations at which the microservice resumed execution, and edge site location at which the microservice completed execution.
  • the one or more network devices generate an integrated microservice chain model for the microservice chain that comprises an indication of the microservice topology, an indication of the critical microservices, and an indication of the types of monitoring data to collect for each of the one or more microservices.
  • the one or more network devices provide the integrated microservice chain model to a microservice tracker, wherein the microservice tracker is to collect, from the distributed mobile edge cloud while the microservice chain is being executed in the distributed mobile edge cloud, monitoring data for each of the one or more microservices corresponding to the types of monitoring data to collect for the microservice as indicated in the integrated microservice chain model and provide the collected monitoring data to a monitoring data analyzer that is to analyze the collected monitoring data for the one or more microservices, together with UE monitoring data and network monitoring data, to generate an analysis result.
  • the microservice tracker is to collect, from the distributed mobile edge cloud while the microservice chain is being executed in the distributed mobile edge cloud, monitoring data for each of the one or more microservices corresponding to the types of monitoring data to collect for the microservice as indicated in the integrated microservice chain model and provide the collected monitoring data to a monitoring data analyzer that is to analyze the collected monitoring data for the one or more microservices, together with UE monitoring data and network monitoring data, to generate an analysis result.
  • the one or more network devices also determine input and output data sets of each microservice in the microservice chain, determine which data items used by microservices in the microservice chain are critical data items, determine data item dependencies between data items used by the microservices in the microservice chain, determine, based on the data item dependencies, which data items used by the microservices in the microservice chain are data items that the critical data items are dependent upon, determine monitoring intervals for each of one or more data items used by the microservices in the microservice chain based on whether the data item was determined to be a critical data item or a data item that a critical item is dependent upon, and add, to the integrated microservice chain model, an indication of the input and output data sets of each microservice in the microservice chain, an indication of the critical data items, an indication of the data items that the critical data items are dependent upon, and an indication of the monitoring intervals for each of the one or more data items, wherein the microservice tracker is to further collect, from the distributed mobile edge cloud, monitoring data for each of
  • the determination of which data items used by the microservices in the microservice chain are critical data items is based on identifying data items in the input and output data sets of the critical microservices as being critical data items.
  • the monitoring intervals for a critical data item or a data item that a critical data item depends on are more granular compared to the monitoring intervals for a non- critical data item.
  • the monitoring intervals for the critical data item and the monitoring intervals for the non-critical data items both comprise times at which the data item is taken as an input of a microservice and times at which the data item is provided as an output of a microservice, wherein the monitoring intervals for the critical data item further comprises one or more other monitoring intervals not included in the monitoring intervals for the non-critical data item.
  • the one or more other monitoring intervals comprise times at which the data item is updated by a microservice.
  • the one or more network devices cause the microservice chain to be instantiated in the distributed mobile edge cloud and provide information regarding the instantiated microservice chain to a UE tracker, wherein the UE tracker is configured to use the information regarding the instantiated microservice chain to collect UE monitoring data from a mobile network for UEs that use the instantiated microservice chain.
  • Figure 8 is a flow diagram showing a method for collecting monitoring data in a tailored manner, according to some embodiments.
  • the method may be implemented by one or more network devices (e.g., implementing the microservice tracker 140).
  • the one or more network devices obtain an integrated microservice chain model for a microservice chain, wherein the integrated microservice chain model comprises an indication of a microservice topology of the microservice chain, an indication of which microservices in the microservice chain are critical microservices, and an indication of which types of monitoring data to collect for each of one or more microservices in the microservice chain.
  • the one or more network devices collect, from the distributed mobile edge cloud while the microservice chain is being executed in in the distributed mobile edge cloud, monitoring data for each of the one or more microservices corresponding to the types of monitoring data to collect for the microservice as indicated in the integrated microservice chain model.
  • monitoring data collected for a critical microservice comprises more types of monitoring data compared to the monitoring data collected for a non-critical microservice.
  • the one or more network devices provide the collected monitoring data for the one or more microservices to a monitoring data analyzer, wherein the monitoring data analyzer is to analyze the collected monitoring data for the one or more microservices, together with user equipment monitoring data and network monitoring data, to generate an analysis result.
  • the analysis result is generated in real-time as monitoring data is being collected.
  • the integrated microservice chain model further includes an indication of input and output data sets of each microservice in the microservice chain, an indication of critical data items used by microservices in the microservice chain, an indication of which data items used by the microservices in the microservice chain are data items that the critical data items are dependent upon, and an indication of monitoring intervals for each of one or more data items.
  • the one or more network devices collect, from the distributed mobile edge cloud, monitoring data for each of the one or more data items according to the monitoring intervals for the data item as indicated in the integrated microservice chain model and provide the collected monitoring data for the one or more data items to the monitoring data analyzer, wherein the monitoring data analyzer is to further analyze the collected monitoring data for the one or more data items to generate the analysis result.
  • the monitoring intervals for a critical data item are more granular compared to the monitoring intervals for a non- critical data item.
  • Figure 9 is a flow diagram showing a method for analyzing monitoring data, according to some embodiments.
  • the method may be implemented by one or more network devices (e.g., implementing the monitoring data analyzer 180).
  • the one or more network devices obtain microservice monitoring data for a microservice chain, UE monitoring data, and network monitoring data, wherein the microservice monitoring data includes first monitoring data for one or more microservices in the microservice chain that were determined to be critical microservices and second monitoring data for one or more microservices in the microservice chain that were determined not to be critical microservices, wherein the first monitoring data comprises more types of monitoring data compared to the second monitoring data.
  • the one or more network devices analyze the microservice monitoring data, the UE monitoring data, and the network monitoring data to generate an analysis result (in real-time as monitoring data is being collected or offline).
  • Figure 10A illustrates connectivity between network devices (NDs) within an exemplary network, as well as three exemplary implementations of the NDs, according to some embodiments.
  • Figure 10A shows NDs 1000A-H, and their connectivity by way of lines between 1000 A- 1000B, 1000B-1000C, 1000C-1000D, 1000D-1000E, 1000E-1000F, 1000F- 1000G, and 1000A-1000G, as well as between 1000H and each of 1000 A, 1000C, WOOD, and 1000G.
  • These NDs are physical devices, and the connectivity between these NDs can be wireless or wired (often referred to as a link).
  • NDs 1000A, 1000E, and lOOOF An additional line extending from NDs 1000A, 1000E, and lOOOF illustrates that these NDs act as ingress and egress points for the network (and thus, these NDs are sometimes referred to as edge NDs; while the other NDs may be called core NDs).
  • Two of the exemplary ND implementations in Figure 10A are: 1) a special-purpose network device 1002 that uses custom application-specific integrated-circuits (ASICs) and a special-purpose operating system (OS); and 2) a general purpose network device 1004 that uses common off-the-shelf (COTS) processors and a standard OS.
  • ASICs application-specific integrated-circuits
  • OS special-purpose operating system
  • COTS common off-the-shelf
  • the special-purpose network device 1002 includes networking hardware 1010 comprising a set of one or more processor(s) 1012, forwarding resource(s) 1014 (which typically include one or more ASICs and/or network processors), and physical network interfaces (NIs) 1016 (through which network connections are made, such as those shown by the connectivity between NDs 1000A-H), as well as non-transitory machine readable storage media 1018 having stored therein networking software 1020.
  • the networking software 1020 may be executed by the networking hardware 1010 to instantiate a set of one or more networking software instance(s) 1022.
  • Each of the networking software instance(s) 1022, and that part of the networking hardware 1010 that executes that network software instance form a separate virtual network element 1030A-R.
  • Each of the virtual network element(s) (VNEs) 1030A-R includes a control communication and configuration module 1032A-R (sometimes referred to as a local control module or control communication module) and forwarding table(s) 1034A-R, such that a given virtual network element (e.g., 1030A) includes the control communication and configuration module (e.g., 1032A), a set of one or more forwarding table(s) (e.g., 1034A), and that portion of the networking hardware 1010 that executes the virtual network element (e.g., 1030A).
  • a control communication and configuration module 1032A-R sometimes referred to as a local control module or control communication module
  • forwarding table(s) 1034A-R forwarding table(s) 1034A-R
  • software 1020 includes code such as tailored monitoring component 1023, which when executed by networking hardware 1010, causes the special -purpose network device 1002 to perform operations of one or more embodiments as part of networking software instances 1022 (e.g., to provide tailored monitoring of microservice chain execution in a distributed mobile edge cloud).
  • tailored monitoring component 1023 when executed by networking hardware 1010, causes the special -purpose network device 1002 to perform operations of one or more embodiments as part of networking software instances 1022 (e.g., to provide tailored monitoring of microservice chain execution in a distributed mobile edge cloud).
  • the special-purpose network device 1002 is often physically and/or logically considered to include: 1) a ND control plane 1024 (sometimes referred to as a control plane) comprising the processor(s) 1012 that execute the control communication and configuration module(s) 1032A-R; and 2) a ND forwarding plane 1026 (sometimes referred to as a forwarding plane, a data plane, or a media plane) comprising the forwarding resource(s) 1014 that utilize the forwarding table(s) 1034A-R and the physical NIs 1016.
  • a ND control plane 1024 (sometimes referred to as a control plane) comprising the processor(s) 1012 that execute the control communication and configuration module(s) 1032A-R
  • a ND forwarding plane 1026 sometimes referred to as a forwarding plane, a data plane, or a media plane
  • the forwarding resource(s) 1014 that utilize the forwarding table(s) 1034A-R and the physical NIs 1016.
  • the ND control plane 1024 (the processor(s) 1012 executing the control communication and configuration module(s) 1032A-R) is typically responsible for participating in controlling how data (e.g., packets) is to be routed (e.g., the next hop for the data and the outgoing physical NI for that data) and storing that routing information in the forwarding table(s) 1034A-R, and the ND forwarding plane 1026 is responsible for receiving that data on the physical NIs 1016 and forwarding that data out the appropriate ones of the physical NIs 1016 based on the forwarding table(s) 1034A-R.
  • data e.g., packets
  • the ND forwarding plane 1026 is responsible for receiving that data on the physical NIs 1016 and forwarding that data out the appropriate ones of the physical NIs 1016 based on the forwarding table(s) 1034A-R.
  • Figure 10B illustrates an exemplary way to implement the special-purpose network device 1002 according to some embodiments.
  • Figure 10B shows a special-purpose network device including cards 1038 (typically hot pluggable). While in some embodiments the cards 1038 are of two types (one or more that operate as the ND forwarding plane 1026 (sometimes called line cards), and one or more that operate to implement the ND control plane 1024 (sometimes called control cards)), alternative embodiments may combine functionality onto a single card and/or include additional card types (e.g., one additional type of card is called a service card, resource card, or multi-application card).
  • additional card types e.g., one additional type of card is called a service card, resource card, or multi-application card.
  • a service card can provide specialized processing (e.g., Layer 4 to Layer 7 services (e.g., firewall, Internet Protocol Security (IPsec), Secure Sockets Layer (SSL) / Transport Layer Security (TLS), Intrusion Detection System (IDS), peer-to-peer (P2P), Voice over IP (VoIP) Session Border Controller, Mobile Wireless Gateways (Gateway General Packet Radio Service (GPRS) Support Node (GGSN), Evolved Packet Core (EPC) Gateway)).
  • Layer 4 to Layer 7 services e.g., firewall, Internet Protocol Security (IPsec), Secure Sockets Layer (SSL) / Transport Layer Security (TLS), Intrusion Detection System (IDS), peer-to-peer (P2P), Voice over IP (VoIP) Session Border Controller, Mobile Wireless Gateways (Gateway General Packet Radio Service (GPRS) Support Node (GGSN), Evolved Packet Core (EPC) Gateway)
  • GPRS General Pack
  • the general purpose network device 1004 includes hardware 1040 comprising a set of one or more processor(s) 1042 (which are often COTS processors) and physical NIs 1046, as well as non-transitory machine-readable storage media 1048 having stored therein software 1050.
  • the processor(s) 1042 execute the software 1050 to instantiate one or more sets of one or more applications 1064A-R. While one embodiment does not implement virtualization, alternative embodiments may use different forms of virtualization.
  • the virtualization layer 1054 represents the kernel of an operating system (or a shim executing on a base operating system) that allows for the creation of multiple instances 1062A-R called software containers that may each be used to execute one (or more) of the sets of applications 1064A-R; where the multiple software containers (also called virtualization engines, virtual private servers, or jails) are user spaces (typically a virtual memory space) that are separate from each other and separate from the kernel space in which the operating system is run; and where the set of applications running in a given user space, unless explicitly allowed, cannot access the memory of the other processes.
  • the multiple software containers also called virtualization engines, virtual private servers, or jails
  • user spaces typically a virtual memory space
  • the virtualization layer 1054 represents a hypervisor (sometimes referred to as a virtual machine monitor (VMM)) or a hypervisor executing on top of a host operating system, and each of the sets of applications 1064A-R is run on top of a guest operating system within an instance 1062A-R called a virtual machine (which may in some cases be considered a tightly isolated form of software container) that is run on top of the hypervisor - the guest operating system and application may not know they are running on a virtual machine as opposed to running on a “bare metal” host electronic device, or through para-virtualization the operating system and/or application may be aware of the presence of virtualization for optimization purposes.
  • a hypervisor sometimes referred to as a virtual machine monitor (VMM)
  • VMM virtual machine monitor
  • one, some or all of the applications are implemented as unikemel(s), which can be generated by compiling directly with an application only a limited set of libraries (e.g., from a library operating system (LibOS) including drivers/! i branes of OS services) that provide the particular OS services needed by the application.
  • libraries e.g., from a library operating system (LibOS) including drivers/! i branes of OS services
  • unikemel can be implemented to run directly on hardware 1040, directly on a hypervisor (in which case the unikemel is sometimes described as running within a LibOS virtual machine), or in a software container
  • embodiments can be implemented fully with unikemels running directly on a hypervisor represented by virtualization layer 1054, unikemels running within software containers represented by instances 1062A-R, or as a combination of unikemels and the abovedescribed techniques (e.g., unikemels and virtual machines both run directly on a hypervisor, unikemels and sets of applications that are run in different software containers).
  • the instantiation of the one or more sets of one or more applications 1064A-R, as well as virtualization if implemented, are collectively referred to as software instance(s) 1052.
  • the virtual network element(s) 1060A-R perform similar functionality to the virtual network element(s) 1030A-R - e.g., similar to the control communication and configuration module(s) 1032A and forwarding table(s) 1034A (this virtualization of the hardware 1040 is sometimes referred to as network function virtualization (NFV)).
  • NFV network function virtualization
  • CPE customer premise equipment
  • each instance 1062A-R corresponding to one VNE 1060A-R
  • alternative embodiments may implement this correspondence at a finer level granularity (e.g., line card virtual machines virtualize line cards, control card virtual machine virtualize control cards, etc.); it should be understood that the techniques described herein with reference to a correspondence of instances 1062A-R to VNEs also apply to embodiments where such a finer level of granularity and/or unikemels are used.
  • the virtualization layer 1054 includes a virtual switch that provides similar forwarding services as a physical Ethernet switch. Specifically, this virtual switch forwards traffic between instances 1062A-R and the physical NI(s) 1046, as well as optionally between the instances 1062A-R; in addition, this virtual switch may enforce network isolation between the VNEs 1060A-R that by policy are not permitted to communicate with each other (e.g., by honoring virtual local area networks (VLANs)).
  • VLANs virtual local area networks
  • software 1050 includes a tailored monitoring component 1053, which when executed by processor(s) 1042, causes the general purpose network device 1004 to perform operations of one or more embodiments as part of software instances 1062A-R (e.g., to provide tailored monitoring of microservice chain execution in a distributed mobile edge cloud).
  • the third exemplary ND implementation in Figure 10A is a hybrid network device 1006, which includes both custom ASICs/special -purpose OS and COTS processors/standard OS in a single ND or a single card within an ND.
  • a platform VM i.e., a VM that that implements the functionality of the special-purpose network device 1002 could provide for para- virtualization to the networking hardware present in the hybrid network device 1006.
  • a network interface may be physical or virtual; and in the context of IP, an interface address is an IP address assigned to a NI, be it a physical NI or virtual NI.
  • a virtual NI may be associated with a physical NI, with another virtual interface, or stand on its own (e.g., a loopback interface, a point-to-point protocol interface).
  • a NI physical or virtual
  • a loopback interface (and its loopback address) is a specific type of virtual NI (and IP address) of a NE/VNE (physical or virtual) often used for management purposes; where such an IP address is referred to as the nodal loopback address.
  • IP addresses of that ND are referred to as IP addresses of that ND; at a more granular level, the IP address(es) assigned to NI(s) assigned to a NE/VNE implemented on a ND can be referred to as IP addresses of that NE/VNE.
  • these quantities take the form of electrical or magnetic signals capable of being stored, transferred, combined, compared, and otherwise manipulated. It has proven convenient at times, principally for reasons of common usage, to refer to these signals as bits, values, elements, symbols, characters, terms, numbers, or the like.
  • An embodiment may be an article of manufacture in which a non-transitory machine- readable storage medium (such as microelectronic memory) has stored thereon instructions (e.g., computer code) which program one or more data processing components (generically referred to here as a “processor”) to perform the operations described above.
  • instructions e.g., computer code
  • processor data processing components
  • some of these operations might be performed by specific hardware components that contain hardwired logic (e.g., dedicated digital filter blocks and state machines). Those operations might alternatively be performed by any combination of programmed data processing components and fixed hardwired circuit components.

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Software Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Data Exchanges In Wide-Area Networks (AREA)

Abstract

A method is disclosed for providing tailored monitoring of microservice chain execution in a distributed mobile edge cloud. The method includes determining a microservice topology of a microservice chain, determining which microservices in the microservice chain are critical microservices based on the microservice topology, determining which types of monitoring data to collect for each of one or more microservices in the microservice chain based on whether the microservice was determined to be a critical microservice, generating an integrated microservice chain model for the microservice chain that comprises an indication of the microservice topology, an indication of the critical microservices, and an indication of the types of monitoring data to collect for each of the one or more microservices, and providing the integrated microservice chain model to a microservice tracker that is to collect monitoring data for the one or more microservices in accordance with the integrated microservice chain model.

Description

SPECIFICATION
METHOD AND APPARATUS FOR TAILORED DATA MONITORING OF MICROSERVICE EXECUTIONS IN MOBILE EDGE CLOUDS
TECHNICAL FIELD
[0001] Embodiments disclosed herein relate to the field of distributed mobile edge clouds, and more specifically to methods and apparatus for tailored monitoring of microservice chain execution in a distributed mobile edge cloud.
BACKGROUND
[0002] Recently, application deployment and execution have been moving in a combination of the following two directions: (1) from being deployed in centralized cloud-based environments to being deployed in highly distributed edge-based environments such as distributed mobile edge clouds; and (2) from the use of monolithic architectures to an architecture that comprises chains of smaller services called “microservices.” This enables the distributed execution of applications as microservice chains across widely distributed edge sites. A natural outcome of this is that microservices are also migrated across edge sites in order to ensure uninterrupted access to end users, who may also be moving around.
[0003] Such a volatile and dynamic infrastructure requires a highly distributed and efficient mechanism of monitoring application execution to enable service assurance. Of particular interest here is data monitoring, that is, monitoring of the application data that is flowing through the system and which is being used by microservices during their execution. Such monitoring may help determine whether the application is performing as designed.
[0004] Data handling is the most important aspect of microservice chain execution in distributed mobile edge clouds. In particular, in domains such as video streaming, transportation/logistics, vehicle to everything (V2X), and gaming, data migration may become a major stumbling block leading to user dissatisfaction if not handled properly. As such, monitoring and governance of the data migration to identify problems and issues, if any, and correct them, is of utmost importance. [0005] However, given the amount of possible monitoring information (also referred to as “monitoring data,” and also sometimes referred to, for ease of explanation, as “metrics” or “metrics data”) that can be collected for microservices being executed in the distributed mobile edge cloud and their data usage, there is a danger of being flooded by too much irrelevant information, which makes it difficult to quickly identify any problems or issues in microservice chain execution. In general, it is well known that the amount of information about data usage is often several orders of magnitude greater than the amount of the actual data itself.
SUMMARY
[0006] An example embodiment is a method performed by one or more network devices to provide tailored monitoring of microservice chain execution in a distributed mobile edge cloud. The method includes determining a microservice topology of a microservice chain, determining which microservices in the microservice chain are critical microservices based on the microservice topology, determining which types of monitoring data to collect for each of one or more microservices in the microservice chain based on whether the microservice was determined to be a critical microservice, generating an integrated microservice chain model for the microservice chain that comprises an indication of the microservice topology, an indication of the critical microservices, and an indication of the types of monitoring data to collect for each of the one or more microservices, and providing the integrated microservice chain model to a microservice tracker, wherein the microservice tracker is configured to collect, from the distributed mobile edge cloud while the microservice chain is being executed in the distributed mobile edge cloud, monitoring data for each of the one or more microservices corresponding to the types of monitoring data to collect for the microservice as indicated in the integrated microservice chain model and provide the collected monitoring data to a monitoring data analyzer that is to analyze the collected monitoring data for the one or more microservices, together with user equipment (UE) monitoring data and network monitoring data, to generate an analysis result.
[0007] An example embodiment is a set of non-transitory machine-readable media having computer code stored therein, which when executed by a set of one or more processors of one or more network devices, causes the one or more network devices to perform operations for providing tailored monitoring of microservice chain execution in a distributed mobile edge cloud. The operations include determining a microservice topology of a microservice chain, determining which microservices in the microservice chain are critical microservices based on the microservice topology, determining which types of monitoring data to collect for each of one or more microservices in the microservice chain based on whether the microservice was determined to be a critical microservice, generating an integrated microservice chain model for the microservice chain that comprises an indication of the microservice topology, an indication of the critical microservices, and an indication of the types of monitoring data to collect for each of the one or more microservices, and providing the integrated microservice chain model to a microservice tracker, wherein the microservice tracker is configured to collect, from the distributed mobile edge cloud while the microservice chain is being executed in the distributed mobile edge cloud, monitoring data for each of the one or more microservices corresponding to the types of monitoring data to collect for the microservice as indicated in the integrated microservice chain model and provide the collected monitoring data to a monitoring data analyzer that is to analyze the collected monitoring data for the one or more microservices, together with UE monitoring data and network monitoring data, to generate an analysis result.
[0008] An example embodiment is a method performed by one or more network devices to provide tailored monitoring of microservice chain execution in a distributed mobile edge cloud. The method includes obtaining an integrated microservice chain model for a microservice chain, wherein the integrated microservice chain model comprises an indication of a microservice topology of the microservice chain, an indication of which microservices in the microservice chain are critical microservices, and an indication of which types of monitoring data to collect for each of one or more microservices in the microservice chain, collecting, from the distributed mobile edge cloud while the microservice chain is being executed in in the distributed mobile edge cloud, monitoring data for each of the one or more microservices corresponding to the types of monitoring data to collect for the microservice as indicated in the integrated microservice chain model, and providing the collected monitoring data for the one or more microservices to a monitoring data analyzer, wherein the monitoring data analyzer is to analyze the collected monitoring data for the one or more microservices, together with UE monitoring data and network monitoring data, to generate an analysis result.
[0009] An example embodiment is a set of non-transitory machine-readable media having computer code stored therein, which when executed by a set of one or more processors of one or more network devices, causes the one or more network devices to perform operations for providing tailored monitoring of microservice chain execution in a distributed mobile edge cloud. The operations include obtaining an integrated microservice chain model for a microservice chain, wherein the integrated microservice chain model comprises an indication of a microservice topology of the microservice chain, an indication of which microservices in the microservice chain are critical microservices, and an indication of which types of monitoring data to collect for each of one or more microservices in the microservice chain, collecting, from the distributed mobile edge cloud while the microservice chain is being executed in the distributed mobile edge cloud, monitoring data for each of the one or more microservices corresponding to the types of monitoring data to collect for the microservice as indicated in the integrated microservice chain model, and providing the collected monitoring data for the one or more microservices to a monitoring data analyzer, wherein the monitoring data analyzer is to analyze the collected monitoring data for the one or more microservices, together with UE monitoring data and network monitoring data, to generate an analysis result. [0010] An advantage of embodiments disclosed herein is that they can monitor microservice chain execution in a tailored manner that allows for gaining valuable insights into the operation of the microservice chain execution while reducing the amount of monitoring data that is collected. Thereby, the embodiments herein help reduce the burden on the underlying infrastructure that has to collect the data and reduce the amount of data that has to be analyzed, thereby resulting in more efficient monitoring and analysis.
BRIEF DESCRIPTION OF THE DRAWINGS
[0011] The invention may best be understood by referring to the following description and accompanying drawings that are used to illustrate embodiments of the invention. In the drawings: [0012] Figure 1 is a diagram showing an environment in which tailored monitoring of microservice chain execution may be performed, according to some embodiments.
[0013] Figure 2 is a diagram showing component interactions for providing tailored monitoring of microservice chain execution in a distributed mobile edge cloud, according to some embodiments.
[0014] Figure 3 is a flow diagram showing a method for tailored monitoring of microservice execution in a distributed mobile edge cloud, according to some embodiments.
[0015] Figure 4 is a flow diagram showing a method for determining data item criticality, according to some embodiments.
[0016] Figure 5 is a diagram showing a microservice topology for a movie delivery service microservice chain, according to some embodiments.
[0017] Figure 6 is a diagram showing a microservice topology for a drone swarm microservice chain, according to some embodiments.
[0018] Figure 7 is a flow diagram showing a method for generating an integrated microservice chain model, according to some embodiments.
[0019] Figure 8 is a flow diagram showing a method for collecting monitoring data in a tailored manner, according to some embodiments.
[0020] Figure 9 is a flow diagram showing a method for analyzing monitoring data, according to some embodiments.
[0021] Figure 10A illustrates connectivity between network devices within an exemplary network, as well as three exemplary implementations of the network devices, according to some embodiments.
[0022] Figure 10B illustrates an exemplary way to implement a special-purpose network device, according to some embodiments. DETAILED DESCRIPTION
[0023] The following description describes methods and apparatus for tailored monitoring of microservice execution in a distributed mobile edge cloud. In the following description, numerous specific details such as logic implementations, opcodes, means to specify operands, resource partitioning/sharing/duplication implementations, types and interrelationships of system components, and logic partitioning/integration choices are set forth in order to provide a more thorough understanding of the present embodiments. It will be appreciated, however, by one skilled in the art that embodiments may be practiced without such specific details. In other instances, control structures, gate level circuits and full software instruction sequences have not been shown in detail in order not to obscure descriptions of the embodiments. Those of ordinary skill in the art, with the included descriptions, will be able to implement appropriate functionality without undue experimentation.
[0024] References in the specification to “one embodiment,” “an embodiment,” “an example embodiment,” etc., indicate that the embodiment described may include a particular feature, structure, or characteristic, but every embodiment may not necessarily include the particular feature, structure, or characteristic. Moreover, such phrases are not necessarily referring to the same embodiment. Further, when a particular feature, structure, or characteristic is described in connection with an embodiment, it is submitted that it is within the knowledge of one skilled in the art to affect such feature, structure, or characteristic in connection with other embodiments whether or not explicitly described.
[0025] Bracketed text and blocks with dashed borders (e.g., large dashes, small dashes, dotdash, and dots) may be used herein to illustrate optional operations that add additional features to embodiments. However, such notation should not be taken to mean that these are the only options or optional operations, and/or that blocks with solid borders are not optional in certain embodiments.
[0026] In the following description and claims, the terms “coupled” and “connected,” along with their derivatives, may be used. It should be understood that these terms are not intended as synonyms for each other. “Coupled” is used to indicate that two or more elements, which may or may not be in direct physical or electrical contact with each other, co-operate or interact with each other. “Connected” is used to indicate the establishment of communication between two or more elements that are coupled with each other.
[0027] An electronic device stores and transmits (internally and/or with other electronic devices over a network) code (which is composed of software instructions and which is sometimes referred to as computer program code or a computer program) and/or data using machine-readable media (also called computer-readable media), such as machine-readable storage media (e.g., magnetic disks, optical disks, solid state drives, read only memory (ROM), flash memory devices, phase change memory) and machine-readable transmission media (also called a carrier) (e.g., electrical, optical, radio, acoustical or other form of propagated signals - such as carrier waves, infrared signals). Thus, an electronic device (e.g., a computer) includes hardware and software, such as a set of one or more processors (e.g., wherein a processor is a microprocessor, controller, microcontroller, central processing unit, digital signal processor, application-specific integrated circuit, field-programmable gate array, other electronic circuitry, a combination of one or more of the preceding) coupled to one or more machine-readable storage media to store code for execution on the set of processors and/or to store data. For instance, an electronic device may include nonvolatile memory containing the code since the non-volatile memory can persist code/data even when the electronic device is turned off (when power is removed), and while the electronic device is turned on that part of the code that is to be executed by the processor(s) of that electronic device is typically copied from the slower non-volatile memory into volatile memory (e.g., dynamic random access memory (DRAM), static random access memory (SRAM)) of that electronic device. Typical electronic devices also include a set of one or more physical network interface(s) (NI(s)) to establish network connections (to transmit and/or receive code and/or data using propagating signals) with other electronic devices. For example, the set of physical NIs (or the set of physical NI(s) in combination with the set of processors executing code) may perform any formatting, coding, or translating to allow the electronic device to send and receive data whether over a wired and/or a wireless connection. In some embodiments, a physical NI may comprise radio circuitry capable of receiving data from other electronic devices over a wireless connection and/or sending data out to other devices via a wireless connection. This radio circuitry may include transmitter(s), receiver(s), and/or trans ceiver(s) suitable for radiofrequency communication. The radio circuitry may convert digital data into a radio signal with appropriate parameters (e.g., frequency, timing, channel, bandwidth, etc.). The radio signal may then be transmitted via antennas to the appropriate recipient(s). In some embodiments, the set of physical NI(s) may comprise network interface controller(s) (NICs), also known as a network interface card, network adapter, or local area network (LAN) adapter. The NIC(s) may facilitate in connecting the electronic device to other electronic devices allowing them to communicate via wire through plugging in a cable to a physical port connected to a NIC. One or more parts of an embodiment may be implemented using different combinations of software, firmware, and/or hardware.
[0028] A network device (ND) is an electronic device that communicatively interconnects other electronic devices on the network (e.g., other network devices, end-user devices). Some network devices are “multiple services network devices” that provide support for multiple networking functions (e.g., routing, bridging, switching, Layer 2 aggregation, session border control, Quality of Service, and/or subscriber management), and/or provide support for multiple application services (e.g., data, voice, and video).
[0029] As mentioned above, the amount of possible monitoring data that can be collected while microservices are being executed in the distributed mobile edge cloud and their data usage can be huge, which creates the danger of being flooded by too much irrelevant information. This makes it difficult for distributed mobile edge cloud operators (or other personnel or systems that are responsible for monitoring the health of microservice chain execution in the distributed mobile edge cloud) to identify any problems or issues in microservice chain execution quickly.
[0030] As such, there is a need for tailored monitoring of microservice chain execution that is specifically targeted at the relevant microservices that are executing across the distributed mobile edge cloud.
[0031] The term “specifically targeted” can be illustrated via the following example. Consider a microservice chain for a movie delivery service comprising the following microservices: movie selection, movie encoding, movie streaming, view reviews, and enter review. The most critical microservices in this microservice chain are movie selection, movie encoding, and movie streaming. As such, it is important that these microservices be specifically targeted for closer monitoring, especially since movie streaming is a highly data-intensive microservice and can have a significant impact on the end-user experience.
[0032] The monitoring data collected for such monitoring purposes would provide useful information regarding how well (or not) the microservice is performing, and could help identify where something may be going wrong. Identifying the critical microservices and focusing the monitoring primarily on them (and collecting less monitoring data for the non-critical microservices) may help reduce the overall amount of monitoring data collected, which may help reduce the burden on the distributed mobile edge cloud operators whose task is to analyze the collected monitoring data and draw inferences from it.
[0033] For example, if a one-hour movie takes more than one hour to stream, this information may be relayed to the operator via monitoring data that indicates the start time (the time when the movie stream began), the intermediate times when the movie stream was interrupted and then resumed, and finally the end time (the time when the movie stream ended). Such monitoring data may provide a good indication of where the movie stream was interrupted or was buffering, thereby causing delays to the viewer. Other relevant monitoring data may also be collected such as monitoring data indicative of the underlying network conditions (that monitor network congestion, if any), monitoring data indicative of user equipment (UE) mobility between edge clouds (the UE may move between the edge clouds, possibly triggering migration of the movie streaming microservice, resulting in possible interruption of service), and/or monitoring data indicative of faults/failures/maintenance of the underlying infrastructure which may have caused a temporary interruption of service.
[0034] The aforementioned monitoring data, together with a system that generates appropriate inferences from the monitoring data, may provide the operator with a clearer picture of how efficiently or inefficiently (or not) the microservice chain is performing. The monitoring data, together with the system helps determine the reasons that caused quality degradation. For example, the difference between the actual length of time that the movie streamed for and the one- hour movie length may be correlated against the other collected monitoring data to arrive at a determination of what may have gone wrong and where.
[0035] Such an approach may be implemented offline (e.g., based on analyzing logs) or online (e.g., by an alerting system that generates alerts when it detects that something may be going wrong). Offline analysis may be particularly useful when there is a need to analyze the overall system as a whole (e.g., analyze multiple instances of the movie delivery service microservice chain).
[0036] Monitoring microservices executing in a distributed mobile edge cloud poses unique challenges compared to monitoring microservices executing in a traditional cloud environment. Unlike traditional cloud environments, edge sites are typically more resource-constrained. That is, they typically have limited network, compute, and storage capacities. Hence there is an even greater need to reduce the amount of monitoring data collected for microservices executing in a distributed mobile edge cloud to ensure that the amount of collected monitoring data does not overwhelm the edge sites. Also, unlike traditional cloud applications, edge-based microservices are mobile (they can be migrated between edge sites - they are designed to be “edge-native” to allow for easy migration between edge sites). As such, the monitoring infrastructure applicable to traditional cloud-based applications may not apply to edge-based microservices.
[0037] Currently, tools exist for collecting telemetry data from distributed software, providing edge-based monitoring, and agentless distributed monitoring for microservices. However, while existing tools provide a way to collect data from a distributed mobile edge cloud, they do not prescribe an automated and tailored approach that minimizes or reduces the amount of data collected while still allowing for effective/useful analysis.
[0038] Embodiments disclosed herein provide tailored monitoring of microservice chain execution in a distributed mobile edge cloud. An embodiment is a method performed by one or more network devices to provide tailored monitoring of microservice chain execution in a distributed mobile edge cloud. The method includes determining a microservice topology of a microservice chain, determining which microservices in the microservice chain are critical microservices based on the microservice topology, determining which types of monitoring data to collect for each of one or more microservices in the microservice chain based on whether the microservice was determined to be a critical microservice, generating an integrated microservice chain model for the microservice chain that comprises an indication of the microservice topology, an indication of the critical microservices, and an indication of the types of monitoring data to collect for each of the one or more microservices, and providing the integrated microservice chain model to a microservice tracker, wherein the microservice tracker is configured to collect, from the distributed mobile edge cloud while the microservice chain is being executed in the distributed mobile edge cloud, monitoring data for each of the one or more microservices corresponding to the types of monitoring data to collect for the microservice as indicated in the integrated microservice chain model and provide the collected monitoring data to a monitoring data analyzer that is to analyze the collected monitoring data for the one or more microservices, together with user equipment (UE) monitoring data and network monitoring data, to generate an analysis result. [0039] The emergence of fifth generation (5G) networks (and later, sixth generation (6G) networks) is resulting in the disaggregation of monolithic applications into chains of microservices that are executed in a distributed manner across edge sites. Given the expected size, scope, and dynamism of 5G networks (and later, 6G networks) based microservice solutions, it is important that microservice chain execution be efficiently monitored in order to discover any problems/issues as soon as possible so that they can be resolved quickly. This should be done without overwhelming the underlying infrastructure with too much data collection, while still collecting the relevant information needed to detect, diagnose, and resolve any problems/issues in the microservice chain execution. Embodiments are disclosed herein that may facilitate the efficient monitoring of microservice chain execution in a distributed mobile edge cloud via tailored monitoring. Embodiments are now described with reference to the accompanying figures. [0040] Figure 1 is a diagram showing an environment in which tailored monitoring of microservice chain execution may be performed, according to some embodiments. As shown in the diagram, the environment includes an operator 120, a tailored monitoring system 100, and a mobile network 110. The mobile network 110 may be a communication network where the link to and from the end nodes (e.g., UEs) is wireless. In an embodiment, the mobile network 110 may be a 5G mobile network or similar network. The mobile network 110 may implement a type of mobile edge computing (also referred to as multi-access edge computing) that provides execution resources (e.g., compute and storage resources) for applications with networking close to the end users. In this regard, the mobile network 110 may include a distributed mobile edge cloud (comprising multiple edge clouds distributed across multiple geographically dispersed edge sites), which may execute microservice chains. The mobile network 110 may facilitate communication between UEs and microservice chains being executed in the distributed mobile edge cloud. While a mobile network 110 is shown in the diagram to help illustrate an embodiment, embodiments may also work with other types of (non-mobile) networks that are able to execute microservice chains across multiple edge sites.
[0041] As shown in the diagram, the tailored monitoring system 100 includes a microservice chain modeler 130, a microservice tracker 140, a UE tracker 160, a network tracker 170, and a monitoring data analyzer 180. In an embodiment, the tailored monitoring system 100 also includes a microservice instance manager 150. The components of the tailored monitoring system 100 may be implemented by one or more network devices. Although the diagram shows a single tailored monitoring system 100 (single monolithic architecture), this is only for ease of explanation. It should be understood that there can be multiple installations of the tailored monitoring system 100 (e.g., that are geographically distributed), through which monitoring data can be collected and provided to the operator 120.
[0042] The operator 120 may be responsible for operating and managing the mobile network 110. The operator 120 may be implemented by one or more network devices that are being controlled by a human user (e.g., a human user responsible for managing the mobile network 110 or aspects thereol). The operator 120 may use the microservice chain modeler 130 to create an integrated microservice chain model. A microservice chain is a composition of microservices, which can execute sequentially and/or in parallel. An integrated microservice chain model may augment a conventional microservice chain model (e.g., which typically only includes basic information about the microservice chain such as microservice topology information) with various other information regarding a microservice chain that can be used by the tailored monitoring system 100 to perform tailored monitoring of microservice chain execution. For example, the integrated microservice chain model for a microservice chain may include an indication of the microservice topology of the microservice chain, an indication of which microservices in the microservice chain are critical microservices, an indication of the types of monitoring data to collect for different microservices in the microservice chain, an indication of the input and output data sets of different microservices in the microservice chain, an indication of which data items used by the microservices in the microservice chain are critical data items, an indication of data item dependencies between data items used by microservices in the microservice chain, and/or an indication of the monitoring intervals for different data items used by microservices in the microservice chain. Some or all of the information included in the integrated microservice chain model may be provided/generated by the operator 120 and/or the microservice chain modeler 130. [0043] The input data set of a microservice may be the set of data items that are provided as an input to the microservice, while the output data set of a microservice may be the set of data items that are provided as an output of the microservice. A data item may be a piece of data that is used by microservices (e.g., as an input or an output of a microservice).
[0044] The critical microservices are the microservices that are deemed more important to monitor more closely. Similarly, the critical data items are the data items that are deemed more important to monitor more closely.
[0045] The data item dependencies indicate dependency relationships between different data items. A data item dependency may be expressed in the form Di -> Dk, which indicates that the data item Di (which could be included in an input data set or output data set of a microservice) is dependent upon the data item Dk (which could also be included in an input data set or output data set of a microservice). A group of data item dependencies may form a data item dependency chain/graph.
[0046] As will be further described herein, in an embodiment, more types of monitoring data are collected for microservices that are determined to be critical (compared to other non-critical microservices). In an embodiment, the types of monitoring data to collect for a microservice may include one or more of: the time at which the microservice began execution, the time at which the microservice completed execution, the edge site location at which the microservice began execution, the times at which the microservice changed state, the edge site locations at which the microservice changed state, the times at which the microservice resumed execution, the edge site locations at which the microservice resumed execution, and the edge site location at which the microservice completed execution. The preceding list is provided purely by way of example to illustrate an embodiment. It should be understood that other embodiments may use different types of monitoring data from what is listed here.
[0047] The monitoring intervals for a data item indicate the points at which monitoring data for the data item is to be collected. In one embodiment, the monitoring intervals for a data item include one or more of: the times at which the data item is taken as an input of a microservice, the times at which the data item is provided as an output of a microservice, and the times at which the data item is updated by a microservice (e.g., after being taken as an input of a microservice and before being provided as an output of the microservice). As will be further described herein, in an embodiment, monitoring data for data items that are determined to be critical is collected with more granularity (collected more frequently and/or at more points of operation compared to other data items).
[0048] An example representation of an integrated microservice chain model is provided below. Microservice Chain C: Topology based on Microservice Tuple: {Si} /* each Si is a microservice; S(i+ 1) is a successor of Si */
Si: {Sj } /* each Sj is a successor of Si - please note that any Si can have more than one successor */
For each Si, for monitoring: monitoring data
Edge_site_location /*location of edge site where it is running */
Start time
End time
{Interruption_time} /* set of timestamps when the microservice execution was interrupted */ {Resumption_time} /* set of timestamps when the microservice execution was resumed */
Data considerations for each service Si:
Si: {Din, Dout} /* Din and Dout are the input and output data sets, respectively - this is the actual application data needed for the microservice to function */
Criticality for each data item that forms part of Din or Dout:
Dj : Boolean /* Dj is a data item that belongs to Din or Dout. Lj is a boolean that is either “true” or “false”, depending on whether the data is considered critical or not; hence non-critical data need not be monitored */
Data Item Dependencies:
Dj -> Dk /* data item Dj is dependent on Dk, hence monitoring of Dj would be incomplete without monitoring of Dk along with it */
[0049] The example representation provided above indicates the microservice topology of the microservice chain C (in set notation), the types of monitoring data to collect for each microservice Si (e.g., edge site location, start time, end time, interruption times, and resumption times), the input data set (Din) and output data set (Dout) of each microservice Si, the criticality of each data item (Dj), and the data item dependencies (expressed in the form Dj -> Dk).
[0050] In an embodiment, the determination of which microservices in the microservice chain are critical is made based on identifying the critical path in the microservice chain. Any microservices that are on the critical path may then be labeled as critical microservices. In an embodiment, any data items that are in the input data sets or the output data sets of one of the critical microservices are determined to be critical data items. In an embodiment, data items that the critical data items are dependent upon are determined based on the data item dependency information. As will be further described herein, critical microservices and critical data items, as well as any data items that the critical data items are dependent upon, may be more closely monitored than other microservices and data items.
[0051] The types of monitoring data to collect for a microservice may be determined based on a variety of factors including whether the microservice has determined to be critical or not. In general, the types of monitoring data to collect for a critical microservice includes more types compared to that of other (non-critical) microservices. For example, the types of monitoring data to collect for a non-critical microservice may only include the time at which the microservice began execution and the time at which the microservice completed execution. In contrast, the types of monitoring data to collect for a critical microservice may include the above types and also include the edge site location at which the microservice began execution, the time instance at which the microservice changed state, the edge site locations at which the microservice changed state, the time instance at which the microservice resumed execution, the edge site locations at which the microservice resumed execution, and/or the edge site location at which the microservice completed execution.
[0052] The monitoring intervals for a data item may be determined based on a variety of factors including whether the data item has been determined to be critical or not and whether the data item is one that a critical data item is dependent upon. In general, the monitoring intervals for a critical data item or a data item that a critical data item is dependent upon may be more granular compared to the monitoring intervals for other data items. For example, the monitoring intervals for a data item that has determined not to be critical and not to be a data item that a critical data item is dependent upon may only include the times at which the data item is taken as an input of a microservice and the times at which the data item is provided as an output of a microservice. In contrast, the monitoring intervals for a critical data item or a data item that a critical data item is dependent upon may include the above monitoring intervals and also include the times at which the data item is updated by a microservice.
[0053] Selecting monitoring intervals that are too granular may flood the system with too much monitoring data, while selecting monitoring intervals that are too coarse may result in not collecting enough monitoring data to perform a useful analysis. If the operator 120 wishes to only know whether a microservice is working as designed or not, then it may be sufficient to only monitor when a data item is taken as input to the microservice and when a data item is provided as an output of the microservice. Otherwise, if the operator 120 wishes to know how well the microservice is performing, then it may be useful to monitor whenever a data item gets transformed within the microservice. The granularity of monitoring intervals may be configurable depending on the requirements of the operator 120 in terms of how much monitoring data they wish to collect in order to analyze how well their microservice chain is performing. In other words, the operator 120 may determine the monitoring intervals based on achieving a balance between monitoring accuracy and the amount of monitoring data being collected.
[0054] Thus, the operator 120 and the microservice chain modeler 130 may work in conjunction to generate an integrated microservice chain model for a particular microservice chain. In an embodiment, the microservice chain modeler 130 sends the integrated microservice chain model to the microservice instance manager 150, which may instantiate the microservice chain in the distributed mobile edge cloud of the mobile network 110 based on the integrated microservice chain model.
[0055] The microservice chain modeler 130 may send the integrated microservice chain model to the microservice tracker 140. The microservice tracker 140 may collect (from the distributed mobile edge cloud) monitoring data for each of one or more microservices in the microservice chain corresponding to the types of monitoring data to collect for the microservice as indicated in the integrated microservice chain model. This may involve, for example, collecting more types of monitoring data for critical microservices compared to other non-critical microservices. In an embodiment, the microservice tracker 140 also collects (from the distributed mobile edge cloud) monitoring data for each of one or more data items used by microservices in the microservice chain according to the monitoring intervals for the data item as indicated in the integrated microservice chain model. This may involve, for example, collecting monitoring data for critical data items (and data items that the critical data items are dependent upon) with more granularity compared to other data items. The monitoring data for the microservices and the monitoring data for the data items may be collectively referred to herein as microservice monitoring data. The microservice tracker 140 may collect the microservice monitoring data from the distributed mobile edge cloud with the help of mobile network 110. The microservice tracker 140 may send the collected microservice monitoring data to the monitoring data analyzer 180.
[0056] The microservice chain modeler 130 may send instantiated microservice chain information to the UE tracker 160. The instantiated microservice chain information may include information about the instances of a microservice chain that are being executed in the distributed mobile edge cloud of the mobile network. This information may allow the UE tracker 160 to be able to track any UEs that are using an instance of the microservice chain.
[0057] The UE tracker 160 may track any UEs that are using an instance of the microservice chain and collect monitoring data for those UEs. For example, the UE tracker 160 may collect monitoring data related to the location and movement of any UEs using an instance of the microservice chain. The monitoring data collected by the UE tracker 160 may be referred to herein as UE monitoring data. The UE tracker 160 may collect the UE monitoring data with the help of components of the mobile network 110. The UE tracker 160 may send the collected UE monitoring data to the monitoring data analyzer 180.
[0058] The network tracker 170 may collect monitoring data related to the mobile network 110. For example, the network tracker 170 may collect monitoring data related to latency of communication between a UE and a microservice in the mobile network 110, latency of communication between microservices executing in the mobile network 110, and/or faults occurring at edge sites or other parts of the mobile network 110. The monitoring data collected by the network tracker 170 may be referred to herein as network monitoring data. The network tracker 170 may collect the network monitoring data with the help of components of the mobile network 110 such as a network exposure function (NEF) of the mobile network 110 (e.g., which can provide a means to securely expose the services and capabilities provided by network functions). Other components of the mobile network 110 that may participate in collecting/providing network monitoring data may include, for example, a network repository function (NRF) (e.g., which can maintain a record of 5G elements that are available in the mobile network 110 and their supported services; it may allow other network function instances to subscribe and be notified of registrations from network function instances of a given type), a network slice selection function (NSSF) (e.g., which can select network slices within which microservices can be implemented), unified data management (UDM) and unified data repository (UDR) (e.g., which can manage network user data in a single/centralized element; with the help of UDR, it may store data such as customer profile information, customer authentication information, and encryption of keys for the information), and/or a policy control function (PCF) (e.g., which can support a unified policy framework within the 5G infrastructure for governing network behavior; the PCF may access the subscription information required to make policy decisions from the UDM and provide the appropriate policy rules to the control plane). The network tracker 170 may send the collected network monitoring data to the monitoring data analyzer 180.
[0059] Thus, the monitoring data analyzer 180 may obtain microservice monitoring data from the microservice tracker 140, UE monitoring data from UE tracker 160, and network monitoring data from network tracker 170. The monitoring data analyzer 180 may analyze the obtained data to generate an analysis result. The monitoring data analyzer 180 may use known data analysis techniques to analyze the obtained monitoring data. For example, the monitoring data analyzer 180 may use machine learning algorithms to determine correlations between monitored data and identify potential problems/issues with the microservice chain execution (e.g., performance issues) and possibly identify root causes of those problems/issues. The analysis can be performed in real-time (e.g., as monitoring data is being collected) and/or offline (e.g., after the monitoring data has been collected and stored in a storage medium (e.g., a log)) depending on the needs of the operator 120. The monitoring data analyzer 180 may send the analysis result to the operator 120. The monitoring data analyzer 180 may be implemented centrally or in a distributed manner. [0060] Embodiments may thus monitor microservice chain execution in a tailored manner that allows for gaining valuable insights into the operation of the microservice chain execution, while reducing the amount of monitoring data that is collected, which helps reduce the burden on the underlying infrastructure that has to collect the data and reduces the amount of data that has to be analyzed, thereby resulting in more efficient monitoring and analysis.
[0061] Figure 2 is a diagram showing component interactions for providing tailored monitoring of microservice chain execution in a distributed mobile edge cloud, according to some embodiments.
[0062] As shown in the diagram, the operator 120 may send a request to the microservice chain modeler 130 to create an integrated microservice chain model for a specified microservice chain. As part of this request or in conjunction with this request, the operator 120 may also send a request to the microservice chain modeler 130 to determine the critical microservices in the microservice chain and/or determine the critical data items used by the microservices in the microservice chain, as well as data items that the critical data items are dependent upon. The microservice chain modeler 130 may send corresponding responses to the requests to the operator 120.
[0063] The microservice chain modeler 130 may then send a request to the microservice instance manager 150 to instantiate the microservice in the distributed mobile edge cloud. In response, the microservice instance manager 150 may instantiate the microservice in the distributed mobile edge cloud and send a corresponding response to the microservice chain modeler 130 (e.g., indicating that the microservice chain was successfully instantiated).
[0064] At this stage, all of the critical information needed for tailored monitoring of the execution of the microservice chain has been generated and may be included in an integrated microservice chain model.
[0065] The microservice chain modeler 130 may send information about the instantiated microservice chain (and information about UEs using the instantiated microservice chain in some embodiments) to the UE tracker 160 and send the integrated microservice chain model to the microservice tracker 140.
[0066] The UE tracker 160 may collect UE monitoring data from the mobile network 110 (e.g., monitoring data related to the locations of UEs that use an instance of the microservice chain) and send the collected UE monitoring data to the monitoring data analyzer 180.
[0067] The microservice tracker 140 may collect microservice monitoring data from the distributed mobile edge cloud of the mobile network 110 (in a tailored manner in accordance with the integrated microservice chain model) and send the collected microservice monitoring data to the monitoring data analyzer 180.
[0068] The network tracker 170 may collect network monitoring data from the mobile network 110 (e.g., monitoring data related to network performance related to the microservice chain execution) and send the collected network monitoring data to the monitoring data analyzer 180.
[0069] The monitoring data analyzer 180 may analyze the microservice monitoring data, the UE monitoring data, and the network monitoring data to generate an analysis result (e.g., identifying potential problems/issues in the microservice chain execution and possible root causes). The monitoring data analyzer 180 may then send the analysis result to the operator 120.
[0070] A brief description of the various components according to an embodiment is provided below in terms of their inputs, outputs, and actions. It should be understood that these descriptions are provided purely as an example to help illustrate an embodiment. It should be understood that the components may have different functionality from what is described here. Also, while a particular division of functionality and arrangement of components are shown in the diagrams, it should be understood that other embodiments may have a different division of functionality and/or a different arrangement of components (e.g., the functionality of multiple components may be combined or the functionality of a single component may be divided among multiple components). [0071] The operator 120 manages and uses the tailored monitoring system 100. The input of the operator 120 may include instructions from end users regarding which microservice chain(s) to monitor. The output of the operator 120 may include a conventional (non-integrated) microservice chain model, an indication of the critical microservices in the microservice chain, and an indication of the critical data items used by the microservices in the microservice chain, along with dependencies thereof. The actions of the operator 120 may include sending messages to the microservice chain modeler 130 asking it to create and maintain the integrated microservice chain model (including an indication of the determined critical microservices) and their data including dependencies thereof. It should be noted that some of the operations of the operator 120 may be performed by a human or by an automated system that is controlled by a human user (e.g., the operator 120 may be an automated agent that is being controlled by a human user).
[0072] The microservice chain modeler 130 is responsible for creating and maintaining the integrated microservice chain model. The input of the microservice chain modeler 130 may include the microservice chain model, an indication of the critical microservices in the microservice chain, and an indication of the critical data items used by the microservices in the microservice chain along with dependencies thereof. The output of the microservice chain modeler 130 may include a microservice instantiation request that is to be sent to the microservice instance manager 150 and integrated UE and microservice instance information that is to be sent to the UE tracker 160 (this may help the UE tracker 160 track UEs that are using an instance of the microservice chain (it should be noted that a UE may be mobile and not fixed to a particular location)). The actions of the microservice chain modeler 130 may include creating the integrated microservice chain model and sending a request to the microservice instance manager 150 to instantiate the microservice chain. The actions of the microservice chain modeler 130 may also include sending the microservice instance information to the UE tracker 160 so that the UE tracker 160 can track the UEs that are using an instance of the microservice chain.
[0073] The microservice instance manager 150 is responsible for instantiating the microservice chain per requests/instructions received from the microservice chain modeler 130 and ensuring that it is running correctly in the distributed mobile edge cloud of the mobile network 110. The input of the microservice instance manager 150 may include the instantiation request received from the microservice chain modeler 130. The output of the microservice instance manager 150 may include an instantiation confirmation response to be sent to the microservice chain modeler 130. The actions of the microservice instance manager 150 may include instantiating the microservice chain and sending the instantiation confirmation response to the microservice chain modeler 130.
[0074] The mobile network 110 may be responsible for providing various monitoring data related to microservice chain execution to the microservice tracker 140, the UE tracker 160, and the network tracker 170. The input of the mobile network 110 may include monitoring data received from underlying network components. The output of the mobile network 110 may include (tailored) microservice monitoring data, UE monitoring data, and network monitoring data. The actions of the mobile network 110 may include providing the microservice monitoring data, UE monitoring data, and network monitoring data to the microservice tracker 140, the UE tracker 160, and the network tracker 170, respectively.
[0075] The microservice tracker 140 is responsible for collecting microservice monitoring data. The input of the microservice tracker 140 may include microservice monitoring data collected from the mobile network 110 according to the integrated microservice chain model. The output of the microservice tracker 140 may include the collected microservice monitoring data. The actions of the microservice tracker 140 may include tracking microservice chain execution (e.g., tracking microservice instances being executed in the distributed mobile edge cloud with the help of mobile network components), collecting relevant microservice monitoring data according to the integrated microservice chain model, and sending collected microservice monitoring data to the monitoring data analyzer 180. [0076] The UE tracker 160 is responsible for collecting UE monitoring data for UEs that use an instance of the microservice chain. The input of the UE tracker 160 may include UE monitoring data collected from the mobile network 110. The output of the UE tracker 160 may include UE monitoring data to be sent to the monitoring data analyzer 180. The actions of the UE tracker 160 may include tracking relevant UE actions/movements with the help of mobile network components, collecting relevant UE monitoring data, and sending collected UE monitoring data to the monitoring data analyzer 180.
[0077] The network tracker 170 is responsible for collecting network monitoring data. The input of the network tracker 170 may include network monitoring data collected from the mobile network 110. The output of the network tracker 170 may include network monitoring data to be sent to the monitoring data analyzer 180. The actions of the network tracker 170 may include collecting relevant network monitoring data and sending it to the monitoring data analyzer 180.
[0078] The monitoring data analyzer 180 is responsible for analyzing various monitoring data to generate an analysis result. The input of the monitoring data analyzer 180 may include microservice monitoring data, UE monitoring data, and network monitoring data. The output of the monitoring data analyzer may include an analysis result (e.g., synthesized/analyzed monitoring information) to be sent to the operator 120. The actions of the monitoring data analyzer 180 may include obtaining microservice monitoring data, UE monitoring data, and network monitoring data and synthesizing/analyzing the obtained monitoring data to generate an analysis result and sending the analysis result to the operator 120.
[0079] Figure 3 is a flow diagram showing a method for tailored monitoring of microservice execution in a distributed mobile edge cloud, according to some embodiments. The method may be performed by one or more network devices (e.g., implementing a tailored monitoring system). [0080] The operations in the flow diagrams will be described with reference to the exemplary embodiments of the other figures. However, it should be understood that the operations of the flow diagrams can be performed by embodiments other than those discussed with reference to the other figures, and the embodiments discussed with reference to these other figures can perform operations different than those discussed with reference to the flow diagrams.
[0081] At block 310, the one or more network devices generate an integrated microservice chain model for a microservice chain.
[0082] At block 320, the one or more network devices determine which microservices in the microservice chain are critical microservices.
[0083] At block 330, the one or more network devices determine which data items used by the microservices in the microservice chain are critical data items or data items that the critical items are dependent upon. [0084] At block 340, the one or more network devices collect more types of monitoring data for the critical microservices (compared to non-critical microservices) and collect monitoring data for critical data items and data items that the critical items are dependent upon with more granularity (compared to other data items).
[0085] At block 350, the one or more network devices analyze the collected monitoring data (e.g., offline or online (in real-time)) to generate an analysis result.
[0086] Figure 4 is a flow diagram showing a method for determining data item criticality, according to some embodiments. The method may be performed by one or more network devices (e.g., implementing an operator 120 and/or a microservice chain modeler 130).
[0087] At block 410, the one or more network devices model a microservice chain as a workflow.
[0088] At block 420, the one or more network devices label all input and output data items for each microservice in the microservice chain as being non-critical data items (this is an initialization step).
[0089] At block 430, the one or more network devices identify a critical path of the workflow. Critical path determination can be performed using any suitable technique. In an embodiment, the critical path is the longest path of the workflow between the start activity and the end activity (e.g., the longest path between the beginning of the microservice chain and the end of the microservice chain).
[0090] At block 440, the one or more network devices determine which microservices are on the critical path.
[0091] At block 450, the one or more network devices label any data items that are in the input data set or output data set of a microservice on the critical path as being critical data items.
[0092] Figure 5 is a diagram showing a microservice topology for a movie delivery service microservice chain, according to some embodiments. As shown by the topology, the movie delivery service microservice chain includes a movie selection microservice (SI) 510, a movie encoding microservice (S2) 520, a movie streaming microservice (S3) 530, a view reviews microservice (S4) 540, and an enter review microservice (S5) 550. The latter three microservices are implemented in parallel. It is assumed in this example that the duration of the movie being streamed is one hour.
[0093] The microservice topology of the movie delivery service microservice chain may be represented as: {SI, {S2, S4, S5}, S3}. The input data set and output data set of each microservice may be as follows:
SI Inputs: User input
Outputs: Movie lD, Raw Movie File
S2
Inputs: Movie lD, Raw Movie File
Outputs: Movie lD, Encoded Movie File
S3:
Inputs: Movie lD, Encoded Movie File Outputs: Movie lD, Movie Stream
S4
Inputs: Movie lD
Outputs: Movie Review List
S5
Inputs: Movie lD
Outputs: User Review
[0094] A critical path identification method may determine that the movie selection 510, movie encoding 520 and movie streaming 530 microservices are critical microservices since they are on the critical path (the longest path in the microservice chain between “begin” and “end” (the path SI -> S2 -> S3)).
[0095] Since these three microservices are determined to be critical microservices, the following types of monitoring data may be collected for these microservices: starting time (the time at which the microservice began execution), the edge site location at which the microservice began execution, the times at which the microservice changed state (e.g., it was migrated or suspended), the edge site locations at which the microservice changed state, the times at which the microservice resumed execution, the edge site locations at which the microservice resumed execution, ending time (the time at which the microservice completed execution), and the edge site location at which the microservice completed execution.
[0096] For the other two non-critical microservices (view reviews 540 and enter review 550 microservices), only the following types of monitoring data may be collected: starting time and ending time.
[0097] The microservice tracker 140 may collect the above types of monitoring data for the respective microservices from the distributed mobile edge cloud. In addition, the relevant network monitoring data and UE monitoring data may be collected for the three critical microservices. This monitoring data may be collected in an automated manner by the mobile network 110 (e.g., using known monitoring techniques that are used by mobile network operators to monitor the operations of their networks at the network layer).
[0098] The network monitoring data (which may be collected by the network tracker 170) may include network latency related data associated with the critical microservices (e.g., at different points of operation mentioned above such as when the microservice began execution, when the microservice changed state, when the microservice resumed execution, and when the microservice completed execution). As an example, the network latency related data may include the latency of interaction of a UE with each critical microservice and the latency of inter-microservice interactions (e.g., latency between the movie selection microservice 510 and the movie encoding microservice 520 and latency between the movie encoding microservice 520 and the movie streaming microservice 530).
[0099] The UE monitoring data (which may be collected by the UE tracker 160) may include the UE location at different points of operation mentioned above (e.g., to which edge site location the UE is connected and/or to which user plane function (UPF) of the mobile network 110 the UE is connected).
[00100] It should be noted that the network monitoring data and UE monitoring data become dependent data for microservice execution time, as collecting just the microservice monitoring data alone may be insufficient without also collecting the network latency-related data and UE location related data. For example, a network latency longer than an expected length may be correlated against the overall duration of the movie stream to determine how much this extra network latency contributed to the overall time to stream the movie.
[00101] The above collected monitoring data may be used to analyze the performance of the one-hour movie. This analysis may be performed by the monitoring data analyzer 180.
[00102] Once the movie stream completes, if the microservice monitoring data shows that it took one hour and five minutes to stream the movie instead of just one hour, then various monitoring data may be correlated, and some conclusions can be reached. For example, if it assumed that the movie streaming was interrupted due to UE migration, this can be determined by correlating the times at which the microservice changed state and the UE locations at that time. If the correlation shows that the state/session change - interruption and resumption - coincide with UE location changes, then this related delay may be calculated. A variant of this could be autoscaling (where a new instance of the microservice is created at the new edge site location to which the UE has moved) but that may also be modeled as a combination of interruption and resumption. [00103] Also, in case that the end users themselves paused the movie in order to execute the view reviews 540 and the enter review 550 microservices, those times may also be determined from data collected regarding the start time and end time of these two microservices. Also, in case there were any network latency issues that caused buffering issues resulting in movie streaming delays, this may also be determined from the network monitoring data. Taken together, the reason for the extra five minutes of movie streaming time may be accounted for via collection/analysis of only the above types of monitoring data (and no others). In this way, tailored monitoring of microservice execution may be achieved based on an understanding of the nature of the constituent microservices.
[00104] It should be noted that the above analysis may also be done in real-time by the monitoring data analyzer 180 while the microservice chain is being executed (e.g., while the movie is still streaming). For example, if the movie has reached the time of 45:53 (45 minutes and 53 seconds) but 48 minutes have actually elapsed, a real-time analysis of the collected monitoring data may help to provide indications of where most of the extra two minutes and seven seconds may have gone, and help operators readjust the underlying infrastructure (whether network or UE migration related or edge cloud related) so that further delays can be avoided.
[00105] Figure 6 is a diagram showing a microservice topology for a drone swarm microservice chain, according to some embodiments.
[00106] This example features a collection (or swarm) of drones traveling together through a set of edge zones each managed by an edge site (represented as an edge router in the diagram). The edge routers are in turn connected to a central cloud. A critical path identification method may determine that image recognition (IR) and obstacle avoidance (OA) microservices belong to the critical path. In turn, these microservices may invoke other microservices (e.g., camera, location, speed, etc.). The actual paths of the drones may be determined via the ConstructRoute microservice running in the cloud.
[00107] IR and OA microservices are therefore offloaded from the drone to the nearest edge router (edge site) where they are executed. As the drone moves between edge sites, these microservices would also need to be migrated to the edge site under whose jurisdiction the drone will move.
[00108] This example has a few unique characteristics. The UEs in this case are the drones (can be assumed to be equipped with subscriber identification module (SIM) cards to help them communicate with edge sites), accessing the latency critical IR and OA microservices. Since the drones need to move to reach their destinations, this makes UE mobility across edge sites a given. There may be network congestion due to large numbers of drones accessing an edge site. As such, even if a drone is under the jurisdiction of, say, edge site El, its IR and/or OA microservices may have to be migrated to another edge site, say, E2, or to the cloud itself.
[00109] Edge sites can fail at any time. As such, if El mentioned above fails - or shows indications that it may fail soon as per data from suitable monitoring tools - then the IR and OA microservices running on it may have to be migrated to other edge sites. Not all of them can be migrated to the same edge site, hence they may have to be distributed among multiple edge sites and even the cloud until El starts functioning properly again.
[00110] The following types of monitoring data may be collected for the critical microservices IR and OA: starting time (the time at which the microservice began execution), the edge site location at which the microservice began execution, the times at which the microservice changed state (e.g., it was migrated or suspended), the times at which the microservice resumed execution, and ending time (the time at which the microservice completed execution).
[00111] For the other non-critical microservices, only the following types of monitoring data may be collected: starting time and ending time.
[00112] The microservice tracker 140 may collect the above types of monitoring data for the respective microservices from the distributed mobile edge cloud. In addition, the relevant network monitoring data and UE monitoring data may be collected for the three critical microservices.
[00113] The network monitoring data (which may be collected by the network tracker 170) may include network latency related data associated with the critical microservices (e.g., at different points of operation mentioned above such as when the microservice began execution, when the microservice changed state, when the microservice resumed execution, and when the microservice completed execution). The network monitoring data may also include data related to infrastructure faults/failure at any edge sites (if any) such as the time of fault, the time when microservices executed on the failed edge site had to be migrated to another edge site, and the time at which microservice execution was resumed.
[00114] The UE monitoring data (which may be collected by the UE tracker 160) may include the UE (e.g., drone) location at different points of operation mentioned above. Here too, as in the movie delivery service example, this data may be needed in order to complete the monitoring of the two critical microservices.
[00115] Assuming that IR and OA microservices are supposed to complete execution within a particular time (e.g., 50 milliseconds each), the actual time taken for each of these microservices to complete execution may then be correlated against both 50 milliseconds as well as the combination of network latency, UE location, and faults/failures if any. Similar to the movie delivery service example, this analysis can be performed by the monitoring data analyzer 180. [00116] This tailored approach to collecting monitoring data may provide the appropriate insights into the operation of the microservice chain execution, while reducing the amount of monitoring data that needs to be collected.
[00117] Figure 7 is a flow diagram showing a method for generating an integrated microservice chain model, according to some embodiments. The method may be implemented by one or more network devices (e.g., implementing the operator 120 and/or the microservice chain modeler 130). [00118] At block 710, the one or more network devices determine a microservice topology of a microservice chain.
[00119] At block 720, the one or more network devices determine which microservices in the microservice chain are critical microservices based on the microservice topology. In an embodiment, the determination of which microservices in the microservice chain are critical microservices is based on identifying a critical path in the microservice chain and identifying microservices on the critical path as being critical microservices.
[00120] At block 730, the one or more network devices determine which types of monitoring data to collect for each of one or more microservices in the microservice chain based on whether the microservice was determined to be a critical microservice. In an embodiment, the types of monitoring data to collect for a critical microservice comprises more types of monitoring data compared to the types of monitoring data to collect for a non-critical microservice. In an embodiment, the types of monitoring data to collect for the critical microservice and the types of monitoring data to collect for the non-critical microservice both comprise a time at which the microservice began execution and a time at which the microservice completed execution, wherein the types of monitoring data to collect for the critical microservice further comprises one or more other types of monitoring data that are not included in the types of monitoring data to collect for the non-critical microservice. In an embodiment, the one or more other types of monitoring data comprises one or more of: edge site location at which the microservice began execution, times at which the microservice changed state, edge site locations at which the microservice changed state, times at which the microservice resumed execution, edge site locations at which the microservice resumed execution, and edge site location at which the microservice completed execution.
[00121] At block 740, the one or more network devices generate an integrated microservice chain model for the microservice chain that comprises an indication of the microservice topology, an indication of the critical microservices, and an indication of the types of monitoring data to collect for each of the one or more microservices.
[00122] At block 750, the one or more network devices provide the integrated microservice chain model to a microservice tracker, wherein the microservice tracker is to collect, from the distributed mobile edge cloud while the microservice chain is being executed in the distributed mobile edge cloud, monitoring data for each of the one or more microservices corresponding to the types of monitoring data to collect for the microservice as indicated in the integrated microservice chain model and provide the collected monitoring data to a monitoring data analyzer that is to analyze the collected monitoring data for the one or more microservices, together with UE monitoring data and network monitoring data, to generate an analysis result.
[00123] In an embodiment, the one or more network devices also determine input and output data sets of each microservice in the microservice chain, determine which data items used by microservices in the microservice chain are critical data items, determine data item dependencies between data items used by the microservices in the microservice chain, determine, based on the data item dependencies, which data items used by the microservices in the microservice chain are data items that the critical data items are dependent upon, determine monitoring intervals for each of one or more data items used by the microservices in the microservice chain based on whether the data item was determined to be a critical data item or a data item that a critical item is dependent upon, and add, to the integrated microservice chain model, an indication of the input and output data sets of each microservice in the microservice chain, an indication of the critical data items, an indication of the data items that the critical data items are dependent upon, and an indication of the monitoring intervals for each of the one or more data items, wherein the microservice tracker is to further collect, from the distributed mobile edge cloud, monitoring data for each of the one or more data items according to the monitoring intervals for the data item as indicated in the integrated microservice chain model and provide the collected monitoring data for the one or more data items to the monitoring data analyzer.
[00124] In an embodiment, the determination of which data items used by the microservices in the microservice chain are critical data items is based on identifying data items in the input and output data sets of the critical microservices as being critical data items.
[00125] In an embodiment, the monitoring intervals for a critical data item or a data item that a critical data item depends on are more granular compared to the monitoring intervals for a non- critical data item. In an embodiment, the monitoring intervals for the critical data item and the monitoring intervals for the non-critical data items both comprise times at which the data item is taken as an input of a microservice and times at which the data item is provided as an output of a microservice, wherein the monitoring intervals for the critical data item further comprises one or more other monitoring intervals not included in the monitoring intervals for the non-critical data item. In an embodiment, the one or more other monitoring intervals comprise times at which the data item is updated by a microservice.
[00126] In an embodiment, the one or more network devices cause the microservice chain to be instantiated in the distributed mobile edge cloud and provide information regarding the instantiated microservice chain to a UE tracker, wherein the UE tracker is configured to use the information regarding the instantiated microservice chain to collect UE monitoring data from a mobile network for UEs that use the instantiated microservice chain.
[00127] Figure 8 is a flow diagram showing a method for collecting monitoring data in a tailored manner, according to some embodiments. The method may be implemented by one or more network devices (e.g., implementing the microservice tracker 140).
[00128] At block 810, the one or more network devices obtain an integrated microservice chain model for a microservice chain, wherein the integrated microservice chain model comprises an indication of a microservice topology of the microservice chain, an indication of which microservices in the microservice chain are critical microservices, and an indication of which types of monitoring data to collect for each of one or more microservices in the microservice chain.
[00129] At block 820, the one or more network devices collect, from the distributed mobile edge cloud while the microservice chain is being executed in in the distributed mobile edge cloud, monitoring data for each of the one or more microservices corresponding to the types of monitoring data to collect for the microservice as indicated in the integrated microservice chain model. In an embodiment, monitoring data collected for a critical microservice comprises more types of monitoring data compared to the monitoring data collected for a non-critical microservice. [00130] At block 830, the one or more network devices provide the collected monitoring data for the one or more microservices to a monitoring data analyzer, wherein the monitoring data analyzer is to analyze the collected monitoring data for the one or more microservices, together with user equipment monitoring data and network monitoring data, to generate an analysis result. In an embodiment, the analysis result is generated in real-time as monitoring data is being collected.
[00131] In an embodiment, the integrated microservice chain model further includes an indication of input and output data sets of each microservice in the microservice chain, an indication of critical data items used by microservices in the microservice chain, an indication of which data items used by the microservices in the microservice chain are data items that the critical data items are dependent upon, and an indication of monitoring intervals for each of one or more data items. In an embodiment, the one or more network devices collect, from the distributed mobile edge cloud, monitoring data for each of the one or more data items according to the monitoring intervals for the data item as indicated in the integrated microservice chain model and provide the collected monitoring data for the one or more data items to the monitoring data analyzer, wherein the monitoring data analyzer is to further analyze the collected monitoring data for the one or more data items to generate the analysis result. In an embodiment, the monitoring intervals for a critical data item are more granular compared to the monitoring intervals for a non- critical data item.
[00132] Figure 9 is a flow diagram showing a method for analyzing monitoring data, according to some embodiments. The method may be implemented by one or more network devices (e.g., implementing the monitoring data analyzer 180).
[00133] At block 910, the one or more network devices obtain microservice monitoring data for a microservice chain, UE monitoring data, and network monitoring data, wherein the microservice monitoring data includes first monitoring data for one or more microservices in the microservice chain that were determined to be critical microservices and second monitoring data for one or more microservices in the microservice chain that were determined not to be critical microservices, wherein the first monitoring data comprises more types of monitoring data compared to the second monitoring data.
[00134] At block 920, the one or more network devices analyze the microservice monitoring data, the UE monitoring data, and the network monitoring data to generate an analysis result (in real-time as monitoring data is being collected or offline).
[00135] At block 930, the one or more network devices provide the analysis result to an operator. [00136] Figure 10A illustrates connectivity between network devices (NDs) within an exemplary network, as well as three exemplary implementations of the NDs, according to some embodiments. Figure 10A shows NDs 1000A-H, and their connectivity by way of lines between 1000 A- 1000B, 1000B-1000C, 1000C-1000D, 1000D-1000E, 1000E-1000F, 1000F- 1000G, and 1000A-1000G, as well as between 1000H and each of 1000 A, 1000C, WOOD, and 1000G. These NDs are physical devices, and the connectivity between these NDs can be wireless or wired (often referred to as a link). An additional line extending from NDs 1000A, 1000E, and lOOOF illustrates that these NDs act as ingress and egress points for the network (and thus, these NDs are sometimes referred to as edge NDs; while the other NDs may be called core NDs).
[00137] Two of the exemplary ND implementations in Figure 10A are: 1) a special-purpose network device 1002 that uses custom application-specific integrated-circuits (ASICs) and a special-purpose operating system (OS); and 2) a general purpose network device 1004 that uses common off-the-shelf (COTS) processors and a standard OS.
[00138] The special-purpose network device 1002 includes networking hardware 1010 comprising a set of one or more processor(s) 1012, forwarding resource(s) 1014 (which typically include one or more ASICs and/or network processors), and physical network interfaces (NIs) 1016 (through which network connections are made, such as those shown by the connectivity between NDs 1000A-H), as well as non-transitory machine readable storage media 1018 having stored therein networking software 1020. During operation, the networking software 1020 may be executed by the networking hardware 1010 to instantiate a set of one or more networking software instance(s) 1022. Each of the networking software instance(s) 1022, and that part of the networking hardware 1010 that executes that network software instance (be it hardware dedicated to that networking software instance and/or time slices of hardware temporally shared by that networking software instance with others of the networking software instance(s) 1022), form a separate virtual network element 1030A-R. Each of the virtual network element(s) (VNEs) 1030A-R includes a control communication and configuration module 1032A-R (sometimes referred to as a local control module or control communication module) and forwarding table(s) 1034A-R, such that a given virtual network element (e.g., 1030A) includes the control communication and configuration module (e.g., 1032A), a set of one or more forwarding table(s) (e.g., 1034A), and that portion of the networking hardware 1010 that executes the virtual network element (e.g., 1030A).
[00139] In one embodiment software 1020 includes code such as tailored monitoring component 1023, which when executed by networking hardware 1010, causes the special -purpose network device 1002 to perform operations of one or more embodiments as part of networking software instances 1022 (e.g., to provide tailored monitoring of microservice chain execution in a distributed mobile edge cloud).
[00140] The special-purpose network device 1002 is often physically and/or logically considered to include: 1) a ND control plane 1024 (sometimes referred to as a control plane) comprising the processor(s) 1012 that execute the control communication and configuration module(s) 1032A-R; and 2) a ND forwarding plane 1026 (sometimes referred to as a forwarding plane, a data plane, or a media plane) comprising the forwarding resource(s) 1014 that utilize the forwarding table(s) 1034A-R and the physical NIs 1016. By way of example, where the ND is a router (or is implementing routing functionality), the ND control plane 1024 (the processor(s) 1012 executing the control communication and configuration module(s) 1032A-R) is typically responsible for participating in controlling how data (e.g., packets) is to be routed (e.g., the next hop for the data and the outgoing physical NI for that data) and storing that routing information in the forwarding table(s) 1034A-R, and the ND forwarding plane 1026 is responsible for receiving that data on the physical NIs 1016 and forwarding that data out the appropriate ones of the physical NIs 1016 based on the forwarding table(s) 1034A-R.
[00141] Figure 10B illustrates an exemplary way to implement the special-purpose network device 1002 according to some embodiments. Figure 10B shows a special-purpose network device including cards 1038 (typically hot pluggable). While in some embodiments the cards 1038 are of two types (one or more that operate as the ND forwarding plane 1026 (sometimes called line cards), and one or more that operate to implement the ND control plane 1024 (sometimes called control cards)), alternative embodiments may combine functionality onto a single card and/or include additional card types (e.g., one additional type of card is called a service card, resource card, or multi-application card). A service card can provide specialized processing (e.g., Layer 4 to Layer 7 services (e.g., firewall, Internet Protocol Security (IPsec), Secure Sockets Layer (SSL) / Transport Layer Security (TLS), Intrusion Detection System (IDS), peer-to-peer (P2P), Voice over IP (VoIP) Session Border Controller, Mobile Wireless Gateways (Gateway General Packet Radio Service (GPRS) Support Node (GGSN), Evolved Packet Core (EPC) Gateway)). By way of example, a service card may be used to terminate IPsec tunnels and execute the attendant authentication and encryption algorithms. These cards are coupled together through one or more interconnect mechanisms illustrated as backplane 1036 (e.g., a first full mesh coupling the line cards and a second full mesh coupling all of the cards).
[00142] Returning to Figure 10A, the general purpose network device 1004 includes hardware 1040 comprising a set of one or more processor(s) 1042 (which are often COTS processors) and physical NIs 1046, as well as non-transitory machine-readable storage media 1048 having stored therein software 1050. During operation, the processor(s) 1042 execute the software 1050 to instantiate one or more sets of one or more applications 1064A-R. While one embodiment does not implement virtualization, alternative embodiments may use different forms of virtualization. For example, in one such alternative embodiment the virtualization layer 1054 represents the kernel of an operating system (or a shim executing on a base operating system) that allows for the creation of multiple instances 1062A-R called software containers that may each be used to execute one (or more) of the sets of applications 1064A-R; where the multiple software containers (also called virtualization engines, virtual private servers, or jails) are user spaces (typically a virtual memory space) that are separate from each other and separate from the kernel space in which the operating system is run; and where the set of applications running in a given user space, unless explicitly allowed, cannot access the memory of the other processes. In another such alternative embodiment the virtualization layer 1054 represents a hypervisor (sometimes referred to as a virtual machine monitor (VMM)) or a hypervisor executing on top of a host operating system, and each of the sets of applications 1064A-R is run on top of a guest operating system within an instance 1062A-R called a virtual machine (which may in some cases be considered a tightly isolated form of software container) that is run on top of the hypervisor - the guest operating system and application may not know they are running on a virtual machine as opposed to running on a “bare metal” host electronic device, or through para-virtualization the operating system and/or application may be aware of the presence of virtualization for optimization purposes. In yet other alternative embodiments, one, some or all of the applications are implemented as unikemel(s), which can be generated by compiling directly with an application only a limited set of libraries (e.g., from a library operating system (LibOS) including drivers/! i branes of OS services) that provide the particular OS services needed by the application. As a unikemel can be implemented to run directly on hardware 1040, directly on a hypervisor (in which case the unikemel is sometimes described as running within a LibOS virtual machine), or in a software container, embodiments can be implemented fully with unikemels running directly on a hypervisor represented by virtualization layer 1054, unikemels running within software containers represented by instances 1062A-R, or as a combination of unikemels and the abovedescribed techniques (e.g., unikemels and virtual machines both run directly on a hypervisor, unikemels and sets of applications that are run in different software containers).
[00143] The instantiation of the one or more sets of one or more applications 1064A-R, as well as virtualization if implemented, are collectively referred to as software instance(s) 1052. Each set of applications 1064A-R, corresponding virtualization constmct (e.g., instance 1062A-R) if implemented, and that part of the hardware 1040 that executes them (be it hardware dedicated to that execution and/or time slices of hardware temporally shared), forms a separate virtual network element(s) 1060A-R.
[00144] The virtual network element(s) 1060A-R perform similar functionality to the virtual network element(s) 1030A-R - e.g., similar to the control communication and configuration module(s) 1032A and forwarding table(s) 1034A (this virtualization of the hardware 1040 is sometimes referred to as network function virtualization (NFV)). Thus, NFV may be used to consolidate many network equipment types onto industry standard high volume server hardware, physical switches, and physical storage, which could be located in Data centers, NDs, and customer premise equipment (CPE). While embodiments are illustrated with each instance 1062A-R corresponding to one VNE 1060A-R, alternative embodiments may implement this correspondence at a finer level granularity (e.g., line card virtual machines virtualize line cards, control card virtual machine virtualize control cards, etc.); it should be understood that the techniques described herein with reference to a correspondence of instances 1062A-R to VNEs also apply to embodiments where such a finer level of granularity and/or unikemels are used.
[00145] In certain embodiments, the virtualization layer 1054 includes a virtual switch that provides similar forwarding services as a physical Ethernet switch. Specifically, this virtual switch forwards traffic between instances 1062A-R and the physical NI(s) 1046, as well as optionally between the instances 1062A-R; in addition, this virtual switch may enforce network isolation between the VNEs 1060A-R that by policy are not permitted to communicate with each other (e.g., by honoring virtual local area networks (VLANs)). [00146] In one embodiment, software 1050 includes a tailored monitoring component 1053, which when executed by processor(s) 1042, causes the general purpose network device 1004 to perform operations of one or more embodiments as part of software instances 1062A-R (e.g., to provide tailored monitoring of microservice chain execution in a distributed mobile edge cloud). [00147] The third exemplary ND implementation in Figure 10A is a hybrid network device 1006, which includes both custom ASICs/special -purpose OS and COTS processors/standard OS in a single ND or a single card within an ND. In certain embodiments of such a hybrid network device, a platform VM (i.e., a VM that that implements the functionality of the special-purpose network device 1002) could provide for para- virtualization to the networking hardware present in the hybrid network device 1006.
[00148] A network interface (NI) may be physical or virtual; and in the context of IP, an interface address is an IP address assigned to a NI, be it a physical NI or virtual NI. A virtual NI may be associated with a physical NI, with another virtual interface, or stand on its own (e.g., a loopback interface, a point-to-point protocol interface). A NI (physical or virtual) may be numbered (a NI with an IP address) or unnumbered (a NI without an IP address). A loopback interface (and its loopback address) is a specific type of virtual NI (and IP address) of a NE/VNE (physical or virtual) often used for management purposes; where such an IP address is referred to as the nodal loopback address. The IP address(es) assigned to the NI(s) of a ND are referred to as IP addresses of that ND; at a more granular level, the IP address(es) assigned to NI(s) assigned to a NE/VNE implemented on a ND can be referred to as IP addresses of that NE/VNE.
[00149] It is to be understood that though the said method and system is developed for monitoring microservices, the method may be extended to any application or service. The proposed system and method may also be used for applications that are distributed across servers. [00150] Some portions of the preceding detailed descriptions have been presented in terms of algorithms and symbolic representations of transactions on data bits within a computer memory. These algorithmic descriptions and representations are the ways used by those skilled in the data processing arts to most effectively convey the substance of their work to others skilled in the art. An algorithm is here, and generally, conceived to be a self-consistent sequence of transactions leading to a desired result. The transactions are those requiring physical manipulations of physical quantities. Usually, though not necessarily, these quantities take the form of electrical or magnetic signals capable of being stored, transferred, combined, compared, and otherwise manipulated. It has proven convenient at times, principally for reasons of common usage, to refer to these signals as bits, values, elements, symbols, characters, terms, numbers, or the like.
[00151] It should be bome in mind, however, that all of these and similar terms are to be associated with the appropriate physical quantities and are merely convenient labels applied to these quantities. Unless specifically stated otherwise as apparent from the above discussion, it is appreciated that throughout the description, discussions utilizing terms such as "processing" or "computing" or "calculating" or "determining" or "displaying" or the like, refer to the action and processes of a computer system, or similar electronic computing device, that manipulates and transforms data represented as physical (electronic) quantities within the computer system's registers and memories into other data similarly represented as physical quantities within the computer system memories or registers or other such information storage, transmission or display devices.
[00152] The algorithms and displays presented herein are not inherently related to any particular computer or other apparatus. Various general-purpose systems may be used with programs in accordance with the teachings herein, or it may prove convenient to construct more specialized apparatus to perform the required method transactions. The required structure for a variety of these systems will appear from the description above. In addition, embodiments are not described with reference to any particular programming language. It will be appreciated that a variety of programming languages may be used to implement the teachings of embodiments as described herein.
[00153] An embodiment may be an article of manufacture in which a non-transitory machine- readable storage medium (such as microelectronic memory) has stored thereon instructions (e.g., computer code) which program one or more data processing components (generically referred to here as a “processor”) to perform the operations described above. In other embodiments, some of these operations might be performed by specific hardware components that contain hardwired logic (e.g., dedicated digital filter blocks and state machines). Those operations might alternatively be performed by any combination of programmed data processing components and fixed hardwired circuit components.
[00154] Throughout the description, embodiments have been presented through flow diagrams. It will be appreciated that the order of transactions and transactions described in these flow diagrams are only intended for illustrative purposes and not intended as a limitation of the present invention. One having ordinary skill in the art would recognize that variations can be made to the flow diagrams without departing from the broader spirit and scope of the invention.
[00155] In the foregoing specification, embodiments have been described with reference to specific example embodiments thereof. It will be evident that various modifications may be made thereto without departing from the broader spirit and scope of the invention as set forth in the following claims. The specification and drawings are, accordingly, to be regarded in an illustrative sense rather than a restrictive sense.

Claims

CLAIMS What is claimed is:
1. A method performed by one or more network devices to provide tailored monitoring of microservice chain execution in a distributed mobile edge cloud, the method comprising: determining (710) a microservice topology of a microservice chain; determining (720) which microservices in the microservice chain are critical microservices based on the microservice topology; determining (730) which types of monitoring data to collect for each of one or more microservices in the microservice chain based on whether the microservice was determined to be a critical microservice; generating (740) an integrated microservice chain model for the microservice chain that comprises an indication of the microservice topology, an indication of the critical microservices, and an indication of the types of monitoring data to collect for each of the one or more microservices; and providing (750) the integrated microservice chain model to a microservice tracker (140), wherein the microservice tracker is configured to collect, from the distributed mobile edge cloud while the microservice chain is being executed in the distributed mobile edge cloud, monitoring data for each of the one or more microservices corresponding to the types of monitoring data to collect for the microservice as indicated in the integrated microservice chain model and provide the collected monitoring data to a monitoring data analyzer (180) that is to analyze the collected monitoring data for the one or more microservices, together with user equipment (UE) monitoring data and network monitoring data, to generate an analysis result.
2. The method of claim 1, wherein the types of monitoring data to collect for a critical microservice comprises more types of monitoring data compared to the types of monitoring data to collect for a non-critical microservice.
3. The method of claim 2, wherein the types of monitoring data to collect for the critical microservice and the types of monitoring data to collect for the non-critical microservice both comprise a time at which the microservice began execution and a time at which the microservice completed execution, wherein the types of monitoring data to collect for the critical microservice further comprises one or more other types of monitoring data that are not included in the types of monitoring data to collect for the non-critical microservice.
34
4. The method of claim 3, wherein the one or more other types of monitoring data comprises one or more of: edge site location at which the microservice began execution, times at which the microservice changed state, edge site locations at which the microservice changed state, times at which the microservice resumed execution, edge site locations at which the microservice resumed execution, and edge site location at which the microservice completed execution.
5. The method of claim 1, further comprising: determining input and output data sets of each microservice in the microservice chain; determining which data items used by microservices in the microservice chain are critical data items; determining data item dependencies between data items used by the microservices in the microservice chain; determining, based on the data item dependencies, which data items used by the microservices in the microservice chain are data items that the critical data items are dependent upon; determining monitoring intervals for each of one or more data items used by the microservices in the microservice chain based on whether the data item was determined to be a critical data item or a data item that a critical item is dependent upon; and adding, to the integrated microservice chain model, an indication of the input and output data sets of each microservice in the microservice chain, an indication of the critical data items, an indication of the data items that the critical data items are dependent upon, and an indication of the monitoring intervals for each of the one or more data items, wherein the microservice tracker is to further collect, from the distributed mobile edge cloud, monitoring data for each of the one or more data items according to the monitoring intervals for the data item as indicated in the integrated microservice chain model and provide the collected monitoring data for the one or more data items to the monitoring data analyzer.
6. The method of claim 5, wherein the determination of which microservices in the microservice chain are critical microservices is based on identifying a critical path in the microservice chain and identifying microservices on the critical path as being critical microservices.
35
7. The method of claim 6, wherein the determination of which data items used by the microservices in the microservice chain are critical data items is based on identifying data items in the input and output data sets of the critical microservices as being critical data items.
8. The method of claim 6, wherein the monitoring intervals for a critical data item or a data item that a critical data item depends on are more granular compared to the monitoring intervals for a non-critical data item.
9. The method of claim 8, wherein the monitoring intervals for the critical data item and the monitoring intervals for the non-critical data items both comprise times at which the data item is taken as an input of a microservice and times at which the data item is provided as an output of a microservice, wherein the monitoring intervals for the critical data item further comprises one or more other monitoring intervals not included in the monitoring intervals for the non-critical data item.
10. The method of claim 9, wherein the one or more other monitoring intervals comprise times at which the data item is updated by a microservice.
11. The method of claim 1 , further comprising: causing the microservice chain to be instantiated in the distributed mobile edge cloud; and providing information regarding the instantiated microservice chain to a user equipment (UE) tracker, wherein the UE tracker is configured to use the information regarding the instantiated microservice chain to collect UE monitoring data from a mobile network for UEs that use the instantiated microservice chain.
12. A method performed by one or more network devices to provide tailored monitoring of microservice chain execution in a distributed mobile edge cloud, the method comprising: obtaining (810) an integrated microservice chain model for a microservice chain, wherein the integrated microservice chain model comprises an indication of a microservice topology of the microservice chain, an indication of which microservices in the microservice chain are critical microservices, and an indication of which types of monitoring data to collect for each of one or more microservices in the microservice chain; collecting (820), from the distributed mobile edge cloud while the microservice chain is being executed in in the distributed mobile edge cloud, monitoring data for each of the one or more microservices corresponding to the types of monitoring data to collect for the microservice as indicated in the integrated microservice chain model; and providing (830) the collected monitoring data for the one or more microservices to a monitoring data analyzer (180), wherein the monitoring data analyzer is to analyze the collected monitoring data for the one or more microservices, together with user equipment (UE) monitoring data and network monitoring data, to generate an analysis result.
13. The method of claim 12, wherein monitoring data collected for a critical microservice comprises more types of monitoring data compared to the monitoring data collected for a non- critical microservice.
14. The method of claim 12, wherein the integrated microservice chain model further includes an indication of input and output data sets of each microservice in the microservice chain, an indication of critical data items used by microservices in the microservice chain, an indication of which data items used by the microservices in the microservice chain are data items that the critical data items are dependent upon, and an indication of monitoring intervals for each of one or more data items.
15. The method of claim 14, further comprising: collecting, from the distributed mobile edge cloud, monitoring data for each of the one or more data items according to the monitoring intervals for the data item as indicated in the integrated microservice chain model; and providing the collected monitoring data for the one or more data items to the monitoring data analyzer, wherein the monitoring data analyzer is to further analyze the collected monitoring data for the one or more data items to generate the analysis result.
16. The method of claim 15, wherein the monitoring intervals for a critical data item are more granular compared to the monitoring intervals for a non-critical data item.
17. The method of claim 12, wherein the analysis result is generated in real-time as monitoring data is being collected.
18. A set of non-transitory machine-readable media having computer code stored therein, which when executed by a set of one or more processors of one or more network devices, causes the one or more network devices to perform operations for providing tailored monitoring of microservice chain execution in a distributed mobile edge cloud, the operations comprising: determining (710) a microservice topology of a microservice chain; determining (720) which microservices in the microservice chain are critical microservices based on the microservice topology; determining (730) which types of monitoring data to collect for each of one or more microservices in the microservice chain based on whether the microservice was determined to be a critical microservice; generating (740) an integrated microservice chain model for the microservice chain that comprises an indication of the microservice topology, an indication of the critical microservices, and an indication of the types of monitoring data to collect for each of the one or more microservices; and providing (750) the integrated microservice chain model to a microservice tracker (140), wherein the microservice tracker is configured to collect, from the distributed mobile edge cloud while the microservice chain is being executed in the distributed mobile edge cloud, monitoring data for each of the one or more microservices corresponding to the types of monitoring data to collect for the microservice as indicated in the integrated microservice chain model and provide the collected monitoring data to a monitoring data analyzer (180) that is to analyze the collected monitoring data for the one or more microservices, together with user equipment (UE) monitoring data and network monitoring data, to generate an analysis result.
19. The set of non-transitory machine-readable media of claim 18, wherein the types of monitoring data to collect for a critical microservice comprises more types of monitoring data compared to the types of monitoring data to collect for a non-critical microservice.
20. The set of non-transitory machine-readable media of claim 19, wherein the types of monitoring data to collect for the critical microservice and the types of monitoring data to collect for the non-critical microservice both comprise a time at which the microservice began execution and a time at which the microservice completed execution, wherein the types of monitoring data to collect for the critical microservice further comprises one or more other types of monitoring data that are not included in the types of monitoring data to collect for the non- critical microservice.
38
PCT/SE2022/050023 2022-01-13 2022-01-13 Method and apparatus for tailored data monitoring of microservice executions in mobile edge clouds WO2023136755A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/SE2022/050023 WO2023136755A1 (en) 2022-01-13 2022-01-13 Method and apparatus for tailored data monitoring of microservice executions in mobile edge clouds

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/SE2022/050023 WO2023136755A1 (en) 2022-01-13 2022-01-13 Method and apparatus for tailored data monitoring of microservice executions in mobile edge clouds

Publications (1)

Publication Number Publication Date
WO2023136755A1 true WO2023136755A1 (en) 2023-07-20

Family

ID=87279563

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/SE2022/050023 WO2023136755A1 (en) 2022-01-13 2022-01-13 Method and apparatus for tailored data monitoring of microservice executions in mobile edge clouds

Country Status (1)

Country Link
WO (1) WO2023136755A1 (en)

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9716617B1 (en) * 2016-06-14 2017-07-25 ShieldX Networks, Inc. Dynamic, load-based, auto-scaling network security microservices architecture
US20200112497A1 (en) * 2018-10-09 2020-04-09 Verizon Patent And Licensing Inc. Monitoring cloud-based services and/or features
US20200162380A1 (en) * 2018-11-19 2020-05-21 International Business Machines Corporation Controlling data communication between microservices
US20200259715A1 (en) * 2019-02-08 2020-08-13 International Business Machines Corporation Topology-Aware Continuous Evaluation of Microservice-based Applications
US20200412623A1 (en) * 2019-06-26 2020-12-31 International Business Machines Corporation Prioritization of service restoration in microservices architecture
US20210191706A1 (en) * 2019-12-18 2021-06-24 Citrix Systems, Inc. Service graphs for canary deployment systems and methods
US20210234930A1 (en) * 2020-01-27 2021-07-29 Dell Products L. P. Microservice management system for recommending modifications to optimize operation of microservice-based systems
US11223522B1 (en) * 2021-01-15 2022-01-11 Dell Products L.P. Context-based intelligent re-initiation of microservices
US20220318671A1 (en) * 2021-04-06 2022-10-06 International Business Machines Corporation Microservice compositions

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9716617B1 (en) * 2016-06-14 2017-07-25 ShieldX Networks, Inc. Dynamic, load-based, auto-scaling network security microservices architecture
US20200112497A1 (en) * 2018-10-09 2020-04-09 Verizon Patent And Licensing Inc. Monitoring cloud-based services and/or features
US20200162380A1 (en) * 2018-11-19 2020-05-21 International Business Machines Corporation Controlling data communication between microservices
US20200259715A1 (en) * 2019-02-08 2020-08-13 International Business Machines Corporation Topology-Aware Continuous Evaluation of Microservice-based Applications
US20200412623A1 (en) * 2019-06-26 2020-12-31 International Business Machines Corporation Prioritization of service restoration in microservices architecture
US20210191706A1 (en) * 2019-12-18 2021-06-24 Citrix Systems, Inc. Service graphs for canary deployment systems and methods
US20210234930A1 (en) * 2020-01-27 2021-07-29 Dell Products L. P. Microservice management system for recommending modifications to optimize operation of microservice-based systems
US11223522B1 (en) * 2021-01-15 2022-01-11 Dell Products L.P. Context-based intelligent re-initiation of microservices
US20220318671A1 (en) * 2021-04-06 2022-10-06 International Business Machines Corporation Microservice compositions

Similar Documents

Publication Publication Date Title
US12074780B2 (en) Active assurance of network slices
US11296923B2 (en) Network fault originator identification for virtual network infrastructure
US10686568B2 (en) Active flow diagnostics for cloud-hosted networks
US20190052551A1 (en) Cloud verification and test automation
US11516050B2 (en) Monitoring network traffic using traffic mirroring
US9596141B2 (en) Representing software defined networks using a programmable graph model
US20150169353A1 (en) System and method for managing data center services
US10764214B1 (en) Error source identification in cut-through networks
US20210312472A1 (en) Method and system for prediction of smart contract violation using dynamic state space creation
John et al. Scalable software defined monitoring for service provider devops
US20190205776A1 (en) Techniques for policy-controlled analytic data collection in large-scale systems
US11539728B1 (en) Detecting connectivity disruptions by observing traffic flow patterns
US10397143B1 (en) Preventing transmission of errors in a computing network
WO2023136755A1 (en) Method and apparatus for tailored data monitoring of microservice executions in mobile edge clouds
US11553001B2 (en) End user security manager
WO2022090809A1 (en) Machine learning life cycle management for networks
US20200195523A1 (en) Server assisted network discovery (sand)
Tairaku et al. Social data driven SDN network operation using northbound interface
US11916701B2 (en) Coordinated observability for dynamic VPN switchover
US20230090203A1 (en) Case deflection using visibility into multi-product clouds
de Freitas Network softwarization for IACS security applications
Arnold Understanding Cloud Network Performance
Risdianto DevOps-based Collaboration for Building and Operating SDN-enabled Multisite Cloud Playground
Raad Protocol architecture and algorithms for distributed data center networks
Nikkhouy Monitoring Service Chains in the Cloud

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22920869

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE