CN112925649A - Unified monitoring method for virtual network functions - Google Patents

Unified monitoring method for virtual network functions Download PDF

Info

Publication number
CN112925649A
CN112925649A CN202110347584.XA CN202110347584A CN112925649A CN 112925649 A CN112925649 A CN 112925649A CN 202110347584 A CN202110347584 A CN 202110347584A CN 112925649 A CN112925649 A CN 112925649A
Authority
CN
China
Prior art keywords
pod
service
monitoring
module
value
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202110347584.XA
Other languages
Chinese (zh)
Other versions
CN112925649B (en
Inventor
周聪
时向泉
陶静
原玉磊
赵宝康
赵锋
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
National University of Defense Technology
Original Assignee
National University of Defense Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by National University of Defense Technology filed Critical National University of Defense Technology
Priority to CN202110347584.XA priority Critical patent/CN112925649B/en
Publication of CN112925649A publication Critical patent/CN112925649A/en
Application granted granted Critical
Publication of CN112925649B publication Critical patent/CN112925649B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/50Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5083Techniques for rebalancing the load in a distributed system
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/50Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005Allocation of resources, e.g. of the central processing unit [CPU] to service a request
    • G06F9/5027Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
    • G06F9/505Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals considering the load

Abstract

The invention discloses a unified monitoring method of virtual network functions, aiming at establishing a 'service-performance' model based on monitoring indexes and guiding calculation and memory resource allocation. The technical scheme is that a virtual network function unified monitoring platform is constructed by a monitoring object discovery module, a service monitoring index generation module, a performance monitoring index generation module, a monitoring index unified acquisition module, a resource centralized distribution module and a time sequence database. All modules of the monitoring platform work in parallel and are matched with each other, the CPU, the memory use condition and the service operation condition of the Pod are monitored, monitoring index metadata are generated, management labels are completed according to the metadata condition and are stored in a time sequence database, when a monitoring threshold value alarm occurs, a service-performance model is established, and a new resource allocation scheme is generated. The invention solves the problem of resource allocation when the domain exceeding value alarm occurs on the premise of ensuring that the service function chain can complete the service smoothly, and ensures that the resource allocation is reasonable and efficient.

Description

Unified monitoring method for virtual network functions
Technical Field
The invention relates to the technical field of network function virtualization, in particular to a unified monitoring method for virtual network functions.
Background
Conventional network architectures use a large number of dedicated hardware devices to provide network functions such as load balancing, intrusion prevention, network address translation, etc. With the rapid development of emerging industries such as big data and cloud computing, the deployment of new applications often needs the support of new special equipment and networks, and brings great challenges to network operation cost and operation and maintenance. To cope with these problems, the European Telecommunications Standards Institute (ETSI) proposes Network Function Virtualization (NFV). The NFV technology utilizes a common commercial server to provide running environments such as a virtual machine and a Docker container for network functions, and software running in the virtual machine or the container is used for replacing hardware equipment, so that the flexibility of the network functions is improved, and the investment cost, the research and development cost and the operation and maintenance cost are reduced.
Docker is a virtual environment container, can pack development environment, code, configuration file, etc. into the container together, and issue and apply to arbitrary platform. Through the management of the life cycles of encapsulation, distribution, deployment, operation and the like of the application components, the purposes of 'one-time encapsulation and everywhere operation' of the application program of a user and the operation environment thereof can be achieved.
Kubernetes (k8s) is an application platform for managing containerization on a plurality of hosts in a cloud platform, aims to simply and efficiently deploy containerization applications, provides a mechanism for deploying, planning, updating and maintaining applications, and is widely applied in academia and industry at present. In k8s, the control node of the cluster is called Master node, and is responsible for management and control of the whole cluster. In addition to the Master Node, other machines are called Node nodes, which are workload nodes in a cluster, and each Node is assigned some workload, such as a Docker container. The Pod is deployed on a Node and is a basic unit scheduled by k8s, each Pod comprises one or more service containers of users, k8s allocates a unique IP address for each Pod, and the containers in the Pod share the Pod IP address. The label Labels is a KV key-value pair, which is defined by users and can be attached to resource objects, such as Node and Pod. The multi-dimensional resource grouping management function is realized by binding one or more different labels to the specified resource object, so that the management work of resource allocation, scheduling, monitoring and the like can be flexibly and conveniently carried out. The management label existing in k8s includes the name of Pod, the name of the Node to which the Pod belongs, and the like.
To complete a certain Function, one data stream usually needs to pass through a plurality of Pod Service Function units, such as intrusion detection, deep packet inspection, load balancing, and the like, and finally reaches a destination, which is the most common scenario of a Service Function Chain (Service Function Chain).
In order to reflect the operation condition and the resource utilization rate of the k8s platform, the monitoring process is indispensable.
In the aspect of VNF Monitoring index types, for the problem that the Monitoring method embedded in the k8s system only focuses on the utilization rate of the CPU, Chang et al published a paper "Dynamic Cloud Resource deployment Monitoring Platform Based on k8 s" on GLOBECOM 2017 (i.e. A k8s-Based Monitoring Platform for Dynamic Cloud Resource Provisioning, referred to as background technology one), it is proposed to use cAdvisor software to replace native embedded Monitoring, and use a heapster to collect Monitoring data to provide richer Monitoring indexes, but this method only enriches relevant performance Monitoring indexes such as CPU, memory, and network, and does not collect VNF service processing process states, and cannot meet the requirement of accurately drawing a VNF service function unit "image", and provide data support for Resource allocation.
In terms of VNF monitoring system architecture, Gokan et al, 2018 CloudNet, published a paper NFV checker: a system method for configuring and analyzing virtual network functions (NFV-instruction: A system adaptive to profile and analysis virtual network functions, referred to as background technology II) provides an open-source platform, and system performance indexes are pushed to a monitoring database of NFV-MON by using NFV-MON Client function. Davide et al published a paper "D-STREAMON" in 2017 at the First International Workshop on software infrastructure for 5G and Fog Computing (First International Workshop for 5G and Fog Computing): and (3) a Network Monitoring Distributed Framework (namely D-STREAMON: NFV-Capable Distributed Framework for Network Monitoring, referred to as background technology III) supporting NFV, and acquiring a Monitoring index in a pull mode through a probe. The two types of monitoring system architectures are based on a pull/push mode, the problem of monitoring index acquisition and transmission is solved, and a correlation model between monitoring indexes is not established.
In monitoring the application of metrics, li bin et al published a paper "hybrid scaler" in 2016 International Symposium on Parallel and Distributed Computing conference (International Symposium on Distributed Computing): a resource-pressure model is provided in Handling burst Workload of a multi-layer Web application program of a Cloud (namely, hybrid scaler: Handling burst Workload for Multi-tier Web Applications in Cloud, which is called background technology four for short), and the model is established on a performance indicator of multi-layer application and belongs to a special case of a service function chain. The resource-pressure model provides a mapping relation between the resource distribution quantity and the bearable load pressure, the monitoring index can provide decision data for the horizontal expansion and the vertical expansion of the virtual machine, but the network time delay is used as the pressure measurement index, the change trend of the service processing quantity cannot be accurately reflected, and the constraint condition that the resource of the whole service function chain is limited cannot be met.
In summary, in the application scenario of NFV, the use of cAdvisor + Prometheus + infiluxdb for monitoring of k8s platform is a typical application. There are the following problems: lack of monitoring of service processing rate index, when the Function and operation and maintenance management data of the virtual/physical network Function unit are stored in their respective logs, rather than being managed in a unified manner, it is not beneficial to implement resource allocation of each Virtual Network Function (VNF); the Node and Pod of the k8s platform are frequently started and closed, and if an automatic mechanism is lacked to discover the change of the Node and Pod and establish the monitoring index association, the difficulty of managing operation and maintenance is increased; the performance monitoring of k8s has already realized performance index alarm over-domain value, but the alarm information cannot be associated with the service processing process, which is not beneficial to realizing calculation and reasonable memory resource allocation from the global perspective.
Therefore, a unified monitoring architecture of the NFV monitoring data is researched, different types of VNF service functional units are accessed to a monitoring platform, and meanwhile, the monitoring flow ratio needs to be reduced; the method comprises the steps of collecting different monitoring indexes of different types of VNFs, automatically establishing correlation among the monitoring indexes, establishing a monitoring index guide model, realizing reasonable resource distribution of VNF service function units on the SFC, and having important significance for SFC providers and users.
Disclosure of Invention
The invention aims to solve the technical problems that the data types of monitoring indexes are insufficient, no correlation exists among the monitoring indexes, a model related to business and performance is lacked, and reasonable distribution of limited resources on the SFC cannot be accurately guided.
The invention provides a unified monitoring method for virtual network functions based on a k8s platform, which uses a distributed module and asynchronous communication system architecture to provide unified interfaces for different VNF service function units, receives metadata of service monitoring indexes, realizes the functions of collecting and fusing different monitoring indexes of 7 different types of VNFs, and reduces the monitoring flow ratio.
The invention utilizes API (Application Programming Interface) of k8s to automatically discover the monitored object and add management label; establishing association between the service monitoring index and the performance index by complementing the monitoring index management label; and acquiring monitoring index metadata from a database, establishing a service-performance model by monitoring the service processing rate, the CPU use condition and the memory use condition of the service function unit, and continuously updating the model according to the monitoring index to guide a system administrator to reasonably distribute limited calculation and memory resources on a service function chain.
The invention comprises the following steps:
firstly, a virtual network function unified monitoring platform is constructed.
1.1, constructing a virtual network function running platform and deploying a service function chain.
Constructing a k8s platform comprising a Master Node and N Node nodes, wherein a VNF service function unit is deployed in a scheduling basic unit Pod corresponding to k8s and is allocated with corresponding CPU and memory resources; pod belongs to different namespaces of the k8s platform and can be specified by the user. If not, the Pod will belong to the default space, represented by "default". The traffic can go through the Pod in order as agreed to form a service function chain. The Pod where the VNF service functional unit is deployed is named Pod _ < VNF >, where < VNF > is an abbreviation of a name of the VNF service functional unit, and the existing 7 types of VNF service functional units include: the system comprises a message filtering function, a load balancing function, a network address conversion function, a deep message detection function, an intrusion prevention function, a protocol identification function and a Web application protection function. If the Pod _ < vnf > needs to be added into the virtual network function unified monitoring platform, so that the unified monitoring platform collects the service index metadata, the tag "monitored" is added to the Pod _ < vnf > (only the Pod with the tag can be added into the unified monitoring platform). Appointing: each Pod only deploys one Docker container related to the VNF service function, and each Pod only belongs to one service function chain.
When the VNF service functional unit accesses the k8s platform, the VNF service functional unit is divided into a VNF unit with a customized function and a VNF unit with a standard function (including a physical functional unit). The VNF unit with the customization function is a VNF capable of modifying source codes to meet special requirements and directly generates service monitoring index metadata; the VNF unit (including the physical function unit) with the standard function belongs to mature software, and the log file includes the service processing state and process of the VNF unit with the standard function, and it is necessary to analyze the fields of the existing log file to generate service monitoring index metadata.
In order to establish an association relationship between different monitoring indexes, on the basis of an original Pod management tag (the name of a Pod) of a k8s platform, 2 management tags are added for each Pod, that is: the service Function chain name sfc (service Function Chaining) to which the Pod belongs, and the position sfcindex (service Function Chaining index) of the service Function chain to which the Pod belongs.
1.2, constructing a virtual network function unified monitoring platform.
The virtual network function unified monitoring platform consists of a monitoring object discovery module, a service monitoring index generation module, a performance monitoring index generation module, a monitoring index unified acquisition module, a resource centralized distribution module and a time sequence database.
And (3) newly building a Pod, named Pod _ dis, at the Master node of the k8s platform, and deploying the monitoring object discovery module and the monitoring index uniform acquisition module at the Pod _ dis.
The monitoring object discovery module is connected with the monitoring index uniform acquisition module and consists of a Node object discovery module, a Pod object discovery module and a monitoring object configuration file autoDisConfig. The autoDisConfig includes 6 fields, which are respectively: the frequency of interface calls scheduleInterval, the location of the configuration file of K8s K8sConfigPath, the Pod monitor object namespace K8SNamespace (at K8s platform, each Pod belongs to a different namespace, monitor object discovery module can discover the Pod under the specified namespace by reading K8 SNamespace), the screening tag monitoral of Pod monitor objects (monitoral is the identification of K8s platform indicating whether the Pod needs to be monitored, if the tag is "monitored", indicating that it needs to be monitored, otherwise it does not need to be monitored. The Node object discovery module is connected with an application program interface of a k8s platform, the Node object discovery module calls an API function listNode of k8s in each schedulolenterval to discover Node objects (making N found together, N being the total number of the Node objects), generates Node monitoring object files for N nodes by taking the IP addresses of the nodes as file names, and makes the Node monitoring object files of the nodes with the IP addresses being the Node IP be < Node IP >. yml. Storing N Node monitoring object files in a directory pwd _ Node _ targets; the Pod object discovery module is connected with an application program interface of the k8s platform, when the Node object discovery module discovers Node objects, the Pod object discovery module calls an API function listnamespaced Pod of k8s to discover Pod objects (M are found in total, M is the total number of Pod objects), and generates a Pod monitoring object file of each Pod by taking the IP addresses of the SFC and the Pod to which the Pod belongs as file names, and the Pod monitoring object file of the Pod of which the IP address is PodIP is made to be < SFC + PodIP >. yml. M Pod monitor object files are stored under the directory pwd _ Pod _ targets.
The performance monitoring index generating module adopts cAdviror (open source software website: https:// githu. com/google/cAdviror, version: V0.35 and above) mentioned in the background art I, and is connected with the monitoring index uniform acquisition module. The performance monitoring index generation module creates a Pod on each Node, named Pod _ prf _ < x >, where < x > <1, 2, … N (since each Node deploys a Pod of cadvisors, that is, N nodes need to deploy N cadvisors in total). The cAdvisor is deployed on the pod _ prf _ < x >, and interacts with the Docker engine to acquire information to generate performance monitoring index metadata (metadata including CPU utilization rate and memory utilization rate indexes) for being pulled by the monitoring index unified acquisition module.
The service monitoring index generation module is deployed in pod _ < vnf >. And the service monitoring index generating module is connected with the restFul data interface of the VNF service functional unit and the monitoring index uniform acquisition module. The system consists of a service index metadata analysis module, a service index release module and a service monitoring index configuration file vnfConfig. vnfConfig includes two domains: service index metadata tag group tags (service index metadata includes a plurality of tags, tags is a name list of the tags), type of service index metadata (includes two values: String indicates that the value of the tag is character type, and Double indicates that the value of the tag is numerical type). The content of vnfConfig is configured according to the type of VNF business functional unit (7 types in total) on pod _ < VNF >. And the service index metadata analysis module is connected with the restFul data interface and the service index publishing module. The service index metadata analysis module monitors a restFul data interface, receives a service monitoring index metadata character string pushed by a VNF service function unit in the pod _ < VNF >, analyzes the character string, caches and generates service monitoring index metadata (including service operation monitoring index metadata and module operation state), and sends the service monitoring index metadata to the service index publishing module. Because the service monitoring index generation module and the VNF service function unit are deployed in the same Pod, the monitoring data flow occupies network resources of a k8s platform by communicating with the VNF service function unit through a local protocol stack and using a cache. The service index issuing module is connected with the service index metadata analyzing module and the monitoring index unified collecting module, monitors a 'pull monitoring index' request of the monitoring index unified collecting module, and issues the service monitoring index metadata to the monitoring index unified collecting module when receiving the 'pull monitoring index' request.
The monitoring index unified acquisition module is connected with the monitoring object discovery module, the time sequence database, the service monitoring index generation module and the performance monitoring index generation module, adopts Prometheus (open source software website: https:// gitubb. com/Prometheus, version V2.16 and above) and is deployed in pod _ dis. The monitoring index acquisition module is provided with a monitoring index acquisition configuration file monConfig, and the monConfig comprises 4 fields which are respectively as follows: the collection cycle scrape _ interval, the storage location pwd _ Node _ targets of the Node object file (i.e. under the "scrape _ configurations" entry of the configuration file of the proceus, one entry of "-jobname" is configured as "Node", the "-files" option of the next entry "file _ sd _ configurations" of "Node" is configured as storage location pwd _ Node _ targets), the storage location pwd _ Pod _ targets of the Pod object file (i.e. under the "scrape _ configurations" entry of the configuration file of the proceus, another entry of "-jobname" is configured as "Pod", the "es" option of the "Pod" next entry "file _ sd _ configurations" is configured as storage location pwd _ Pod _ target), and the database write address is configured as write address. The unified monitoring index acquisition module periodically acquires a monitoring object from the lists of pwd _ node _ targets and pwd _ pod _ targets according to an acquisition period scrape _ interval configured by monConfig, sends a 'pull monitoring index' request to the performance monitoring index generation module, and collects performance monitoring index metadata generated by the performance monitoring index generation module; meanwhile, a 'pull monitoring index' request is sent to a service monitoring index generation module, and service monitoring index metadata is collected. And adding labels to the performance monitoring index metadata and the service monitoring index metadata, and storing the performance monitoring index metadata and the service monitoring index metadata added with the labels in a time sequence database.
The time sequence database is connected with the monitoring index unified acquisition module and the resource centralized distribution module, is created by using open source software InfluxDB (version V1.7.9 and above), is named as 'nfvMonitor', is deployed on a server outside a k8s platform and is used for storing service monitoring index metadata and performance monitoring index metadata, and each index metadata is independently stored in a form of a table, and comprises the following steps: the system comprises a service monitoring index generation module state index table, a service operation monitoring index detail table, a service function unit CPU use condition table and a service function unit memory use condition table. The state _ up of the state index table of the service monitoring index generation module comprises 9 table entries: a timestamp (i.e., system time generated by a record item), a CPU configuration value (i.e., the number of CPU cores configured by a system administrator for a Pod), a memory configuration value (i.e., the number of memories configured by the system administrator for the Pod), a VNF name (i.e., the name of a VNF service functional unit), a Node name (i.e., the name of a Pod deployed in a Node), a Pod name (i.e., the name of a Pod belonging to the Node), an SFC name (i.e., the name of a service functional chain belonging to the Pod), an SFC location (i.e., the location number of the service functional chain belonging to the Pod), and a start state of a monitoring acquisition module (i.e., whether the monitoring acquisition module is in a start state); the service operation monitoring index detail table detail _ count comprises table entries: a timestamp (i.e., system time generated by the entry), a VNF metadata set (i.e., a tag item of service index metadata of a VNF service function unit configuration file vnfConfig), a Node name (i.e., a name of a Node to which a Pod belongs), a Pod name (i.e., a name of a Pod to which a container belongs), an SFC name (i.e., a name of a service function chain to which a Pod belongs), an SFC location (i.e., a location serial number of the service function chain to which a Pod belongs), and a service processing count value (i.e., a number of service processes); service function unit CPU usage table (i.e. container _ CPU _ user _ seconds _ total table of cAdvisor) user _ CPU, including table entry: a timestamp (i.e., system time generated by the entry), a container tag set (i.e., a system management tag set generated by the k8s platform for the container), a Pod name (i.e., the name of a Pod to which the container belongs), a Node name (i.e., the name of a Node to which the Pod belongs), an SFC name (i.e., the name of a service function chain to which the Pod belongs), an SFC location (i.e., the location number of the service function chain to which the Pod belongs), and a CPU usage value (i.e., time used by the CPU); the service function unit memory usage table (i.e. container _ memory _ usage _ bytes table of cAdvisor) usage _ memory includes entries: a timestamp (i.e., a system time generated by the entry), a container tag set (i.e., a system management tag set generated by the k8s platform for the container), a Pod name (i.e., a name of a Pod to which the container belongs), a Node name (i.e., a name of a Node to which the Pod belongs), an SFC name (i.e., a name of a service function chain to which the Pod belongs), an SFC location (i.e., a location number of the service function chain to which the Pod belongs), and a memory usage value (i.e., a memory usage amount).
The resource centralized distribution module obtains the running state index metadata of the service monitoring index generation module of the Pod from the state _ up table, and obtains the current CPU and memory distribution situation of the Pod; obtaining metadata of the VNF service function unit processing traffic volume rate from a detail _ count table; and obtaining metadata of the CPU utilization rate and the memory utilization rate when the current service is processed from the user _ CPU and the user _ memory table. And establishing a service-performance model according to the 4 index metadata to guide a user to allocate calculation and memory resources.
The resource centralized distribution module and the time sequence database are deployed in the same server, the resource centralized distribution module is connected with the time sequence database, and the resource centralized distribution module is composed of a service-performance model building module, a resource distribution scheme generating module and a resource configuration file resAdmConfig. The resource profile includes 6 fields: a time series dataBase URL (Uniform Resource Locator), a dataBase name dataBase, a frequency searchInterval of querying the dataBase, a time offset timeOffset of querying the dataBase, a CPU warning threshold cpuTSD, and a memory warning threshold memTSD. The service-performance model building module is connected with the time sequence dataBase and the resource allocation scheme generating module, data between the current time and the previous time offset seconds are obtained from a detail _ count table, a user _ CPU table and a use _ memory table respectively according to the frequency searchInterval of the query dataBase in the dataBase under the URL of the time sequence dataBase, and the CPU utilization rate, the memory utilization rate and the service processing rate are calculated; and obtains the number of CPU cores and the current status of memory allocation from the state _ up table. And when the CPU utilization rate exceeds the CPU TSD or the memory utilization rate exceeds the memTSD, establishing a service-performance model, and sending the service-performance model to the resource allocation scheme generation module. The resource allocation scheme generation module is connected with the business-performance model building module, generates a Pod new CPU and memory resource allocation scheme according to the business-performance model received from the business-performance model building module, and sends the CPU and memory resource allocation scheme to a K8S platform administrator. The K8S platform administrator can adjust the CPU and memory resource allocation of the corresponding Pod on the service function chain according to the scheme, and reconfigure the resources by interacting with the K8s platform.
Therefore, the resource centralized allocation module provides data support for the next decision round.
Secondly, initializing a virtual network function unified monitoring platform, wherein the method comprises the following steps:
2.1 configuring a monitoring object configuration file autoDisConfig of the monitoring object discovery module:
2.1.1 setting the frequency schedulelnterval value of the interface calling to be more than or equal to 60 seconds, and setting the default value to be 60 seconds;
2.1.2 set the value of the configuration file Config location k8sConfigPath of k8s to the location of the file Config, the default value "/root/. kube/Config";
2.1.3, setting the value of the Pod monitoring object namespace K8SNamespace as a default value 'default' (namely, under the default condition, the Pod with the namespace being 'default' is possibly added into the unified monitoring platform), supporting a plurality of namespace settings, using and separating namespaces;
2.1.4 setting the value of the screening label monitore Label of the Pod monitoring object to be "monitored", and adding the Pod comprising the label "monitored" into the unified monitoring platform;
2.1.5 setting the storage position pwd _ Node _ targets value of the Node object file as default value "/home/targets/nodes";
2.1.6 sets the storage location pwd _ Pod _ targets value of the Pod object file to the default value "/home/targets/pods".
2.2 configuring a monitoring index configuration file monConfig of the monitoring index uniform acquisition module:
2.2.1, setting the value of pwd _ node _ targets as a default value "[ '/home/targets/nodes/. yml' ]" (the monitoring index unified collection module and the monitoring object discovery module are deployed by using a dokcer container, the operating environments are independent from each other, and configuration items with the same name cannot cause conflict);
2.2.2 set pwd _ pod _ targets value to default value "[ '/home/targets/pod/. yml' ]";
2.2.3 configure remote _ write as "-url http: /< time-series database IP address >: < port >/api/v1/prom/write ═ nfvmmonitor ", where < time series database IP address > is the time series database IP address and < port > defaults to 8086;
2.2.4 set the acquisition period script _ interval value to 60 seconds.
And 2.3, configuring a service monitoring index configuration file vnfConfig of the service monitoring index generating module according to the type of the VNF service functional unit on the pod _ < VNF >.
2.3.1 if the VNF service function unit on pod _ < VNF > is a packet filtering service unit, configure vnfConfig to the following values: tags options are configured as "[ 'rule _ id', 'sip', 'dip', 'sport', 'dport', 'protocol', 'npkt', 'nbytes' ]", and typeoptions are configured as "[ 'Double', 'String', 'Double' ]". Wherein, tags is as follows: rule _ ID is a rule ID of message filtering, sip is a source IP address of the message, dip is a destination IP address of the message, sport is a source port of the message, dport is a destination port of the message, protocol is a layer 4 protocol number of the message, npkt is the number of flow messages, and nbbytes is the number of bytes of the flow; types is type: double indicates that the content of the tag is of a numeric type, and String indicates that the content of the tag is of a character type.
2.3.2 if the VNF traffic functional unit on pod _ < VNF > is a load balancing traffic unit, configure vnfConfig to the following values: tags options are configured as "[ 'rule _ id', 'dip', 'sip', 'nip', 'csnpkt', 'csnbyte', 'scnpkt', 'scnbyte' ]", and type options are configured as "[ 'Double', 'String', 'Double' ]". Wherein, tags is as follows: rule _ ID is a rule ID of load balancing, dip is a destination IP address of a message, sip is a source IP address of the message, nip is the destination IP address of load balancing, csnpkt is the number of traffic messages sent by a client to a server, csnbyte is the number of bytes of the traffic messages sent by the client to the server, scnpkt is the number of the traffic messages sent by the server to the client, and scnbyte is the number of bytes of the traffic messages sent by the server to the client; types is type: double indicates that the content of the tag is of a numeric type, and String indicates that the content of the tag is of a character type.
2.3.3 if the VNF business function on pod _ < VNF > is a Network Address Translation (NAT) business unit, configure vnfConfig to the following values: tags options are configured as "[ 'rule _ id', 'vip', 'sip', 'tip', 'csnpkt', 'csnbyte', 'scnpkt', 'scnbyte' ]", and type options are configured as "[ 'Double', 'String', 'Double' ]". Wherein, tags is as follows: rule _ ID is NAT rule ID, vip is a target IP address before address conversion, sip is a source IP address of a message, tip is a target IP after address conversion, csnpkt is the number of flow messages sent by a client to a server, csnbyte is the number of bytes of the flow messages sent by the client to the server, scnpkt is the number of the flow messages sent by the server to the client, and scnbyte is the number of bytes of the flow messages sent by the server to the client; types is type: double indicates that the content of the tag is of a numeric type, and String indicates that the content of the tag is of a character type.
2.3.4 if the VNF service functional unit on the pod _ < VNF > is a deep packet inspection service unit, configure vnfConfig to the following values: the tags option is configured as "[ ' bytes _ send ', ' bytes _ rcvd ', ' host ', ' ifid ', ' packets _ send ', ' packets _ rcvd ', ' protocol ', ' l4 protocol '," and the type option is configured as "[ ' Double ', ' String ', ' Double ', ' String ' ] '. Wherein, tags is as follows: bytes _ send is the number of bytes of the sent message, bytes _ rcvd is the number of bytes of the received message, host is the host name, ifid is the host network card name, packets _ send is the number of the sent message, packets _ rcvd is the number of the received message, protocol is the protocol name of the application layer, and l4proto is the transport layer protocol name; types is type: double indicates that the content of the tag is of a numeric type, and String indicates that the content of the tag is of a character type.
2.3.5 if a VNF business functional unit on pod _ < VNF > is an intrusion prevention function business unit, configure vnfConfig to the value: tags options are configured as "[ 'action', 'rule _ info', 'sip', 'sport', 'dip', 'dport', 'protocol' ]", and type options are configured as "[ 'String', 'String' ]". Wherein, tags is as follows: action is the action taken by the intrusion prevention function on the message matched with the rule, rule _ info is the description of the intrusion prevention function on the rule matched with the message, sip is the source IP address of the message, sport is the source port of the message, dip is the destination IP address of the message, dport is the destination port of the message, and protocol is the layer 4 protocol number of the message; types is type: string indicates that the content of the tag is of character type.
2.3.6 if the VNF business function unit on pod _ < VNF > is a protocol identification business unit, configure vnfConfig to the following values: tags options are configured as "[ ' protocol name ', ' appname ', ' sip ', ' dip ', ' sport ', ' dport ', ' protocol ' ]", and type options are configured as "[ ' String ', ' Double '," Double ' ] ". Wherein, tags is as follows: the protocol name is the identified protocol type, the appname is the identified application name, the sip is the source IP address of the message, the dip is the destination IP address of the message, the sport is the source port of the message, the dport is the destination port of the message, and the protocol is the layer 4 protocol number of the message; types is type: double indicates that the content of the tag is of a numeric type, and String indicates that the content of the tag is of a character type.
2.3.7 if the VNF business function unit on pod _ < VNF > is a Web application protection (WAF) business unit, configure vnfConfig to the following values: tags options are configured as "[ ' domain ', ' method ', ' attribute ', ' attacktype ', ' attacker ', ' attackinfo ', ' uri ', ' scheme ' ]", and types options are configured as "[ ' String ', ' etc. ]. Wherein, tags is as follows: domain is a domain name of web application attacked by waf, method is a request method of http used in the waf attack, attacktype is the type of the waf attack, attacker is a client ip address sending the waf attack, attackinfo is specific content of the waf attack, uri is uri data of http, and scheme is http or https; types is type: string indicates that the content of the tag is of character type.
2.4 configuring resource configuration file resAdmConfig of resource centralized allocation module, the method is:
2.4.1 configure the timing database URL as "http:/< timing data IP address >: port number >, with the default values: http://127.0.0.1: 8086.
2.4.2 set the time series dataBase name dataBase to "nfvMonitor".
2.4.3 set the frequency searchInterval of querying the database to be 300 seconds or more, with a default value of 300 seconds.
2.4.4 set the time offset timeOffset of the query database to be 600 seconds or more, with a default value of 600 seconds.
2.4.5 set the CPU warning threshold CPU UTSD to between 60% and 80% with a default value of 80%.
2.4.6 the memory warning threshold memTSD is set to 60% to 80% with a default value of 80%.
And thirdly, a monitoring object discovery module, a performance monitoring index generation module, a service monitoring index generation module, a monitoring index unified acquisition module and a resource centralized distribution module of the virtual network function unified monitoring platform work in parallel and are matched with each other to monitor the CPU use condition, the memory use condition and the service operation condition of the Pod, generate monitoring index metadata, collect the metadata, complement a management label, store the metadata in a time sequence database, and establish a service-performance model and generate a new resource distribution scheme when a monitoring domain value alarm occurs. The method is that a monitored object discovery module generates a file of a monitored object according to the flow of the step 3.1; meanwhile, the performance monitoring index generating module generates performance monitoring index metadata according to the flow of the step 3.2; meanwhile, the service monitoring index generating module generates service monitoring index metadata according to the flow of the step 3.3; meanwhile, the monitoring index unified acquisition module acquires the service index and the performance index according to the 3.4-step flow and stores the service index and the performance index into a time sequence database; meanwhile, the resource centralized allocation module alarms according to the process monitoring threshold value of the 3.5 steps, establishes a model and generates a resource allocation scheme.
3.1 the monitoring object discovery module obtains Node and Pod objects of k8s by using the API interface (version client-java-API-8.0.0) of k8s, generates Node monitoring object files and Pod monitoring object files, and stores them in classification.
The method comprises the following specific steps:
3.1.1 monitor object discovery module initialization. The method comprises the following steps:
3.1.1.1Node object discovery module reads parameters from the monitored object configuration file autoDisConfig: scheduleInterval, K8sConfigPath, K8SNamespace, monitorLabel, pwd _ node _ targets, pwd _ pod _ targets.
3.1.1.2Node object discovery Module initializes the Node object control list nodeTargets to be empty, making the list length nodeListLen, each entry of nodeTargets is a "key-value" structure, where "key" is the IP address of Node and "value" is three flag characters: "I" indicates initialization, "E" indicates existing, "A" indicates to be added.
3.1.1.3Pod object discovery module initializes Pod object control list podtarget to null, making the list length podListLen, each item of podTargets is a "key-value" structure, where "key" is the string concatenation of the Pod's belonging Service Function Chain (SFC) name and the Pod's IP address, and the content of "value" is the same as nodeTargets, also three flag characters: "I" indicates initialization, "E" indicates existing, "A" indicates to be added.
3.1.2Node object discovery module records start time t1 ═ system time;
3.1.3Node object discovery Module updates nodeTargets. The method comprises the following steps:
3.1.3.1 judging whether the nodeTargets contains a key-value pair (when the nodeTargets is operated for the first time, the nodeTargets does not contain any key-value pair, and when the nodeTargets is operated for the second time, the nodeTargets is not operated for the second time), if the nodeTargets contains the key-value pair, turning to 3.1.3.2, otherwise, turning to 3.1.3.3.
3.1.3.2 set the "value" of all "keys" of nodeTargets to "I".
3.1.3.3 calls listNode function in API of k8s to get Node object list nodeList, nodeList list has nodeListLen entries, wherein, the x-th entry is represented by nodeListItem [ x ], x is more than or equal to 1 and less than or equal to nodeListLen, the content of nodeListItem [ x ] is the IP address of x-th Node.
3.1.3.4 let nodeList number x be 1;
3.1.3.5 obtaining nodeList item x of nodeList from nodeList, if nodeList item x exists in the ' key ' of nodeTargets, putting ' value ' E ' of the ' key ' nodeList item x of nodeTargets, turning to 3.1.3.6; otherwise, add to nodeTargets with nodeListItem [ x ] as the new "key", place "value" to "A" for "key" nodeListItem [ x ], go to 3.1.3.6.
3.1.3.6 if x < nodelislen, let x ═ x +1, turn 3.1.3.5; otherwise, go to 3.1.4.
3.1.4Pod object discovery module updates the podtarges variable. The method comprises the following steps:
3.1.4.1 if the podtarges already contain a "key-value" pair, turn 3.1.4.2. Otherwise, turn to 3.1.4.3.
3.1.4.2 sets the "value" of all "keys" of the podtarges to "I".
3.1.4.3 calling listNamespacedPod function in API of k8s to get Pod object list podList, there are podListLen entries in the podList list, where the y-th entry is represented by podListItem [ y ], y is greater than or equal to 1 and less than or equal to podListLen, podListItem [ y ] is a structure body, including: the name SFC of the SFC to which the Pod belongs, the position of the SFC to which the Pod belongs, the name of the Node to which the Pod belongs, the IP address ipAddr of the Pod, the number of CPU cores allocated by the Pod and the size of the memory allocated by the Pod.
3.1.4.4 let podList number y equal to 1.
3.1.4.5 obtaining the y-th item podList item [ y ] of podList from podList, if the character string formed by splicing the SFC field of podList item [ y ] and the ipAddr field already exists in the 'key' of podTtarget, then setting the 'value' corresponding to the 'key' podList item [ y ] of podTtarget to 'E', and turning to 3.1.4.6; otherwise, add to podtarget with the SFC field + iPAdr field of podListItem [ y ] as the new "key", put "A" in the "value" of the "podLisstItem [ y ], go to 3.1.4.6.
3.1.4.6 if y < podListLen, let y be y +1, turn 3.1.4.5; otherwise, go to 3.1.5.
3.1.5Node object discovery module creates or deletes Node monitoring object file, the method is:
3.1.5.1 setting nodetargets number m to 1;
3.1.5.2 obtains the m-th item nodeTargetItem [ m ] of nodeTargets from nodeTargets, m is not less than 1 and not more than nodeTargetLen, nodeTargetLen is the length of nodeTarget list.
3.1.5.3 if the 'value' of the node targetitem [ m ] is 'A', a node object monitoring file is newly created, the file name is < node targetitem [ m ] >. yml, stored in the directory pwd _ node _ targets, the file content includes: "targets: nodeTargetItem [ m ] "(target is the label of the collection object which can be identified by the monitoring index uniform collection module, nodeTargetItem [ m ] is the IP address of the mth Node, and the monitoring index uniform collection module sends a pull performance monitoring index request to the IP), turning to 3.1.5.4; if the "value" of the nodeTargetItem [ m ] is "I", deleting the file < nodeTargetItem [ m ] >. yml under the directory pwd _ node _ targets, and deleting nodeTargetItem [ m ] in nodeTargets, go to 3.1.5.4; if the "value" of nodeTargetItem [ m ] is "E", the file is retained, go to 3.1.5.4.
3.1.5.4 if m < nodeTargetLen, let m be m +1, turn 3.1.5.2; otherwise, go to 3.1.6.
3.1.6Pod object discovery module creates or deletes Pod monitoring object file, the method is:
3.1.6.1 let the number n of podtarges be 1;
3.1.6.2 the nth podtargetItem [ n ] of the podTargets is obtained from podTargets, n is 1. ltoreq. n.ltoreq.podtargetLen, which is the length of the podTargets list.
3.1.6.3 if the "value" of the podtagetsutem [ n ] is "A", a pod monitoring object file is newly created with a file name of < podtagetsutem [ n ] >. yml and stored in the directory pwd _ pod _ targets. The file content includes management tags: "targets: IP address of Pod, node: name of Node, pod: the name of Pod, sfc: name of SFC, sfcIndex: SFC position, CPU: core number of CPU, Mem: (target is an acquisition object label which can be identified by the monitoring index uniform acquisition module, the monitoring index uniform acquisition module sends a pulling service monitoring index request to the IP, and node, pod, sfc, sfcndex, CPU and Mem complete the service monitoring index in a management label form) is converted to 3.1.6.4; if the "value" of the podtargettsitem [ n ] is "I", delete the file < podtargettsitem [ n ] >. yml, and delete podtargettsitem [ n ] in podtarges, go to 3.1.6.4; if the "value" of the podtargetItem [ n ] is "E", the file is retained, leading to 3.6.1.4.
3.1.6.4 if n < podTargetLen, let n be n +1, turn 3.1.6.2; otherwise, go to 3.1.7.
3.1.7node object discovery module reads system time, let t2 be system time, if t2-t1 is not less than scheduleelnterval, turn to 3.1.2; otherwise, go to 3.1.7.
3.2 Performance monitoring index Generation Module uses open source software cAdvisor. The cAdviror reads related files on the Node host regularly, and takes each Docker as an object to count the occupation time and the memory usage of the CPU of the Docker container, and generate performance monitoring index metadata (including the CPU usage condition and the memory usage condition) for being pulled by the monitoring index unified acquisition module. Because the invention stipulates that only one VNF service functional unit, namely a Docker container, is arranged in each Pod, the CPU occupation time and the memory usage of the Docker container are equal to the resource consumption condition of the Pod where the VNF service functional unit is positioned.
3.3 the business monitoring index generation module receives the business monitoring index metadata character string pushed by the VNF business function unit in the pod _ < VNF >, analyzes, caches, summarizes and counts, generates the business monitoring index metadata, and includes: and monitoring the metadata of the indexes and the metadata of the indexes of the module running state by the service running state.
The method comprises the following specific steps:
3.3.1 the service index metadata parsing module reads the service monitoring index configuration file vnfConfig, obtains tags and types, makes tagsLen be the length of tags, and the length of types be the same as tags, initializes the mapping list metaDataMap of monitoring index metadata, the length of metaDataMap is the same as tags, each item of metaDataMap is a "key-value" structure, where "key" is a metadata tag, and "value" is a metadata type. The specific method comprises the following steps:
3.3.1.1 let the tag number p equal to 1.
3.3.1.2 obtaining the pth item of tags [ p ] from tags, obtaining the pth item of type [ p ] from type, 1 ≦ p ≦ tagsLen. With tags [ p ] as "key" and types [ p ] as "value" as "key-value" structure, the "key-value" structure is added to metaDataMap.
3.3.1.3 if p < tagsLen, let p ═ p +1, turn 3.3.1.2, otherwise get initialized metaDataMap, turn 3.3.2.
3.3.2 initialize the service monitoring index buffer variable vnf _ buffer _ count to be empty, which is a "key-value" structure, where "key" is the string vnfTagContest of the service monitoring index metadata combination, and "value" includes two fields: a service monitoring index statistic value count and a system timestamp. Meanwhile, setting a state variable restful _ up of the service index metadata analysis module to 1 (indicating that the service index metadata analysis module is in a working state and waiting for the service function unit to push data).
3.3.3 the service index metadata parsing module monitors whether a service monitoring index metadata character string is received from the VNF service function unit, if so, the step 3.3.4 is carried out; if not, go to 3.3.7.
3.3.4 service index metadata parsing module parses service monitoring index metadata character string, the method is:
3.3.4.1 receives the service monitoring index metadata character string pushed by the VNF service function unit from the restFul data port, i.e. the metadata list json, whose length is json len.
3.3.4.2 initialize the monitor indicator metaData list metaData to null, where each entry of metaData is a "key-value" structure, the "key" is the metaData tag vnfTag, the "value" is the metaData content, and metaDataLen is the metaData list length.
3.3.4.3, json number q is 1.
3.3.4.4 extracts the q-th item json q of json, which is a "key-value" structure, where "key" is the label of the business monitoring index metaData and "value" is the content of the business monitoring index metaData, and if "value" of json q in the metaDataMap is "String", join json q to metaData, go to 3.3.4.6.
3.3.4.5 if json [ q ] has a "value" of "Double" in metaDataMap ("Double" indicates that the content of the tag is a numeric value and not a management tag for metadata), go directly to 3.3.4.6.
3.3.4.6 if q < jsonLen, let q be q +1, go to 3.3.4.4, otherwise go to 3.3.5.
3.3.5 the service index metadata parsing module buffers the service monitoring index metadata to a service monitoring index buffer variable vnf _ buffer _ count, the method is:
3.3.5.1, let the serial number r of metaData be 1 and let vnfTagContest be null.
3.3.5.2 obtains the r-th entry metaData [ r ] from metaData, merges metaData [ r ] with vnftagContest, and saves to vnftagContest.
3.3.5.3 if r < metaDataLen, let r be r +1, go to 3.3.5.2, otherwise go to 3.3.5.4.
3.3.5.4, if vnf _ buffer _ count already contains "key" vnftagtest, add 1 to the service monitoring index statistic count of the original vnf _ buffer _ count "value", and replace the timestamp field of the original "value" with the system current timestamp, turning to 3.3.6. If vnf _ buffer _ count does not contain the "key" vnftagContest, the vnftagContest is used as a new "key" to be added to the vnf _ buffer _ count, the count field corresponding to the "value" is set to be 1, the timestamp field is used as the timestamp of the current system, and then the operation is turned to be 3.3.6.
3.3.6 the service index issuing module monitors whether a 'pull monitoring index request' sent by the monitoring index unified collecting module is received, if so, the step 3.3.7 is switched to; if not, go to step 3.3.3.
3.3.7 the service index issuing module calls MetricFamilysamples function (version io. Prometheus-0.8.0) of API of Prometheus, converts variable restful _ up into service monitoring index generating module status index, issues to the monitoring index uniform collecting module, the index includes label: VNF name, index value 1, index name state _ up. The index tag and the index value form metadata of the state index of the module.
3.3.8, the service index issuing module judges whether vnf _ buffer _ count is empty, if not, go to step 3.3.9; otherwise, go to step 3.3.3.
3.3.9, the service index issuing module calls a MetricFamilysamples function of the API of Prometheus, converts the variable vnf _ buffer _ count into a service operation monitoring index, and issues the service operation monitoring index to the monitoring index unified collecting module, wherein the index includes a label: vnfTagContest, index value, namely service processing counting value, and index name is detail _ count. And the index label and the index value form service operation monitoring index metadata.
3.3.10, the service index issuing module empties the service monitoring index buffer vnf _ buffer _ count, and goes to step 3.3.3.
3.4 the monitoring index uniform acquisition module adopts open source software Prometheus. The monitoring index unified acquisition module acquires the monitoring indexes according to the following procedures:
3.4.1 reading the Node monitoring object file generated in the step 3.1, if the Node monitoring object file exists, switching to 3.4.2; if the Node monitoring object file does not exist (namely the Node monitoring object file is not generated), 3.4.3 is converted;
3.4.2 sequentially reads the targets fields of N Node monitoring object files (IP addresses of Node monitoring objects), respectively sends requests of pulling monitoring indexes to the N IP addresses, collects the performance monitoring index metadata generated in step 3.2 from the performance monitoring index generation module, writes the performance monitoring index metadata into the time sequence database, and changes to 3.4.4.
3.4.3, after waiting for the scratch _ interval second, turning to 3.4.1 to read the Node monitoring object file again;
3.4.4 reading the Pod monitoring object file generated in the step 3.1, and if the Pod monitoring object file exists, turning to 3.4.5; if the Pod monitoring object file does not exist (namely, the Pod monitoring object file is not generated), turning to 3.4.1;
3.4.5 sequentially reading the target fields (IP addresses of the Pod monitoring objects) of the M Pod monitoring object files, respectively sending a request of pulling the monitoring index to the M IP addresses, collecting the service operation monitoring index metadata and the module operation state index metadata generated in the step 3.3 from the service monitoring index generation module, and converting to 3.4.6.
And 3.4.6, complementing the management label in the Pod monitoring object file into the module running state index metadata, writing the label running state monitoring index metadata added with the label into a time sequence database, and turning to 3.4.7. (because the state variable restful _ up of the service index metadata analysis module is set to be in a working state when the module is started, when the monitoring index uniform acquisition module sends a 'pull monitoring index' request to the service index generation module, the state index metadata of the module can be collected, wherein the state index metadata comprises the name of a service function unit);
3.4.7 if the service monitoring index metadata (including the character string of the service monitoring index metadata combination) is not empty, completing the management tag in the Pod monitoring object file into the service monitoring index metadata, writing the service monitoring index metadata added with the tag into a time sequence database, and turning to 3.4.1; otherwise, directly turn to 3.4.1. (since the service operation monitoring index needs to be generated in the service processing process of the service function unit, if the service function unit does not push the service monitoring index metadata string, when the monitoring index unified collection module sends a "pull monitoring index" request, the service monitoring index metadata will not be collected).
3.5 the resource centralized allocation module obtains the monitoring index metadata from the time sequence database, when the CPU utilization rate or the memory utilization rate exceeds the threshold set by the user, a service-performance model is established, a resource allocation scheme is dynamically generated, the resource allocation scheme is output to the user, and data support is provided for the reasonable allocation of Pod resources by the user. The method comprises the following steps:
3.5.1 resource centralized allocation module initialization, the method is:
3.5.1.1 "Business-Performance" model building Module reads parameters from the resource configuration File resAdmConfig: URL, dataBase, searchInterval, timeOffset, cputSD, memTSSD.
3.5.1.2 the model building module of "business-performance" calls the connect function (version inflixdb-java-2.19 and above) of API interface of inflixDB to build database connection.
3.5.2 "business-performance" model building module records current system time t3, and initializes three control variables of resource allocation scheme, business monitoring index indication variable, performance monitoring index indication variable, the method is:
3.5.2.1 resource allocation scheme resDis is initialized to null, resDis is a "key-value" list, each entry is a "key-value" structure, representing the resource configuration of a Pod, where a "key" includes two fields: the name of the SFC and the name of the Pod, the "value" includes two fields: the number of cores of the CPU and the memory Mem size.
3.5.2.2 initializing the service operation monitoring index to indicate that the variable vnfIndex is null, the vnfIndex is a "key-value" list, each entry is a "key-value" structure, where a "key" includes two fields: the name of the SFC and the name of the Pod, the "value" is the rate at which the VNF function processes traffic.
3.5.2.3 the initialized Performance monitoring indicator indicates that the variable perfIndex is null, perfIndex is a list of "key-values", each entry is a "key-value" structure, where a "key" includes three fields: the name of the SFC, the name of the Pod, the location of the SFC, the "value" includes two fields: and the CPU utilization rate and the memory Mem utilization rate of the VNF business function unit.
3.5.3, the method for updating resDis by the service-performance model establishing module is as follows:
3.5.3.1 calls the Influxdb interface to inquire the state _ up table in the time sequence database, groups the state _ up table with the SFC name and the Pod, calls the aggregation class function last () of the Influxdb interface to obtain the inquiry result, and the final result includes: the name of the SFC, the name of the Pod, the number of cores of the CPU, and the size of the memory Mem are stored in a resource state list resdiscist (a list obtained by returning a query result to an aggregation function of the infiluxdb interface, and converting the query result into a resDis key value structure by using a temporary variable resdiscist), and each element of the resdiscist includes four fields, that is: the name SFC of the SFC, the name Pod of the Pod, the core number CPU of the CPU, and the memory Mem size Mem, where resdiscist len is the length of the resdiscist list.
3.5.3.2 if resDisListLen <1, go to 3.5.9, otherwise go to 3.5.3.3.
3.5.3.3 let resdiscist number s equal to 1.
3.5.3.4 obtaining resDisList with item s as resDisList item [ s ], storing SFC field and Pod field of resDisList item [ s ] into SFC field and Pod field of resDis 'key' respectively; the CPU field and the Mem field of resDisListItem [ s ] store the "value" of resDis in the CPU field and memory Mem field, respectively.
3.5.3.5 if s < resdisclistlen, let s be s +1, go to 3.5.3.4, otherwise go to 3.5.4.
3.5.4 the "Business-Performance" model building Module updates the CPU domain of perfIndex. The method comprises the following steps:
3.5.4.1 calling an Influxdb interface to query a user _ cpu table in a time sequence database, obtaining a query result in timeOffset seconds in the table, grouping the query result by an SFC name, an SFC position and a Pod, calling an Influxdb aggregation function derivative () to calculate a speed, calling an Influxdb aggregation function mean () to calculate an average value, wherein the final query result comprises: the name of the SFC, the position of the SFC, the name of the Pod, and the CPU utilization rate are stored in a variable CPU state list cpuList, each element of the cpuList is four fields, that is, the name SFC of the SFC, the position sfcIndex of the SFC, the name SFC of the Pod, and the CPU utilization rate CPU, and the cpuListlen is the length of the list cpuList.
3.5.4.2 if cpuListLen <1, go to 3.5.9, otherwise go to 3.5.4.3.
3.5.4.3 specifies the cpuList number t as 1.
3.5.4.4 obtaining that the t-th item of the cpuList is cpuItem [ t ], respectively storing the SFC field, sfcIndex field and Pod field of the cpuItem [ t ] into the SFC field, SFCIndex field and Pod field of the perfIndex 'key', and storing the CPU field of the cpuItem [ t ] into the CPU field of the perfIndex 'value'.
3.5.4.5 if t < cpulislen, let t be t +1, go to 3.5.4.4, otherwise go to 3.5.5.
3.5.5 "Business-Performance" model building Module updates the Mem field of perfIndex. The method comprises the following steps:
3.5.5.1 calling an Influxdb interface to query a use _ memory table in a time sequence database, obtaining a query result in timeOffset seconds in the table, grouping the query result by an SFC name, an SFC position and a Pod, calling an Influxdb aggregation function derivative () to calculate a speed, then calling an Influxdb aggregation function mean () to calculate an average value, wherein the final query result comprises: the name of the SFC, the location of the SFC, the name of the Pod, and the memory utilization rate are stored in a memory state list memList, each element of the memList is four fields, that is, the name SFC of the SFC, the location sfcIndex of the SFC, the name SFC of the Pod, and the memory utilization rate mem, and the memList len is the length of the memList list.
3.5.5.2 if memListLen <1, go 3.5.9, otherwise go 3.5.5.3.
3.5.5.3 let memList number u equal to 1.
3.5.5.4 obtaining the u-th item of memList as memItem [ u ], updating the Mem field "value" of the "key" of the memItem [ u ] of the variable perfIndex to the Mem field of the memItem [ u ].
3.5.5.5 if u < memListLen, let u be u +1, go to 3.5.5.4, otherwise go to 3.5.6.
3.5.6 model establishing module of service-performance updates vnfIndex, the method is:
3.5.6.1, calling an inflixdb interface to query a detail _ count table in a time sequence database, obtaining a query result in timeOffset seconds in the table, grouping the query result with SFC name and Pod, calling an inflixdb aggregation function derivative () to calculate a speed, then calling an inflixdb aggregation function mean () to calculate an average value, wherein the final query result comprises SFC name, Pod name and service processing speed, and is stored in a service processing state list vnfList, each element of the vnfList comprises three fields, namely name SFC of SFC, name Pod of Pod, service processing speed bpRate, and vnfListLen is the length of the vnfList list.
3.5.6.2 if vnfListLen <1, go 3.5.9, otherwise go 3.5.6.3
3.5.6.3 make vnfList number v equal to 1.
3.5.6.4 get vnfList with the v-th item as vnfItem [ v ], store the SFC field and the Pod field of vnfItem [ v ] into the SFC field and the Pod field of vnfIndex "key", respectively, and store the bpRate field of vnfItem [ x ] into the vnfIndex "value" field.
3.5.6.5 if v < vnfListLen, let v ═ v +1, go 3.5.6.4, otherwise go 3.5.7.
3.5.7 resource allocation scheme generating module finds out the warning of the over-range value and updates resDis, the method is:
3.5.7.1 has the perfIndex variable index numbered w 1.
3.5.7.2 if the performance monitoring indicator indicates that the CPU field of the "value" of the w-th item of the variable perfIndexItem [ w ] exceeds the CPU SD, go to 3.5.7.3, otherwise go to 3.5.7.4.
3.5.7.3 the CPU field that records the "value" of perfIndexItem [ w ] is tsdCPU; marking the character string of 'SFC position' deleted by the 'key' of perfIndexItem [ w ] as Pod _ tsdCPU, and finding the Pod capable of balancing CPU resources from perfIndex by the following method:
3.5.7.3.1 let perfIndex number w1 be 1 and CPU field value comparison variable minCPU be 1.0 x 106
3.5.7.3.2 if the SFC field of perfIndexItem [ w ] is equal to the SFC field of perfIndex [ w1], and the SFCIndex field of perfIndexItem [ w ] is smaller than the SFCIndex field of perfIndex [ w1], and minCPU is smaller than the CPU field of perfIndex [ w1], go to 3.5.7.3.3, otherwise go to 3.5.7.3.4.
3.5.7.3.3 designates minCPU as a cpu field of perfIndex [ w1], and records the "key" deleted "SFC position" character string of perfIndex [ w1] as pod _ minCPU.
3.5.7.3.4 if w1< perfIndexLen, let w1 ═ w1+1, go to 3.5.7.3.2, otherwise go to 3.5.7.3.5.
3.5.7.3.5 if minCPU is not equal to 1.0 x 106And minCPU is less than CPU SD, go to step 3.5.7.3.6, otherwise, go to step 3.5.10
3.5.7.3.6 let the total CPU to be allocated cpussum be tsdCPU + minCPU.
3.5.7.3.7 takes an entry in the list vnfIndex for which the "key" is pod _ tsdCPU, and let tsdCPUVNF be the "value" of the entry.
3.5.7.3.8 takes an entry in the list vnfIndex with "key" as pod _ minCPU, let minCPUVNF be the "value" of the entry.
3.5.7.3.9 update the CPU field of the resDis variable to pod _ tsdCPU "Key" corresponds to "value
Figure BDA0003001275680000241
3.5.7.3.10 pod _ minCPU updating resDis variableThe CPU field of the "key" corresponding to the "value" is
Figure BDA0003001275680000242
3.5.7.4 if the Mem field of the "value" of perfIndexItem [ w ] exceeds memTSD, go to 3.5.7.5, otherwise go to 3.5.7.6.
3.5.7.5 the Mem field recording the "value" of perfIndexItem [ w ] is tsdMem; after the "SFC position" character string is deleted by the "key" of perfIndexItem [ w ], the "SFC position" character string is marked as Pod _ tsdMem, and Pod which can balance memory resources is found from the variable perfIndex, and the method is as follows.
3.5.7.5.1 let perfIndex number w2 be 1 and internal memory value comparison variable minMem be 1.0 x 106
3.5.7.5.2 if the SFC field of perfIndexItem [ w ] is equal to the SFC field of perfIndex [ w2], and the SFCIndex field of perfIndexItem [ w ] is smaller than the SFCIndex field of perfIndex [ w2], and minMem is smaller than the Mem field of perfIndex [ w2], roll 3.5.7.5.3, otherwise roll 3.5.7.5.4.
3.5.7.5.3, minMem is defined as pod _ minMem by deleting the "SFC position" character string from the "key" of perfIndex [ w2] in the Mem field of perfIndex [ w2 ].
3.5.7.5.4 if w2< perfIndexLen, let w2 ═ w2+1, go to 3.5.7.5.2, otherwise go to 3.5.7.5.5.
3.5.7.5.5 if minMem is not equal to 1.0 x 106And minMem is less than memTSD, go to 3.5.7.5.6, otherwise, go to step 3.5.10.
3.5.7.5.6 let the total amount of memory to be allocated, memSum, tsdMem + minMem.
3.5.7.5.7 takes an item in the list vnfndex with "key" pod _ tsdMem, let tsdmemf vnf be the "value" of the item.
3.5.7.5.8 takes an entry in the list vnfIndex with "key" pod _ minMem, let minMemVNF be the "value" of the entry.
3.5.7.5.9 update the pod _ tsdMem "key" of the resDis variable to "value" with the mem field
Figure BDA0003001275680000251
3.5.7.5.10 update pod of resDis variableThe mem field of a _ minMem "key" corresponding to a "value" is
Figure BDA0003001275680000252
3.5.7.6 if w < perfIndexLen, let w be w +1, go to 3.5.7.2, otherwise go to 3.5.8.
3.5.8 resource allocation scheme generating module outputs resource allocation scheme resDis, the method is:
3.5.8.1 let the sequence number z of the item in resDis equal to 1.
3.5.8.2 obtains the z-th item resDisItem [ z ] of resDis, and outputs the "key" and "value" of resDisItem [ z ] to the user.
3.5.8.3 if z < resDisLen, z ═ z +1, go to 3.5.8.2, otherwise go to 3.5.9;
3.5.9 records the current system time t4, if t4-t3 is equal to searchInterval, turn to 3.5.2, otherwise, turn to 3.5.9.
3.5.10 output "original resources are not enough, cannot be allocated between the Pod, and new resources need to be added to the SFC", and end.
The invention can achieve the following technical effects:
1) the resource centralized distribution module of the virtual network function unified monitoring platform constructed in the first step of the invention guides resource distribution according to the monitoring index, uses the monitoring data, and establishes a dynamic mapping model for the distribution process of CPU and memory resources and the service processing process, namely, when the service volume is increased, the distributed resources are also adaptively increased, on the premise of ensuring that the service is successfully completed by the upstream node of the service function chain, the resources are properly reduced from the downstream node, and the problem that the resource can not be allocated from the global perspective when the domain-exceeding alarm occurs is solved. Under the condition of the limitation of the quantity of CPU and memory resources of the whole SFC, the resource allocation process is more reasonable.
2) The invention provides a technical framework for unified access monitoring of different types of VNF service functional units, provides a unified restFul data interface for 7 different types of VNF service functional units, receives and analyzes service monitoring index metadata, and generates a service monitoring index to enrich the monitoring index types. Meanwhile, a new management label is added to the service monitoring index and the performance monitoring index, and the association between different monitoring indexes is automatically established based on the same management label. The technical architecture can be suitable for a VNF unit with a customized function and a VNF unit with a standard function to be accessed to a monitoring platform.
3) The VNF used by the invention pushes the service monitoring index metadata and the monitoring index unified acquisition module pulls the mixed mode of the service monitoring indexes. The push/pull hybrid mode + caching mechanism reduces the network resources of the platform occupied by the monitoring traffic. In the process from pushing to pulling, the invention uses the service monitoring index metadata buffer area, when the service monitoring indexes with the same label enter the data buffer area, the data buffer area completes the classification statistics of the service monitoring indexes, compared with a single service monitoring index which is pulled by the monitoring index uniform acquisition module, after the two indexes are combined, the number of the pulled index bytes is reduced by about half compared with the number of the pulled index bytes of the single service monitoring index.
4) The monitoring object discovery module provided by the invention performs logical grouping on the monitoring objects by initializing two variables of Namespace and monitoreLabel of k8s, so that on one hand, the monitoring index unified acquisition module cannot become a performance bottleneck of a system. In practical deployment, all VNF business functional units on one service function chain are usually divided into a Namespace, and the combination of Namespace and monitorLabel of k8s can evolve the monitoring object as: all VNFs on the whole SFC, partial VNFs on a certain SFC, partial VNFs on a different SFC, all VNFs on a different SFC. The monitoring index uniform acquisition module only receives the monitoring object generated by the monitoring object discovery module, so that the load balancing target is realized; on the other hand, the monitored object discovery module interacts with the k8s platform to discover the monitored object, which is beneficial to reducing the work of the administrator in the monitoring process and simultaneously ensuring that the monitored object timely joins or exits the monitoring unified monitoring platform.
5) The modules provided by the invention are communicated in an asynchronous mode, the modules are not influenced mutually, and the modules can be upgraded and replaced independently.
Drawings
FIG. 1 is a logical block diagram of an exemplary monitoring application of the k8s platform;
FIG. 2 is a logical structure diagram of the virtual network function unified monitoring platform of the present invention;
fig. 3 is an overall flow chart of the present invention.
Detailed Description
The invention, as shown in fig. 3, includes the following steps:
firstly, a virtual network function unified monitoring platform is constructed.
1.1, constructing a virtual network function running platform and deploying a service function chain.
Constructing a k8s platform comprising a Master Node and N Node nodes, wherein a VNF service function unit is deployed in a scheduling basic unit Pod corresponding to k8s and is allocated with corresponding CPU and memory resources; pod belongs to different namespaces of the k8s platform and can be specified by the user. If not, the Pod will belong to the default space, represented by "default". The traffic can go through the Pod in order as agreed to form a service function chain. The Pod where the VNF service functional unit is deployed is named Pod _ < VNF >, where < VNF > is an abbreviation of a name of the VNF service functional unit, and the existing 7 types of VNF service functional units include: the system comprises a message filtering function, a load balancing function, a network address conversion function, a deep message detection function, an intrusion prevention function, a protocol identification function and a Web application protection function. If the Pod _ < vnf > needs to be added into the virtual network function unified monitoring platform, so that the unified monitoring platform collects the service index metadata, the tag "monitored" is added to the Pod _ < vnf > (only the Pod with the tag can be added into the unified monitoring platform). Appointing: each Pod only deploys one Docker container related to the VNF service function, and each Pod only belongs to one service function chain.
On the basis of the original Pod management tag (the name of the Pod) of the k8s platform, 2 new management tags are added for each Pod, namely: the name SFC of the service function chain to which the Pod belongs, and the location SFCIndex of the service function chain to which the Pod belongs.
1.2, constructing a virtual network function unified monitoring platform as shown in fig. 2, and also showing the relationship between the virtual network function unified monitoring platform and the virtual network function operating platform from fig. 2.
The virtual network function unified monitoring platform consists of a monitoring object discovery module, a service monitoring index generation module, a performance monitoring index generation module, a monitoring index unified acquisition module, a resource centralized distribution module and a time sequence database.
And (3) newly building a Pod, named Pod _ dis, at the Master node of the k8s platform, and deploying the monitoring object discovery module and the monitoring index uniform acquisition module at the Pod _ dis.
The monitoring object discovery module is connected with the monitoring index uniform acquisition module and consists of a Node object discovery module, a Pod object discovery module and a monitoring object configuration file autoDisConfig. The autoDisConfig includes 6 fields, which are respectively: frequency scheduleInterval of interface call, location K8sConfigPath of configuration file of K8s, Pod monitor object namespace K8SNamespace (in K8s platform, each Pod belongs to different namespace, monitor object discovery module can discover Pod under specified namespace by reading K8 SNamespace), screening tag monitorelabel of Pod monitor object, storage location pwd _ Node _ target of Node object file, storage location pwd _ Pod _ target of Pod object file. The Node object discovery module is connected with an application program interface of a k8s platform, the Node object discovery module calls an API function listNode of k8s in each schedulolenterval to discover Node objects (making N found together, N being the total number of the Node objects), generates Node monitoring object files for N nodes by taking the IP addresses of the nodes as file names, and makes the Node monitoring object files of the nodes with the IP addresses being the Node IP be < Node IP >. yml. Storing N Node monitoring object files in a directory pwd _ Node _ targets; the Pod object discovery module is connected with an application program interface of the k8s platform, when the Node object discovery module discovers Node objects, the Pod object discovery module calls an API function listnamespaced Pod of k8s to discover Pod objects (M are found in total, M is the total number of Pod objects), and generates a Pod monitoring object file of each Pod by taking the IP addresses of the SFC and the Pod to which the Pod belongs as file names, and the Pod monitoring object file of the Pod of which the IP address is PodIP is made to be < SFC + PodIP >. yml. M Pod monitor object files are stored under the directory pwd _ Pod _ targets.
The performance monitoring index generation module adopts cAdviror (version: V0.35 and above) mentioned in the background technology I, and is connected with the monitoring index uniform acquisition module. The performance monitoring index generation module creates a Pod on each Node, which is named Pod _ prf _ < x >, wherein < x > <1, 2, … N (N nodes need to deploy N cadvisors). The cAdvisor is deployed on the pod _ prf _ < x >, and interacts with the Docker engine to acquire information to generate performance monitoring index metadata (metadata including CPU utilization rate and memory utilization rate indexes) for being pulled by the monitoring index unified acquisition module.
The service monitoring index generation module is deployed in pod _ < vnf >. And the service monitoring index generating module is connected with the restFul data interface of the VNF service functional unit and the monitoring index uniform acquisition module. The system consists of a service index metadata analysis module, a service index release module and a service monitoring index configuration file vnfConfig. vnfConfig includes two domains: service index metadata tag group tags (service index metadata includes a plurality of tags, tags is a name list of the tags), type of service index metadata (includes two values: String indicates that the value of the tag is character type, and Double indicates that the value of the tag is numerical type). The content of vnfConfig is configured according to the type of VNF business functional unit (7 types in total) on pod _ < VNF >. And the service index metadata analysis module is connected with the restFul data interface and the service index publishing module. The service index metadata analysis module monitors a restFul data interface, receives a service monitoring index metadata character string pushed by a VNF service function unit in the pod _ < VNF >, analyzes the character string, caches and generates service monitoring index metadata (including service operation monitoring index metadata and module operation state), and sends the service monitoring index metadata to the service index publishing module. Because the service monitoring index generation module and the VNF service function unit are deployed in the same Pod, the monitoring data flow occupies network resources of a k8s platform by communicating with the VNF service function unit through a local protocol stack and using a cache. The service index issuing module is connected with the service index metadata analyzing module and the monitoring index unified collecting module, monitors a 'pull monitoring index' request of the monitoring index unified collecting module, and issues the service monitoring index metadata to the monitoring index unified collecting module when receiving the 'pull monitoring index' request.
The monitoring index unified acquisition module is connected with the monitoring object discovery module, the time sequence database, the service monitoring index generation module and the performance monitoring index generation module, adopts Prometheus (version V2.16 and above) and is deployed in pod _ dis. The monitoring index acquisition module is provided with a monitoring index acquisition configuration file monConfig, and the monConfig comprises 4 fields which are respectively as follows: the collection cycle scrape _ interval, the storage location pwd _ Node _ targets of the Node object file (i.e. under the "scrape _ configurations" entry of the configuration file of the proceus, one entry of "-jobname" is configured as "Node", the "-files" option of the next entry "file _ sd _ configurations" of "Node" is configured as storage location pwd _ Node _ targets), the storage location pwd _ Pod _ targets of the Pod object file (i.e. under the "scrape _ configurations" entry of the configuration file of the proceus, another entry of "-jobname" is configured as "Pod", the "es" option of the "Pod" next entry "file _ sd _ configurations" is configured as storage location pwd _ Pod _ target), and the database write address is configured as write address. The unified monitoring index acquisition module periodically acquires a monitoring object from the lists of pwd _ node _ targets and pwd _ pod _ targets according to an acquisition period scrape _ interval configured by monConfig, sends a 'pull monitoring index' request to the performance monitoring index generation module, and collects performance monitoring index metadata generated by the performance monitoring index generation module; meanwhile, a pulling monitoring index request is sent to a service monitoring index generation module, and service monitoring index metadata is collected. And adding labels to the performance monitoring index metadata and the service monitoring index metadata, and storing the performance monitoring index metadata and the service monitoring index metadata added with the labels in a time sequence database.
The time sequence database is connected with the monitoring index unified acquisition module and the resource centralized distribution module, is created by using open source software InfluxDB (version V1.7.9 and above), is named as 'nfvMonitor', is deployed on a server outside a k8s platform and is used for storing service monitoring index metadata and performance monitoring index metadata, and each index metadata is independently stored in a form of a table, and comprises the following steps: the system comprises a service monitoring index generation module state index table, a service operation monitoring index detail table, a service function unit CPU use condition table and a service function unit memory use condition table. The state _ up of the state index table of the service monitoring index generation module comprises 9 table entries: a timestamp (i.e., system time generated by a record item), a CPU configuration value (i.e., the number of CPU cores configured by a system administrator for a Pod), a memory configuration value (i.e., the number of memories configured by the system administrator for the Pod), a VNF name (i.e., the name of a VNF service functional unit), a Node name (i.e., the name of a Pod deployed in a Node), a Pod name (i.e., the name of a Pod belonging to the Node), an SFC name (i.e., the name of a service functional chain belonging to the Pod), an SFC location (i.e., the location number of the service functional chain belonging to the Pod), and a start state of a monitoring acquisition module (i.e., whether the monitoring acquisition module is in a start state); the service operation monitoring index detail table detail _ count comprises table entries: a timestamp (i.e., system time generated by the entry), a VNF metadata set (i.e., a tag item of service index metadata of a VNF service function unit configuration file vnfConfig), a Node name (i.e., a name of a Node to which a Pod belongs), a Pod name (i.e., a name of a Pod to which a container belongs), an SFC name (i.e., a name of a service function chain to which a Pod belongs), an SFC location (i.e., a location serial number of the service function chain to which a Pod belongs), and a service processing count value (i.e., a number of service processes); service function unit CPU usage table (i.e. container _ CPU _ user _ seconds _ total table of cAdvisor) user _ CPU, including table entry: a timestamp (i.e., system time generated by the entry), a container tag set (i.e., a system management tag set generated by the k8s platform for the container), a Pod name (i.e., the name of a Pod to which the container belongs), a Node name (i.e., the name of a Node to which the Pod belongs), an SFC name (i.e., the name of a service function chain to which the Pod belongs), an SFC location (i.e., the location number of the service function chain to which the Pod belongs), and a CPU usage value (i.e., time used by the CPU); the service function unit memory usage table (i.e. container _ memory _ usage _ bytes table of cAdvisor) usage _ memory includes entries: a timestamp (i.e., a system time generated by the entry), a container tag set (i.e., a system management tag set generated by the k8s platform for the container), a Pod name (i.e., a name of a Pod to which the container belongs), a Node name (i.e., a name of a Node to which the Pod belongs), an SFC name (i.e., a name of a service function chain to which the Pod belongs), an SFC location (i.e., a location number of the service function chain to which the Pod belongs), and a memory usage value (i.e., a memory usage amount).
The resource centralized distribution module obtains the running state index metadata of the service monitoring index generation module of the Pod from the state _ up table, and obtains the current CPU and memory distribution situation of the Pod; obtaining metadata of the VNF service function unit processing traffic volume rate from a detail _ count table; and obtaining metadata of the CPU utilization rate and the memory utilization rate when the current service is processed from the user _ CPU and the user _ memory table. And establishing a service-performance model according to the 4 index metadata to guide a user to allocate calculation and memory resources.
The resource centralized distribution module and the time sequence database are deployed in the same server, the resource centralized distribution module is connected with the time sequence database, and the resource centralized distribution module is composed of a service-performance model building module, a resource distribution scheme generating module and a resource configuration file resAdmConfig. The resource profile includes 6 fields: a time series dataBase URL (Uniform Resource Locator), a dataBase name dataBase, a frequency searchInterval of querying the dataBase, a time offset timeOffset of querying the dataBase, a CPU warning threshold cpuTSD, and a memory warning threshold memTSD. The service-performance model building module is connected with the time sequence dataBase and the resource allocation scheme generating module, data between the current time and the previous time offset seconds are obtained from a detail _ count table, a user _ CPU table and a use _ memory table respectively according to the frequency searchInterval of the query dataBase in the dataBase under the URL of the time sequence dataBase, and the CPU utilization rate, the memory utilization rate and the service processing rate are calculated; and obtains the number of CPU cores and the current status of memory allocation from the state _ up table. And when the CPU utilization rate exceeds the CPU TSD or the memory utilization rate exceeds the memTSD, establishing a service-performance model, and sending the service-performance model to the resource allocation scheme generation module. The resource allocation scheme generation module is connected with the business-performance model building module, generates a Pod new CPU and memory resource allocation scheme according to the business-performance model received from the business-performance model building module, and sends the CPU and memory resource allocation scheme to a K8S platform administrator. The K8S platform administrator can adjust the CPU and memory resource allocation of the corresponding Pod on the service function chain according to the scheme, and reconfigure the resources by interacting with the K8s platform.
Secondly, initializing a virtual network function unified monitoring platform, wherein the method comprises the following steps:
2.1 configuring a monitoring object configuration file autoDisConfig of the monitoring object discovery module:
2.1.1 setting the frequency schedulelnterval value of the interface calling to be more than or equal to 60 seconds, and setting the default value to be 60 seconds;
2.1.2 set the value of the configuration file Config location k8sConfigPath of k8s to the location of the file Config, the default value "/root/. kube/Config";
2.1.3, setting the value of the Pod monitoring object namespace K8SNamespace as a default value 'default', supporting the setting of a plurality of namespaces, using and separating the namespaces;
2.1.4 setting the value of the screening label monitore Label of the Pod monitoring object to be "monitored", and adding the Pod comprising the label "monitored" into the unified monitoring platform;
2.1.5 setting the storage position pwd _ Node _ targets value of the Node object file as default value "/home/targets/nodes";
2.1.6 sets the storage location pwd _ Pod _ targets value of the Pod object file to the default value "/home/targets/pods".
2.2 configuring a monitoring index configuration file monConfig of the monitoring index uniform acquisition module:
2.2.1 set pwd _ node _ targets value to default value "[ '/home/targets/nodes/. yml' ]";
2.2.2 set pwd _ pod _ targets value to default value "[ '/home/targets/pod/. yml' ]";
2.2.3 configure remote _ write as "-url http: /< time-series database IP address >: < port >/api/v1/prom/write ═ nfvmmonitor ", where < time series database IP address > is the time series database IP address and < port > defaults to 8086;
2.2.4 set the acquisition period script _ interval value to 60 seconds.
And 2.3, configuring a service monitoring index configuration file vnfConfig of the service monitoring index generating module according to the type of the VNF service functional unit on the pod _ < VNF >.
2.3.1 if the VNF service function unit on pod _ < VNF > is a packet filtering service unit, configure vnfConfig to the following values: tags options are configured as "[ 'rule _ id', 'sip', 'dip', 'sport', 'dport', 'protocol', 'npkt', 'nbytes' ]", and typeoptions are configured as "[ 'Double', 'String', 'Double' ]". Wherein, tags is as follows: rule _ ID is a rule ID of message filtering, sip is a source IP address of the message, dip is a destination IP address of the message, sport is a source port of the message, dport is a destination port of the message, protocol is a layer 4 protocol number of the message, npkt is the number of flow messages, and nbbytes is the number of bytes of the flow; types is type: double indicates that the content of the tag is of a numeric type, and String indicates that the content of the tag is of a character type.
2.3.2 if the VNF traffic functional unit on pod _ < VNF > is a load balancing traffic unit, configure vnfConfig to the following values: tags options are configured as "[ 'rule _ id', 'dip', 'sip', 'nip', 'csnpkt', 'csnbyte', 'scnpkt', 'scnbyte' ]", and type options are configured as "[ 'Double', 'String', 'Double' ]". Wherein, tags is as follows: rule _ ID is a rule ID of load balancing, dip is a destination IP address of a message, sip is a source IP address of the message, nip is the destination IP address of load balancing, csnpkt is the number of traffic messages sent by a client to a server, csnbyte is the number of bytes of the traffic messages sent by the client to the server, scnpkt is the number of the traffic messages sent by the server to the client, and scnbyte is the number of bytes of the traffic messages sent by the server to the client; types is type: double indicates that the content of the tag is of a numeric type, and String indicates that the content of the tag is of a character type.
2.3.3 if the VNF business function on pod _ < VNF > is a Network Address Translation (NAT) business unit, configure vnfConfig to the following values: tags options are configured as "[ 'rule _ id', 'vip', 'sip', 'tip', 'csnpkt', 'csnbyte', 'scnpkt', 'scnbyte' ]", and type options are configured as "[ 'Double', 'String', 'Double' ]". Wherein, tags is as follows: rule _ ID is NAT rule ID, vip is a target IP address before address conversion, sip is a source IP address of a message, tip is a target IP after address conversion, csnpkt is the number of flow messages sent by a client to a server, csnbyte is the number of bytes of the flow messages sent by the client to the server, scnpkt is the number of the flow messages sent by the server to the client, and scnbyte is the number of bytes of the flow messages sent by the server to the client; types is type: double indicates that the content of the tag is of a numeric type, and String indicates that the content of the tag is of a character type.
2.3.4 if the VNF service functional unit on the pod _ < VNF > is a deep packet inspection service unit, configure vnfConfig to the following values: the tags option is configured as "[ ' bytes _ send ', ' bytes _ rcvd ', ' host ', ' ifid ', ' packets _ send ', ' packets _ rcvd ', ' protocol ', ' l4 protocol '," and the type option is configured as "[ ' Double ', ' String ', ' Double ', ' String ' ] '. Wherein, tags is as follows: bytes _ send is the number of bytes of the sent message, bytes _ rcvd is the number of bytes of the received message, host is the host name, ifid is the host network card name, packets _ send is the number of the sent message, packets _ rcvd is the number of the received message, protocol is the protocol name of the application layer, and l4proto is the transport layer protocol name; types is type: double indicates that the content of the tag is of a numeric type, and String indicates that the content of the tag is of a character type.
2.3.5 if a VNF business functional unit on pod _ < VNF > is an intrusion prevention function business unit, configure vnfConfig to the value: tags options are configured as "[ 'action', 'rule _ info', 'sip', 'sport', 'dip', 'dport', 'protocol' ]", and type options are configured as "[ 'String', 'String' ]". Wherein, tags is as follows: action is the action taken by the intrusion prevention function on the message matched with the rule, rule _ info is the description of the intrusion prevention function on the rule matched with the message, sip is the source IP address of the message, sport is the source port of the message, dip is the destination IP address of the message, dport is the destination port of the message, and protocol is the layer 4 protocol number of the message; types is type: string indicates that the content of the tag is of character type.
2.3.6 if the VNF business function unit on pod _ < VNF > is a protocol identification business unit, configure vnfConfig to the following values: tags options are configured as "[ ' protocol name ', ' appname ', ' sip ', ' dip ', ' sport ', ' dport ', ' protocol ' ]", and type options are configured as "[ ' String ', ' Double '," Double ' ] ". Wherein, tags is as follows: the protocol name is the identified protocol type, the appname is the identified application name, the sip is the source IP address of the message, the dip is the destination IP address of the message, the sport is the source port of the message, the dport is the destination port of the message, and the protocol is the layer 4 protocol number of the message; types is type: double indicates that the content of the tag is of a numeric type, and String indicates that the content of the tag is of a character type.
2.3.7 if the VNF business function unit on pod _ < VNF > is a Web application protection (WAF) business unit, configure vnfConfig to the following values: tags options are configured as "[ ' domain ', ' method ', ' attribute ', ' attacktype ', ' attacker ', ' attackinfo ', ' uri ', ' scheme ' ]", and types options are configured as "[ ' String ', ' etc. ]. Wherein, tags is as follows: domain is a domain name of web application attacked by waf, method is a request method of http used in the waf attack, attacktype is the type of the waf attack, attacker is a client ip address sending the waf attack, attackinfo is specific content of the waf attack, uri is uri data of http, and scheme is http or https; types is type: string indicates that the content of the tag is of character type.
2.4 configuring resource configuration file resAdmConfig of resource centralized allocation module, the method is:
2.4.1 configure the timing database URL as "http: < timing data IP address >: port number >, with the default values: http://127.0.0.1: 8086.
2.4.2 set the time series dataBase name dataBase to "nfvMonitor".
2.4.3 set the frequency searchInterval of querying the database to be 300 seconds or more, with a default value of 300 seconds.
2.4.4 set the time offset timeOffset of the query database to be 600 seconds or more, with a default value of 600 seconds.
2.4.5 set the CPU warning threshold CPU UTSD to between 60% and 80% with a default value of 80%.
2.4.6 the memory warning threshold memTSD is set to 60% to 80% with a default value of 80%.
And thirdly, a monitoring object discovery module, a performance monitoring index generation module, a service monitoring index generation module, a monitoring index unified acquisition module and a resource centralized distribution module of the virtual network function unified monitoring platform work in parallel and are matched with each other to monitor the CPU use condition, the memory use condition and the service operation condition of the Pod, generate monitoring index metadata, collect the metadata, complement a management label, store the metadata in a time sequence database, and establish a service-performance model and generate a new resource distribution scheme when a monitoring domain value alarm occurs. The method is that a monitored object discovery module generates a file of a monitored object according to the flow of the step 3.1; meanwhile, the performance monitoring index generating module generates performance monitoring index metadata according to the flow of the step 3.2; meanwhile, the service monitoring index generating module generates service monitoring index metadata according to the flow of the step 3.3; meanwhile, the monitoring index unified acquisition module acquires the service index and the performance index according to the 3.4-step flow and stores the service index and the performance index into a time sequence database; meanwhile, the resource centralized allocation module alarms according to the process monitoring threshold value of the 3.5 steps, establishes a model and generates a resource allocation scheme.
3.1 the monitoring object discovery module obtains Node and Pod objects of k8s by using the API interface (version client-java-API-8.0.0) of k8s, generates Node monitoring object files and Pod monitoring object files, and stores them in classification.
The method comprises the following specific steps:
3.1.1 monitor object discovery module initialization. The method comprises the following steps:
3.1.1.1Node object discovery module reads parameters from the monitored object configuration file autoDisConfig: scheduleInterval, K8sConfigPath, K8SNamespace, monitorLabel, pwd _ node _ targets, pwd _ pod _ targets.
3.1.1.2Node object discovery Module initializes the Node object control list nodeTargets to be empty, making the list length nodeListLen, each entry of nodeTargets is a "key-value" structure, where "key" is the IP address of Node and "value" is three flag characters: "I" indicates initialization, "E" indicates existing, "A" indicates to be added.
3.1.1.3Pod object discovery module initializes Pod object control list podtarget to null, making the list length podListLen, each item of podTargets is a "key-value" structure, where "key" is the string concatenation of the Pod's belonging Service Function Chain (SFC) name and the Pod's IP address, and the content of "value" is the same as nodeTargets, also three flag characters: "I" indicates initialization, "E" indicates existing, "A" indicates to be added.
3.1.2Node object discovery module records start time t1 ═ system time;
3.1.3Node object discovery Module updates nodeTargets. The method comprises the following steps:
3.1.3.1 judging whether the nodeTargets contains a key-value pair (when the nodeTargets is operated for the first time, the nodeTargets does not contain any key-value pair, and when the nodeTargets is operated for the second time, the nodeTargets is not operated for the second time), if the nodeTargets contains the key-value pair, turning to 3.1.3.2, otherwise, turning to 3.1.3.3.
3.1.3.2 set the "value" of all "keys" of nodeTargets to "I".
3.1.3.3 calls listNode function in API of k8s to get Node object list nodeList, nodeList list has nodeListLen entries, wherein, the x-th entry is represented by nodeListItem [ x ], x is more than or equal to 1 and less than or equal to nodeListLen, the content of nodeListItem [ x ] is the IP address of x-th Node.
3.1.3.4 let nodeList number x be 1;
3.1.3.5 obtaining nodeList item x of nodeList from nodeList, if nodeList item x exists in the ' key ' of nodeTargets, putting ' value ' E ' of the ' key ' nodeList item x of nodeTargets, turning to 3.1.3.6; otherwise, add to nodeTargets with nodeListItem [ x ] as the new "key", place "value" to "A" for "key" nodeListItem [ x ], go to 3.1.3.6.
3.1.3.6 if x < nodelislen, let x ═ x +1, turn 3.1.3.5; otherwise, go to 3.1.4.
3.1.4Pod object discovery module updates the podtarges variable. The method comprises the following steps:
3.1.4.1 if the podtarges already contain a "key-value" pair, turn 3.1.4.2. Otherwise, turn to 3.1.4.3.
3.1.4.2 sets the "value" of all "keys" of the podtarges to "I".
3.1.4.3 calling listNamespacedPod function in API of k8s to get Pod object list podList, there are podListLen entries in the podList list, where the y-th entry is represented by podListItem [ y ], y is greater than or equal to 1 and less than or equal to podListLen, podListItem [ y ] is a structure body, including: the name SFC of the SFC to which the Pod belongs, the position of the SFC to which the Pod belongs, the name of the Node to which the Pod belongs, the IP address ipAddr of the Pod, the number of CPU cores allocated by the Pod and the size of the memory allocated by the Pod.
3.1.4.4 let podList number y equal to 1.
3.1.4.5 obtaining the y-th item podList item [ y ] of podList from podList, if the character string formed by splicing the SFC field of podList item [ y ] and the ipAddr field already exists in the 'key' of podTtarget, then setting the 'value' corresponding to the 'key' podList item [ y ] of podTtarget to 'E', and turning to 3.1.4.6; otherwise, add to podtarget with the SFC field + iPAdr field of podListItem [ y ] as the new "key", put "A" in the "value" of the "podLisstItem [ y ], go to 3.1.4.6.
3.1.4.6 if y < podListLen, let y be y +1, turn 3.1.4.5; otherwise, go to 3.1.5.
3.1.5Node object discovery module creates or deletes Node monitoring object file, the method is:
3.1.5.1 setting nodetargets number m to 1;
3.1.5.2 obtains the m-th item nodeTargetItem [ m ] of nodeTargets from nodeTargets, m is not less than 1 and not more than nodeTargetLen, nodeTargetLen is the length of nodeTarget list.
3.1.5.3 if the 'value' of the node targetitem [ m ] is 'A', a node object monitoring file is newly created, the file name is < node targetitem [ m ] >. yml, stored in the directory pwd _ node _ targets, the file content includes: "targets: nodeTargetItem [ m ] "(target is the label of the collection object which can be identified by the monitoring index uniform collection module, nodeTargetItem [ m ] is the IP address of the mth Node, and the monitoring index uniform collection module sends a pull performance monitoring index request to the IP), turning to 3.1.5.4; if the "value" of the nodeTargetItem [ m ] is "I", deleting the file < nodeTargetItem [ m ] >. yml under the directory pwd _ node _ targets, and deleting nodeTargetItem [ m ] in nodeTargets, go to 3.1.5.4; if the "value" of nodeTargetItem [ m ] is "E", the file is retained, go to 3.1.5.4.
3.1.5.4 if m < nodeTargetLen, let m be m +1, turn 3.1.5.2; otherwise, go to 3.1.6.
3.1.6Pod object discovery module creates or deletes Pod monitoring object file, the method is:
3.1.6.1 let the number n of podtarges be 1;
3.1.6.2 the nth podtargetItem [ n ] of the podTargets is obtained from podTargets, n is 1. ltoreq. n.ltoreq.podtargetLen, which is the length of the podTargets list.
3.1.6.3 if the "value" of the podtagetsutem [ n ] is "A", a pod monitoring object file is newly created with a file name of < podtagetsutem [ n ] >. yml and stored in the directory pwd _ pod _ targets. The file content includes management tags: "targets: IP address of Pod, node: name of Node, pod: the name of Pod, sfc: name of SFC, sfcIndex: SFC position, CPU: core number of CPU, Mem: (target is an acquisition object label which can be identified by the monitoring index uniform acquisition module, the monitoring index uniform acquisition module sends a pulling service monitoring index request to the IP, and node, pod, sfc, sfcndex, CPU and Mem complete the service monitoring index in a management label form) is converted to 3.1.6.4; if the "value" of the podtargettsitem [ n ] is "I", delete the file < podtargettsitem [ n ] >. yml, and delete podtargettsitem [ n ] in podtarges, go to 3.1.6.4; if the "value" of the podtargetItem [ n ] is "E", the file is retained, leading to 3.6.1.4.
3.1.6.4 if n < podTargetLen, let n be n +1, turn 3.1.6.2; otherwise, go to 3.1.7.
3.1.7node object discovery module reads system time, let t2 be system time, if t2-t1 is not less than scheduleelnterval, turn to 3.1.2; otherwise, go to 3.1.7.
3.2 Performance monitoring index Generation Module uses open source software cAdvisor. The cAdviror reads related files on the Node host regularly, and takes each Docker as an object to count the occupation time and the memory usage of the CPU of the Docker container, and generate performance monitoring index metadata (including the CPU usage condition and the memory usage condition) for being pulled by the monitoring index unified acquisition module. Because the invention stipulates that only one VNF service functional unit, namely a Docker container, is arranged in each Pod, the CPU occupation time and the memory usage of the Docker container are equal to the resource consumption condition of the Pod where the VNF service functional unit is positioned.
3.3 the business monitoring index generation module receives the business monitoring index metadata character string pushed by the VNF business function unit in the pod _ < VNF >, analyzes, caches, summarizes and counts, generates the business monitoring index metadata, and includes: and monitoring the metadata of the indexes and the metadata of the indexes of the module running state by the service running state.
The method comprises the following specific steps:
3.3.1 the service index metadata parsing module reads the service monitoring index configuration file vnfConfig, obtains tags and types, makes tagsLen be the length of tags, and the length of types be the same as tags, initializes the mapping list metaDataMap of monitoring index metadata, the length of metaDataMap is the same as tags, each item of metaDataMap is a "key-value" structure, where "key" is a metadata tag, and "value" is a metadata type. The specific method comprises the following steps:
3.3.1.1 let the tag number p equal to 1.
3.3.1.2 obtaining the pth item of tags [ p ] from tags, obtaining the pth item of type [ p ] from type, 1 ≦ p ≦ tagsLen. With tags [ p ] as "key" and types [ p ] as "value" as "key-value" structure, the "key-value" structure is added to metaDataMap.
3.3.1.3 if p < tagsLen, let p ═ p +1, turn 3.3.1.2, otherwise get initialized metaDataMap, turn 3.3.2.
3.3.2 initialize the service monitoring index buffer variable vnf _ buffer _ count to be empty, which is a "key-value" structure, where "key" is the string vnfTagContest of the service monitoring index metadata combination, and "value" includes two fields: a service monitoring index statistic value count and a system timestamp. Meanwhile, the state variable restful _ up of the service index metadata analysis module is set to 1.
3.3.3 the service index metadata parsing module monitors whether a service monitoring index metadata character string is received from the VNF service function unit, if so, the step 3.3.4 is carried out; if not, go to 3.3.7.
3.3.4 service index metadata parsing module parses service monitoring index metadata character string, the method is:
3.3.4.1 receives the service monitoring index metadata character string pushed by the VNF service function unit from the restFul data port, i.e. the metadata list json, whose length is json len.
3.3.4.2 initialize the monitor indicator metaData list metaData to null, where each entry of metaData is a "key-value" structure, the "key" is the metaData tag vnfTag, the "value" is the metaData content, and metaDataLen is the metaData list length.
3.3.4.3, json number q is 1.
3.3.4.4 extracts the q-th item json q of json, which is a "key-value" structure, where "key" is the label of the business monitoring index metaData and "value" is the content of the business monitoring index metaData, and if "value" of json q in the metaDataMap is "String", join json q to metaData, go to 3.3.4.6.
3.3.4.5 if json [ q ] has a "value" of "Double" in metaDataMap, go directly to 3.3.4.6.
3.3.4.6 if q < jsonLen, let q be q +1, go to 3.3.4.4, otherwise go to 3.3.5.
3.3.5 the service index metadata parsing module buffers the service monitoring index metadata to a service monitoring index buffer variable vnf _ buffer _ count, the method is:
3.3.5.1, let the serial number r of metaData be 1 and let vnfTagContest be null.
3.3.5.2 obtains the r-th entry metaData [ r ] from metaData, merges metaData [ r ] with vnftagContest, and saves to vnftagContest.
3.3.5.3 if r < metaDataLen, let r be r +1, go to 3.3.5.2, otherwise go to 3.3.5.4.
3.3.5.4, if vnf _ buffer _ count already contains "key" vnftagtest, add 1 to the service monitoring index statistic count of the original vnf _ buffer _ count "value", and replace the timestamp field of the original "value" with the system current timestamp, turning to 3.3.6. If vnf _ buffer _ count does not contain the "key" vnftagContest, the vnftagContest is used as a new "key" to be added to the vnf _ buffer _ count, the count field corresponding to the "value" is set to be 1, the timestamp field is used as the timestamp of the current system, and then the operation is turned to be 3.3.6.
3.3.6 the service index issuing module monitors whether a 'pull monitoring index request' sent by the monitoring index unified collecting module is received, if so, the step 3.3.7 is switched to; if not, go to step 3.3.3.
3.3.7 the service index issuing module calls MetricFamilysamples function (version io. Prometheus-0.8.0) of API of Prometheus, converts variable restful _ up into service monitoring index generating module status index, issues to the monitoring index uniform collecting module, the index includes label: VNF name, index value 1, index name state _ up. The index tag and the index value form metadata of the state index of the module.
3.3.8, the service index issuing module judges whether vnf _ buffer _ count is empty, if not, go to step 3.3.9; otherwise, go to step 3.3.3.
3.3.9, the service index issuing module calls a MetricFamilysamples function of the API of Prometheus, converts the variable vnf _ buffer _ count into a service operation monitoring index, and issues the service operation monitoring index to the monitoring index unified collecting module, wherein the index includes a label: vnfTagContest, index value, namely service processing counting value, and index name is detail _ count. And the index label and the index value form service operation monitoring index metadata.
3.3.10, the service index issuing module empties the service monitoring index buffer vnf _ buffer _ count, and goes to step 3.3.3.
3.4 the monitoring index uniform acquisition module adopts open source software Prometheus. The monitoring index unified acquisition module acquires the monitoring indexes according to the following procedures:
3.4.1 reading the Node monitoring object file generated in the step 3.1, if the Node monitoring object file exists, switching to 3.4.2; if the Node monitoring object file does not exist, switching to 3.4.3;
3.4.2 sequentially reads the targets fields of N Node monitoring object files (IP addresses of Node monitoring objects), respectively sends requests of pulling monitoring indexes to the N IP addresses, collects the performance monitoring index metadata generated in step 3.2 from the performance monitoring index generation module, writes the performance monitoring index metadata into the time sequence database, and changes to 3.4.4.
3.4.3, after waiting for the scratch _ interval second, turning to 3.4.1 to read the Node monitoring object file again;
3.4.4 reading the Pod monitoring object file generated in the step 3.1, and if the Pod monitoring object file exists, turning to 3.4.5; if the Pod monitoring object file does not exist (namely, the Pod monitoring object file is not generated), turning to 3.4.1;
3.4.5 sequentially reading the target fields (IP addresses of the Pod monitoring objects) of the M Pod monitoring object files, respectively sending a request of pulling the monitoring index to the M IP addresses, collecting the service operation monitoring index metadata and the module operation state index metadata generated in the step 3.3 from the service monitoring index generation module, and converting to 3.4.6.
And 3.4.6, complementing the management label in the Pod monitoring object file into the module running state index metadata, writing the label running state monitoring index metadata added with the label into a time sequence database, and turning to 3.4.7. (because the state variable restful _ up of the service index metadata analysis module is set to be in a working state when the module is started, when the monitoring index uniform acquisition module sends a 'pull monitoring index' request to the service index generation module, the state index metadata of the module can be collected, wherein the state index metadata comprises the name of a service function unit);
3.4.7 if the service monitoring index metadata (including the character string of the service monitoring index metadata combination) is not empty, completing the management tag in the Pod monitoring object file into the service monitoring index metadata, writing the service monitoring index metadata added with the tag into a time sequence database, and turning to 3.4.1; otherwise, directly turn to 3.4.1.
3.5 the resource centralized allocation module obtains the monitoring index metadata from the time sequence database, when the CPU utilization rate or the memory utilization rate exceeds the threshold set by the user, a service-performance model is established, a resource allocation scheme is dynamically generated, the resource allocation scheme is output to the user, and data support is provided for the reasonable allocation of Pod resources by the user. The method comprises the following steps:
3.5.1 resource centralized allocation module initialization, the method is:
3.5.1.1 "Business-Performance" model building Module reads parameters from the resource configuration File resAdmConfig: URL, dataBase, searchInterval, timeOffset, cputSD, memTSSD.
3.5.1.2 the model building module of "business-performance" calls the connect function (version inflixdb-java-2.19 and above) of API interface of inflixDB to build database connection.
3.5.2 "business-performance" model building module records current system time t3, and initializes three control variables of resource allocation scheme, business monitoring index indication variable, performance monitoring index indication variable, the method is:
3.5.2.1 resource allocation scheme resDis is initialized to null, resDis is a "key-value" list, each entry is a "key-value" structure, representing the resource configuration of a Pod, where a "key" includes two fields: the name of the SFC and the name of the Pod, the "value" includes two fields: the number of cores of the CPU and the memory Mem size.
3.5.2.2 initializing the service operation monitoring index to indicate that the variable vnfIndex is null, the vnfIndex is a "key-value" list, each entry is a "key-value" structure, where a "key" includes two fields: the name of the SFC and the name of the Pod, the "value" is the rate at which the VNF function processes traffic.
3.5.2.3 the initialized Performance monitoring indicator indicates that the variable perfIndex is null, perfIndex is a list of "key-values", each entry is a "key-value" structure, where a "key" includes three fields: the name of the SFC, the name of the Pod, the location of the SFC, the "value" includes two fields: and the CPU utilization rate and the memory Mem utilization rate of the VNF business function unit.
3.5.3, the method for updating resDis by the service-performance model establishing module is as follows:
3.5.3.1 calls the Influxdb interface to inquire the state _ up table in the time sequence database, groups the state _ up table with the SFC name and the Pod, calls the aggregation class function last () of the Influxdb interface to obtain the inquiry result, and the final result includes: the name of the SFC, the name of the Pod, the number of cores of the CPU, and the size of the memory Mem are stored in a resource state list resdiscist, each element of the resdiscist includes four fields, that is: the name SFC of the SFC, the name Pod of the Pod, the core number CPU of the CPU, and the memory Mem size Mem, where resdiscist len is the length of the resdiscist list.
3.5.3.2 if resDisListLen <1, go to 3.5.9, otherwise go to 3.5.3.3.
3.5.3.3 let resdiscist number s equal to 1.
3.5.3.4 obtaining resDisList with item s as resDisList item [ s ], storing SFC field and Pod field of resDisList item [ s ] into SFC field and Pod field of resDis 'key' respectively; the CPU field and the Mem field of resDisListItem [ s ] store the "value" of resDis in the CPU field and memory Mem field, respectively.
3.5.3.5 if s < resdisclistlen, let s be s +1, go to 3.5.3.4, otherwise go to 3.5.4.
3.5.4 the "Business-Performance" model building Module updates the CPU domain of perfIndex. The method comprises the following steps:
3.5.4.1 calling an Influxdb interface to query a user _ cpu table in a time sequence database, obtaining a query result in timeOffset seconds in the table, grouping the query result by an SFC name, an SFC position and a Pod, calling an Influxdb aggregation function derivative () to calculate a speed, calling an Influxdb aggregation function mean () to calculate an average value, wherein the final query result comprises: the name of the SFC, the position of the SFC, the name of the Pod, and the CPU utilization rate are stored in a variable CPU state list cpuList, each element of the cpuList is four fields, that is, the name SFC of the SFC, the position sfcIndex of the SFC, the name SFC of the Pod, and the CPU utilization rate CPU, and the cpuListlen is the length of the list cpuList.
3.5.4.2 if cpuListLen <1, go to 3.5.9, otherwise go to 3.5.4.3.
3.5.4.3 specifies the cpuList number t as 1.
3.5.4.4 obtaining that the t-th item of the cpuList is cpuItem [ t ], respectively storing the SFC field, sfcIndex field and Pod field of the cpuItem [ t ] into the SFC field, SFCIndex field and Pod field of the perfIndex 'key', and storing the CPU field of the cpuItem [ t ] into the CPU field of the perfIndex 'value'.
3.5.4.5 if t < cpulislen, let t be t +1, go to 3.5.4.4, otherwise go to 3.5.5.
3.5.5 "Business-Performance" model building Module updates the Mem field of perfIndex. The method comprises the following steps:
3.5.5.1 calling an Influxdb interface to query a use _ memory table in a time sequence database, obtaining a query result in timeOffset seconds in the table, grouping the query result by an SFC name, an SFC position and a Pod, calling an Influxdb aggregation function derivative () to calculate a speed, then calling an Influxdb aggregation function mean () to calculate an average value, wherein the final query result comprises: the name of the SFC, the location of the SFC, the name of the Pod, and the memory utilization rate are stored in a memory state list memList, each element of the memList is four fields, that is, the name SFC of the SFC, the location sfcIndex of the SFC, the name SFC of the Pod, and the memory utilization rate mem, and the memList len is the length of the memList list.
3.5.5.2 if memListLen <1, go 3.5.9, otherwise go 3.5.5.3.
3.5.5.3 let memList number u equal to 1.
3.5.5.4 obtaining the u-th item of memList as memItem [ u ], updating the Mem field "value" of the "key" of the memItem [ u ] of the variable perfIndex to the Mem field of the memItem [ u ].
3.5.5.5 if u < memListLen, let u be u +1, go to 3.5.5.4, otherwise go to 3.5.6.
3.5.6 model establishing module of service-performance updates vnfIndex, the method is:
3.5.6.1, calling an inflixdb interface to query a detail _ count table in a time sequence database, obtaining a query result in timeOffset seconds in the table, grouping the query result with SFC name and Pod, calling an inflixdb aggregation function derivative () to calculate a speed, then calling an inflixdb aggregation function mean () to calculate an average value, wherein the final query result comprises SFC name, Pod name and service processing speed, and is stored in a service processing state list vnfList, each element of the vnfList comprises three fields, namely name SFC of SFC, name Pod of Pod, service processing speed bpRate, and vnfListLen is the length of the vnfList list.
3.5.6.2 if vnfListLen <1, go 3.5.9, otherwise go 3.5.6.3
3.5.6.3 make vnfList number v equal to 1.
3.5.6.4 get vnfList with the v-th item as vnfItem [ v ], store the SFC field and the Pod field of vnfItem [ v ] into the SFC field and the Pod field of vnfIndex "key", respectively, and store the bpRate field of vnfItem [ x ] into the vnfIndex "value" field.
3.5.6.5 if v < vnfListLen, let v ═ v +1, go 3.5.6.4, otherwise go 3.5.7.
3.5.7 resource allocation scheme generating module finds out the warning of the over-range value and updates resDis, the method is:
3.5.7.1 has the perfIndex variable index numbered w 1.
3.5.7.2 if the performance monitoring indicator indicates that the CPU field of the "value" of the w-th item of the variable perfIndexItem [ w ] exceeds the CPU SD, go to 3.5.7.3, otherwise go to 3.5.7.4.
3.5.7.3 the CPU field that records the "value" of perfIndexItem [ w ] is tsdCPU; marking the character string of 'SFC position' deleted by the 'key' of perfIndexItem [ w ] as Pod _ tsdCPU, and finding the Pod capable of balancing CPU resources from perfIndex by the following method:
3.5.7.3.1 let perfIndex number w1 be 1 and CPU field value comparison variable minCPU be 1.0 x 106
3.5.7.3.2 if the SFC field of perfIndexItem [ w ] is equal to the SFC field of perfIndex [ w1], and the SFCIndex field of perfIndexItem [ w ] is smaller than the SFCIndex field of perfIndex [ w1], and minCPU is smaller than the CPU field of perfIndex [ w1], go to 3.5.7.3.3, otherwise go to 3.5.7.3.4.
3.5.7.3.3 designates minCPU as a cpu field of perfIndex [ w1], and records the "key" deleted "SFC position" character string of perfIndex [ w1] as pod _ minCPU.
3.5.7.3.4 if w1< perfIndexLen, let w1 ═ w1+1, go to 3.5.7.3.2, otherwise go to 3.5.7.3.5.
3.5.7.3.5 if minCPU not equal to 1.0 x 106And minCPU is less than CPU SD, go to step 3.5.7.3.6, otherwise, go to step 3.5.10
3.5.7.3.6 let the total CPU to be allocated cpussum be tsdCPU + minCPU.
3.5.7.3.7 takes an entry in the list vnfIndex for which the "key" is pod _ tsdCPU, and let tsdCPUVNF be the "value" of the entry.
3.5.7.3.8 takes an entry in the list vnfIndex with "key" as pod _ minCPU, let minCPUVNF be the "value" of the entry.
3.5.7.3.9 update the CPU field of the resDis variable to pod _ tsdCPU "Key" corresponds to "value
Figure BDA0003001275680000471
3.5.7.3.10 update the CPU field of the resDis variable to the value corresponding to the pod _ minCPU key
Figure BDA0003001275680000472
3.5.7.4 if the Mem field of the "value" of perfIndexItem [ w ] exceeds memTSD, go to 3.5.7.5, otherwise go to 3.5.7.6.
3.5.7.5 the Mem field recording the "value" of perfIndexItem [ w ] is tsdMem; after the "SFC position" character string is deleted by the "key" of perfIndexItem [ w ], the "SFC position" character string is marked as Pod _ tsdMem, and Pod which can balance memory resources is found from the variable perfIndex, and the method is as follows.
3.5.7.5.1 let perfIndex number w2 be 1 and internal memory value comparison variable minMem be 1.0 x 106
3.5.7.5.2 if the SFC field of perfIndexItem [ w ] is equal to the SFC field of perfIndex [ w2], and the SFCIndex field of perfIndexItem [ w ] is smaller than the SFCIndex field of perfIndex [ w2], and minMem is smaller than the Mem field of perfIndex [ w2], roll 3.5.7.5.3, otherwise roll 3.5.7.5.4.
3.5.7.5.3, minMem is defined as pod _ minMem by deleting the "SFC position" character string from the "key" of perfIndex [ w2] in the Mem field of perfIndex [ w2 ].
3.5.7.5.4 if w2< perfIndexLen, let w2 ═ w2+1, go to 3.5.7.5.2, otherwise go to 3.5.7.5.5.
3.5.7.5.5 if minMem is not equal to 1.0 x 106And minMem is less than memTSD, go to 3.5.7.5.6, otherwise, go to step 3.5.10.
3.5.7.5.6 let the total amount of memory to be allocated, memSum, tsdMem + minMem.
3.5.7.5.7 takes an item in the list vnfndex with "key" pod _ tsdMem, let tsdmemf vnf be the "value" of the item.
3.5.7.5.8 takes an entry in the list vnfIndex with "key" pod _ minMem, let minMemVNF be the "value" of the entry.
3.5.7.5.9 update the pod _ tsdMem "key" of the resDis variable to "value" with the mem field
Figure BDA0003001275680000481
3.5.7.5.10 update the mem field of the podjminmem "key" corresponding to "value" of the resDis variable to
Figure BDA0003001275680000482
3.5.7.6 if w < perfIndexLen, let w be w +1, go to 3.5.7.2, otherwise go to 3.5.8.
3.5.8 resource allocation scheme generating module outputs resource allocation scheme resDis, the method is:
3.5.8.1 let the sequence number z of the item in resDis equal to 1.
3.5.8.2 obtains the z-th item resDisItem [ z ] of resDis, and outputs the "key" and "value" of resDisItem [ z ] to the user.
3.5.8.3 if z < resDisLen, z ═ z +1, go to 3.5.8.2, otherwise go to 3.5.9;
3.5.9 records the current system time t4, if t4-t3 is equal to searchInterval, turn to 3.5.2, otherwise, turn to 3.5.9.
3.5.10 output "original resources are not enough, cannot be allocated between the Pod, and new resources need to be added to the SFC", and end.

Claims (8)

1. A unified monitoring method of virtual network function is characterized by comprising the following steps:
firstly, constructing a virtual network function unified monitoring platform;
1.1, constructing a virtual network function operation platform and deploying a service function chain;
constructing a k8s platform comprising a Master Node and N Node nodes, wherein a VNF service function unit is deployed in a scheduling basic unit Pod corresponding to k8s and is allocated with corresponding CPU and memory resources; the Pod belongs to different namespaces of the k8s platform, if not specified, the Pod belongs to a default space and is represented by default; the service flow sequentially passes through the Pod according to a predetermined sequence to form a service function chain; the Pod where the VNF service functional unit is deployed is named Pod _ < VNF >, where < VNF > is an abbreviation of a name of the VNF service functional unit, and the existing 7 types of VNF service functional units include: the system comprises a message filtering function, a load balancing function, a network address conversion function, a deep message detection function, an intrusion prevention function, a protocol identification function and a Web application protection function; if the pod _ < vnf > needs to be added into the virtual network function unified monitoring platform, so that the unified monitoring platform collects the service index metadata, adding a label 'monitored' to the pod _ < vnf >; each Pod only deploys a Docker container related to a VNF service function, and each Pod only belongs to a service function chain;
the VNF business function unit is divided into a VNF unit with a customized function and a VNF unit with a standard function; the VNF unit of the customization function is a VNF which can modify source codes to meet special requirements and directly generates service monitoring index metadata; the VNF unit with the standard function belongs to mature software, and the log file contains the service processing state and process of the VNF unit with the standard function, and the fields of the existing log file need to be analyzed to generate service monitoring index metadata;
on the basis of the original Pod management tag of the k8s platform, 2 new management tags are added for each Pod, namely: the name SFC of the service function chain to which the Pod belongs, and the position SFCIndex of the service function chain to which the Pod belongs;
1.2, constructing a virtual network function unified monitoring platform:
the virtual network function unified monitoring platform consists of a monitoring object discovery module, a service monitoring index generation module, a performance monitoring index generation module, a monitoring index unified acquisition module, a resource centralized distribution module and a time sequence database;
a Pod is newly built in a Master node of a k8s platform and named Pod _ dis, and a monitoring object discovery module and a monitoring index unified acquisition module are deployed in the Pod _ dis;
the monitoring object discovery module is connected with the monitoring index uniform acquisition module and consists of a Node object discovery module, a Pod object discovery module and a monitoring object configuration file autoDisConfig; the autoDisConfig includes 6 fields, which are respectively: the interface calling frequency schedulernerval, the position K8sConfigPath of the configuration file of K8s, the Pod monitoring object namespace K8SNamespace, and the filter tag monitore Label of the Pod monitoring object, wherein the monitore Label is the identification of whether the K8s platform represents the monitored Pod, if the tag is "monitored", the monitored Pod is represented, otherwise, the monitored Pod is not required; the storage positions of the Node object files, pwd _ Node _ targets and the storage positions of the Pod object files, pwd _ Pod _ targets; the Node object discovery module is connected with an application program interface of a k8s platform, and calls an API function listNode of k8s to discover Node objects in each interval schedulellnterval, so that N Node objects are discovered, wherein N is the total number of the Node objects; generating Node monitoring object files for N nodes by taking the IP address of a Node as a file name, and enabling the Node monitoring object files of the Node with the IP address of the Node IP to be < Node IP >. yml; storing N Node monitoring object files in a directory pwd _ Node _ targets; the Pod object discovery module is connected with an application program interface of a k8s platform, when the Node object discovery module discovers a Node object, the Pod object discovery module calls an API function listNamespaced Pod of k8s to discover Pod objects, so that M Pod objects are discovered together, wherein M is the total number of the Pod objects, the IP addresses of the SFC where the Pod is located and the Pod are used as file names, a Pod monitoring object file of each Pod is generated, and the Pod monitoring object file of the Pod with the IP address of the Pod of the PodIP is made to be < SFC + PodIP >. yml; storing M Pod monitoring object files under a directory pwd _ Pod _ targets;
the performance monitoring index generating module adopts cAdvisor and is connected with the monitoring index uniform acquisition module; the performance monitoring index generating module newly builds a Pod on each Node, which is named Pod _ prf _ < x >, wherein < x > <1, 2, … N, and each Node deploys a Pod of cadvisors, namely N nodes deploy N cadvisors together; the cAdvisor is deployed on the pod _ prf _ < x >, acquires information through interaction with the Docker engine, and generates performance monitoring index metadata for being pulled by the monitoring index unified acquisition module, wherein the performance monitoring index metadata comprises metadata of CPU utilization rate and memory utilization rate indexes;
the service monitoring index generating module is deployed at pod _ < vnf >; the service monitoring index generating module is connected with a restFul data interface and a monitoring index uniform acquisition module of the NVF service functional unit; the system comprises a service index metadata analysis module, a service index release module and a service monitoring index configuration file vnfConfig; vnfConfig includes two domains: service index metadata tag group tags, type of service index metadata, tags is a name list of tags of service index metadata, and type includes two values: string indicates that the value of the tag is of a character type, Double indicates that the value of the tag is of a numerical type; the content of vnfConfig is configured according to the type of the VNF business functional unit on pod _ < VNF >; the service index metadata analysis module is connected with the restFul data interface and the service index publishing module, monitors the restFul data interface, receives service monitoring index metadata pushed by a VNF service function unit in pod _ < VNF >, analyzes the service monitoring index metadata, caches and generates the service monitoring index metadata, and sends the service monitoring index metadata to the service index publishing module, wherein the service monitoring index metadata comprises service operation monitoring index metadata and a module operation state; the service index issuing module is connected with the service index metadata analyzing module and the monitoring index unified collecting module, monitors a 'pull monitoring index' request of the monitoring index unified collecting module, and issues service monitoring index metadata to the monitoring index unified collecting module when receiving the 'pull monitoring index' request;
the monitoring index unified acquisition module is connected with the monitoring object discovery module, the time sequence database, the service monitoring index generation module and the performance monitoring index generation module, adopts Prometous and is deployed in the pod _ dis; the monitoring index acquisition module is provided with a monitoring index acquisition configuration file monConfig, and the monConfig comprises 4 fields which are respectively as follows: collecting a cycle script _ interval, a storage position pwd _ Node _ targets of a Node object file, a storage position pwd _ Pod _ targets of a Pod object file and a write address remote _ write of a time sequence database; the unified monitoring index acquisition module periodically acquires a monitoring object from the lists of pwd _ node _ targets and pwd _ pod _ targets according to an acquisition period scrape _ interval configured by monConfig, sends a 'pull monitoring index' request to the performance monitoring index generation module, and collects performance monitoring index metadata generated by the performance monitoring index generation module; meanwhile, sending a 'pull monitoring index' request to a service monitoring index generation module, and collecting service monitoring index metadata; adding labels to the performance monitoring index metadata and the service monitoring index metadata, and storing the performance monitoring index metadata and the service monitoring index metadata added with the labels in a time sequence database;
the time sequence database is connected with the monitoring index uniform acquisition module and the resource centralized distribution module, is named as 'nfvMonitor', is deployed on a server outside a k8s platform and is used for storing service monitoring index metadata and performance monitoring index metadata, and each index metadata is independently stored in a form of a table, and comprises: a service monitoring index generation module state index table, a service operation monitoring index detail table, a service function unit CPU use condition table and a service function unit memory use condition table; the state _ up of the state index table of the service monitoring index generation module comprises 9 table entries: the system comprises a timestamp, namely system time generated by a record item, a CPU configuration numerical value, namely the number of CPU cores configured by a system administrator for a Pod, a memory configuration numerical value, namely the number of memories configured by the system administrator for the Pod, a VNF name, namely the name of a VNF service function, a Node name, namely the name of a Node, a Pod name, an SFC name, namely the name of a service function chain to which the Pod belongs, an SFC position, namely the position serial number of the service function chain to which the Pod belongs, and a starting state of a monitoring acquisition module; the service operation monitoring index detail table detail _ count comprises table entries: the method comprises the steps of time stamping, a VNF metadata set, namely a tag item of service index metadata of a configuration file vnfConfig of a VNF service function unit, a Node name, namely the name of a Node to which a Pod belongs, a Pod name, an SFC position, namely the position serial number of a service function chain to which the Pod belongs, and a service processing count value, namely the number of service processing; the service function unit CPU usage table user _ CPU includes table entries: a timestamp, a container tag set, namely a system management tag set generated by a k8s platform for a container, a Pod name, namely the name of a Pod to which the container belongs, a node name, an SFC position, and a CPU usage value, namely the CPU usage time; the service function unit memory usage table user _ memory comprises table entries: a system management tag set, a pod name, a node name, an SFC position and a memory use value, namely memory use amount, are generated for a container by a timestamp and a container tag set, namely a k8s platform;
the resource centralized allocation module obtains the CPU and memory allocation status of Pod from the state _ up table; obtaining metadata of the VNF service function unit processing traffic volume rate from a detail _ count table; obtaining metadata of CPU utilization rate and memory utilization rate when processing the current service from a user _ CPU and user _ memory table; according to the 4 index metadata, establishing a service-performance model for resource allocation;
the resource centralized distribution module and the time sequence database are deployed in the same server, the resource centralized distribution module is connected with the time sequence database, and the resource centralized distribution module consists of a service-performance model building module, a resource distribution scheme generating module and a resource configuration file resAdmConfig; the resource profile includes 6 fields: a time sequence dataBase URL, a dataBase name dataBase, a frequency searchInterval of inquiring the dataBase, a time offset timeOffset of inquiring the dataBase, a CPU warning threshold CPU TSD and a memory warning threshold memTSD; the service-performance model building module is connected with the time sequence dataBase and the resource allocation scheme generating module, data between the current time and the previous time offset seconds are obtained from a detail _ count table, a user _ CPU table and a use _ memory table respectively according to the frequency searchInterval of the query dataBase in the dataBase under the URL of the time sequence dataBase, and the CPU utilization rate, the memory utilization rate and the service processing rate are calculated; obtaining the CPU and memory allocation status of Pod from the state _ up table; when the CPU utilization rate exceeds the CPU TSD or the memory utilization rate exceeds the memTSD, a 'business-performance' model is established, and the 'business-performance' model is sent to a resource allocation scheme generation module; the resource allocation scheme generation module is connected with the business-performance model establishment module, generates a Pod new CPU and memory resource allocation scheme according to the business-performance model received from the business-performance model establishment module, and sends the CPU and memory resource allocation scheme to a K8S platform administrator;
secondly, initializing a virtual network function unified monitoring platform, wherein the method comprises the following steps:
2.1 configuring a monitoring object configuration file autoDisConfig of the monitoring object discovery module:
2.1.1 setting the frequency schedulelnterval value of the interface calling to be more than or equal to 60 seconds;
2.1.2 set the value of the configuration file Config location k8 sfonfigpath of k8s to the location of the file Config;
2.1.3, setting the value of the Pod monitoring object namespace K8SNamespace as a default value 'default', supporting the setting of a plurality of namespaces, using and separating the namespaces;
2.1.4 setting the value of the screening tag monitore Label of the Pod monitoring object to "monitored";
2.1.5 setting the storage position pwd _ Node _ targets value of the Node object file to be '/home/targets/nodes';
2.1.6 setting the storage location pwd _ Pod _ targets value of the Pod object file to "/home/targets/pods";
2.2 configuring a monitoring index configuration file monConfig of the monitoring index uniform acquisition module:
2.2.1 set pwd _ node _ targets value to "[ '/home/targets/nodes/. yml' ]";
2.2.2 sets the pwd _ pod _ targets value to "[ '/home/targets/pod/. yml' ]";
2.2.3 configure remote _ write as "-url http: /< time-series database IP address >: < port >/api/v1/prom/write ═ nfvmmonitor ", where < time series database IP address > is the time series database IP address and < port > is the time series database port;
2.2.4, setting the value of the scrape _ interval of the acquisition period to be more than or equal to 60 seconds;
2.3 configuring a service monitoring index configuration file vnfConfig of a service monitoring index generating module according to the type of the VNF service functional unit on the pod _ < VNF >;
2.4 configuring resource configuration file resAdmConfig of resource centralized allocation module, the method is:
2.4.1 configure the timing database URL as "http:/< timing data IP address > < port number >;
2.4.2 set the time series dataBase name dataBase to "nfvMonitor";
2.4.3 setting the frequency searchInterval of the query database to be more than or equal to 300 seconds;
2.4.4 setting the time offset timeOffset of the query database to be more than or equal to 600 seconds;
2.4.5 set the CPU warning threshold CPU UTSD to between 60% and 80%;
2.4.6 set the memory warning threshold memTSD to 60% to 80%;
thirdly, a monitoring object discovery module, a performance monitoring index generation module, a service monitoring index generation module, a monitoring index unified acquisition module and a resource centralized distribution module of the virtual network function unified monitoring platform work in parallel and are matched with each other to monitor the CPU use condition, the memory use condition and the service operation condition of the Pod, generate monitoring index metadata, complete a management label according to the metadata condition, store the monitoring index metadata into a time sequence database, and establish a service-performance model to generate a new resource distribution scheme when a monitoring domain value alarm occurs; the method comprises the following steps: the monitored object discovery module generates a file list of the monitored object according to the flow of the step 3.1; meanwhile, the performance monitoring index generating module generates performance monitoring index metadata according to the flow of the step 3.2; meanwhile, the service monitoring index generating module generates service monitoring index metadata according to the flow of the step 3.3; meanwhile, the monitoring index unified acquisition module acquires the service index and the performance index according to the 3.4-step flow and stores the service index and the performance index into a time sequence database; meanwhile, the resource centralized allocation module alarms according to the process monitoring threshold value of the 3.5 steps, establishes a model and generates a resource allocation scheme:
3.1 the monitoring object discovery module obtains Node and Pod objects of k8s by using the API interface of k8s, generates Node monitoring object files and Pod monitoring object files, and stores the Node monitoring object files and the Pod monitoring object files in a classified manner; the method comprises the following steps:
3.1.1 initializing a monitoring object discovery module; the method comprises the following steps:
3.1.1.1Node object discovery module reads parameters from the monitored object configuration file autoDisConfig: scheduleIhterval, K8s configpath, K8SNamespace, monitorLabel, pwd _ node _ targets, pwd _ pod _ targets;
3.1.1.2Node object discovery Module initializes the Node object control list nodeTargets to be empty, making the list length nodeListLen, each entry of nodeTargets is a "key-value" structure, where "key" is the IP address of Node and "value" is three flag characters: the 'I' represents initialization, the 'E' represents existing, and the 'A' represents to-be-added;
3.1.1.3Pod object discovery module initializes Pod object control list podtarget to null, making the list length podListLen, each item of podTargets is a "key-value" structure, where "key" is the string concatenation of the service function chain name to which Pod belongs and the IP address of Pod, and the content of "value" is the same as nodeTargets, and is also three flag characters: the 'I' represents initialization, the 'E' represents existing, and the 'A' represents to-be-added;
3.1.2Node object discovery module records start time t1 ═ system time;
3.1.3Node object discovery Module updates nodeTargets; the method comprises the following steps:
3.1.3.1 judging whether the nodeTargets contain a 'key-value' pair, if so, turning to 3.1.3.2, otherwise, turning to 3.1.3.3;
3.1.3.2 set the "value" of all "keys" of nodeTargets to "I";
3.1.3.3 calls listNode function in API of k8s to get Node object list nodeList, nodeList list has nodeListLen entries, wherein, the x-th entry is represented by nodeListItem [ x ], x is more than or equal to 1 and less than or equal to nodeListLen, the content of nodeListItem [ x ] is the IP address of x-th Node;
3.1.3.4 let nodeList number x be 1;
3.1.3.5 obtaining nodeList item x of nodeList from nodeList, if nodeList item x exists in the ' key ' of nodeTargets, putting ' value ' E ' of the ' key ' nodeList item x of nodeTargets, turning to 3.1.3.6; otherwise, add to nodeTargets with nodeListItem [ x ] as new "key", put "value" of "key" nodeListItem [ x ] to "A", go to 3.1.3.6;
3.1.3.6 if x is less than nodelislen, let x be x +1, change 3.1.3.5; otherwise, turning to 3.1.4;
3.1.4Pod object discovery module updates the podtarges variable; the method comprises the following steps:
3.1.4.1 if the podtarges already contain a "key-value" pair, turn 3.1.4.2; otherwise, turning to 3.1.4.3;
3.1.4.2 sets the "value" of all "keys" of the podtarges to "I";
3.1.4.3 calling listNamespacedPod function in API of k8s to get Pod object list podList, there are podListLen entries in the podList list, where the y-th entry is represented by podListItem [ y ], y is greater than or equal to 1 and less than or equal to podListLen, podListItem [ y ] is a structure body, including: the name SFC of the SFC where the Pod is located, the position of the SFC where the Pod belongs, the name of the Node where the Pod is located, the IP address ipAddr of the Pod, the number of CPU cores allocated by the Pod and the size of the memory allocated by the Pod;
3.1.4.4 let podList number y be 1;
3.1.4.5 obtaining the y-th item podList item [ y ] of podList from podList, if the character string formed by splicing the SFC field of podList item [ y ] and the ipAddr field already exists in the 'key' of podTtarget, then setting the 'value' corresponding to the 'key' podList item [ y ] of podTtarget to 'E', and turning to 3.1.4.6; otherwise, add the SFC field + iPAddr field of podListItem [ y ] as new "key" to podtarget, put "value" to "A" of "podLisstItem [ y ], go to 3.1.4.6;
3.1.4.6 if y is less than podListLen, let y be y +1, turn 3.1.4.5; otherwise, turning to 3.1.5;
3.1.5Node object discovery module creates or deletes Node monitoring object file, the method is:
3.1.5.1 setting nodetargets number m to 1;
3.1.5.2 obtaining the m-th item nodeTargetItem [ m ] of nodeTargets from nodeTargets, m is more than or equal to 1 and less than or equal to nodeTargetLen, nodeTargetLen is the length of nodeTarget list;
3.1.5.3 if the 'value' of the node targetitem [ m ] is 'A', a node object monitoring file is newly created, the file name is < node targetitem [ m ] >. yml, stored in the directory pwd _ node _ targets, the file content includes: "targets: nondeTargetItem [ m ] "targets are labels of objects collected in the monitoring index unified collection module, nondeTargetItem [ m ] is an IP address of the mth Node, and 3.1.5.4 is transferred; if the "value" of the nodeTargetItem [ m ] is "I", deleting the file < nodeTargetItem [ m ] >. yml under the directory pwd _ node _ targets, and deleting nodeTargetItem [ m ] in nodeTargets, go to 3.1.5.4; if the value of nodeTargetItem [ m ] is "E", the file is reserved, go to 3.1.5.4;
3.1.5.4 if m is less than nodeTargetLen, making m equal to m +1, turning to 3.1.5.2; otherwise, turning to 3.1.6;
3.1.6Pod object discovery module creates or deletes Pod monitoring object file, the method is:
3.1.6.1 let the number n of podtarges be 1;
3.1.6.2 obtaining an nth podtargetItem [ n ] of the podTargets from the podTargets, n is greater than or equal to 1 and less than or equal to podtargetLen, and the podtargetLen is the length of the podTargets list;
3.1.6.3, if the 'value' of the podtagetsutem [ n ] is 'A', a pod monitoring object file is newly created, the file name of which is < podtagetsutem [ n ] >. yml, and the file is stored in the directory pwd _ pod _ targets; the file content includes management tags: "targets: IP address of Pod, node: name of Node, pod: the name of Pod, sfc: name of SFC, sfcIndex: SFC position, CPU: core number of CPU, Mem: memory size ", go to 3.1.6.4; if the "value" of the podtargettsitem [ n ] is "I", delete the file < podtargettsitem [ n ] >. yml, and delete podtargettsitem [ n ] in podtarges, go to 3.1.6.4; if the "value" of the podtargetItem [ n ] is "E", the file is retained, go to 3.6.1.4;
3.1.6.4 if n is less than podTargetLen, let n be n +1, turn 3.1.6.2; otherwise, turning to 3.1.7;
3.1.7node object discovery module reads system time, let t2 be system time, if t2-t1 is not less than scheduleelnterval, turn to 3.1.2; otherwise, turning to 3.1.7;
3.2 the performance monitoring index generation module periodically reads related files on the Node host, takes each Docker as an object, and counts the occupation time and the memory usage of the CPU of the Docker container to generate performance monitoring index metadata for being pulled by the monitoring index unified acquisition module;
3.3 the service monitoring index generating module receives the service monitoring index metadata character string pushed by the VNF service function unit in the pod _ < VNF >, analyzes, caches, summarizes and counts, and generates the service operation state monitoring index and the module operation state index metadata; the method comprises the following steps:
3.3.1 the service index metadata parsing module reads the service monitoring index configuration file vnfConfig, obtains tags and types, makes tagsLen be the length of tags, and the length of types be the same as tags, initializes the mapping list metaDataMap of monitoring index metadata, the length of metaDataMap is tagsLen, each item of metaDataMap is a "key-value" structure, where "key" is the metadata label vnfTag, and "value" is the metadata type;
3.3.2 initialize the service monitoring index buffer variable vnf _ buffer _ count to be empty, which is a "key-value" structure, where "key" is the string vnfTagContest of the service monitoring index metadata combination, and "value" includes two fields: a business monitoring index statistic value count and a system timestamp; meanwhile, setting a state variable restful _ up of the service index metadata analysis module to be 1, wherein the state variable restful _ up indicates that the service index metadata analysis module is in a working state and waits for a service function unit to push data;
3.3.3 the service index metadata parsing module monitors whether a service monitoring index metadata character string is received from the VNF service function unit, if so, the step 3.3.4 is carried out; if not, go to 3.3.7;
3.3.4 service index metadata parsing module parses service monitoring index metadata character string, the method is:
3.3.4.1, receiving a service monitoring index metadata string pushed by the VNF service function unit from the restFul data port, that is, a metadata list json, the length of which is json len;
3.3.4.2, initializing the monitor index metaData list metaData to be null, wherein each item of metaData is a "key-value" structure, the "key" is the metaData tag vnfTag, the "value" is the metaData content, and metaDataLen is the metaData list length;
3.3.4.3, let json number q be 1;
3.3.4.4 extracting the q-th item json q of json, which is a structure of "key-value", wherein "key" is the label of the metaData of the business monitoring index and "value" is the content of the metaData of the business monitoring index, if the "value" of json q in the metaDataMap is "String", adding json q to metaData, and turning to 3.3.4.6;
3.3.4.5 if json q in metaDataMap "value" is "Double", which indicates that the content of the tag is a numeric value and not as a management tag of metadata, go directly to 3.3.4.6;
3.3.4.6 if q is less than jsonLen, making q +1, turning to 3.3.4.4, otherwise, turning to 3.3.5;
3.3.5 the service index metadata parsing module buffers the service monitoring index metadata into a service monitoring index buffer area variable vnf _ buffer _ count;
3.3.6 the service index issuing module monitors whether a 'pull monitoring index request' sent by the monitoring index unified collecting module is received, if so, the step 3.3.7 is switched to; if not, go to step 3.3.3;
3.3.7 the business index release module calls MetricFamilysamples function of Prometheus API, converts variable restful _ up into business monitoring index generation module status index, and releases to the monitoring index uniform collection module, the index includes label: VNF name, index value of 1 and index name of state _ up; the index tag and the index value form metadata of a state index of the module;
3.3.8, the service index issuing module judges whether vnf _ buffer _ count is empty, if not, go to step 3.3.9; otherwise, turning to step 3.3.3;
3.3.9, the service index issuing module calls a MetricFamilysamples function of the API of Prometheus, converts the variable vnf _ buffer _ count into a service monitoring index, and issues the service monitoring index to the monitoring index unified collecting module, wherein the index includes a label: vnftagcontrolst, index value: a service processing counting value with an index name of detail _ count; the index label and the index value form service monitoring index metadata;
3.3.10, the service index issuing module empties the service monitoring index buffer vnf _ buffer _ count, and goes to step 3.3.3;
3.4 the unified monitoring index acquisition module acquires the monitoring indexes according to the following procedures:
3.4.1 reading the Node monitoring object file generated in the step 3.1, if the Node monitoring object file exists, switching to 3.4.2; if the Node monitoring object file does not exist, switching to 3.4.3;
3.4.2 reads the targets of N Node monitoring object files in turn, sends the request of pulling monitoring index to the N IP addresses, collects the performance monitoring index metadata generated in step 3.2 from the performance monitoring index generation module, writes the performance monitoring index metadata into the time sequence database, and changes to 3.4.4.
3.4.3 after waiting for script _ interval second, turn to 3.4.1;
3.4.4 reading the Pod monitoring object file generated in the step 3.1, and if the Pod monitoring object file exists, turning to 3.4.5; if the Pod monitoring object file does not exist, turning to 3.4.1;
3.4.5 sequentially reading the target fields of the M Pod monitoring object files, respectively sending a request of pulling the monitoring index to the M IP addresses, collecting the service operation monitoring index metadata and the module operation state index metadata generated in the step 3.3 from the service monitoring index generation module, and turning to 3.4.6;
and 3.4.6, complementing the management label in the Pod monitoring object file into the module running state index metadata, writing the label running state monitoring index metadata added with the label into a time sequence database, and turning to 3.4.7.
3.4.7 if the metadata of the service monitoring index is not empty, completing the management tag in the Pod monitoring object file into the metadata of the service monitoring index, writing the metadata of the service monitoring index added with the tag into a time sequence database, and converting to 3.4.1; otherwise, directly turn to 3.4.1.
3.5 the resource centralized allocation module obtains the monitoring index metadata from the time sequence database, when the CPU utilization rate or the memory utilization rate exceeds the threshold value set by the user, a service-performance model is established, and a resource allocation scheme is dynamically generated, the method is as follows:
3.5.1 resource centralized allocation module initialization, the method is:
3.5.1.1 "Business-Performance" model building Module reads parameters from the resource configuration File resAdmConfig: URL, dataBase, searchInterval, timeOffset, cputSD, memTSSD;
3.5.1.2, the model building module of 'business-performance' calls the connect function of API interface of influxDB to build database connection;
3.5.2 "business-performance" model building module records current system time t3, and initializes three control variables of resource allocation scheme, business monitoring index indication variable, performance monitoring index indication variable, the method is:
3.5.2.1 resource allocation scheme resDis is initialized to null, resDis is a "key-value" list, each entry is a "key-value" structure, representing the resource configuration of a Pod, where a "key" includes two fields: the name of the SFC and the name of the Pod, the "value" includes two fields: the core number of the CPU and the memory Mem size;
3.5.2.2 initializing the service operation monitoring index to indicate that the variable vnfIndex is null, the vnfIndex is a "key-value" list, each entry is a "key-value" structure, where a "key" includes two fields: the name of SFC and the name of Pod, "value" is the rate at which the VNF function processes traffic;
3.5.2.3 the initialized Performance monitoring indicator indicates that the variable perfIndex is null, perfIndex is a list of "key-values", each entry is a "key-value" structure, where a "key" includes three fields: the name of the SFC, the name of the Pod, the location of the SFC, the "value" includes two fields: the CPU utilization rate and the memory Mem utilization rate of the VNF business function unit;
3.5.3 model building module of "business-performance" updates resource allocation scheme resDis, the method is:
3.5.3.1 calls the Influxdb interface to inquire the state _ up table in the time sequence database, groups the state _ up table with the SFC name and the Pod, calls the aggregation class function last () of the Influxdb interface to obtain the inquiry result, and the final result includes: the name of the SFC, the name of the Pod, the core number of the CPU, and the memory configuration value are stored in a resource state list resdiscist, and each element of the resdiscist includes four fields, that is: the name SFC of the SFC, the name Pod of the Pod, the core number CPU of the CPU, the memory Mem size Mem, and the resDisList len is the length of the resDisList list;
3.5.3.2 if resDisListLen <1, go to 3.5.9, otherwise go to 3.5.3.3;
3.5.3.3 make resDisList sequence number s equal to 1;
3.5.3.4 obtaining resDisList with item s as resDisList item [ s ], storing SFC field and Pod field of resDisList item [ s ] into SFC field and Pod field of resDis 'key' respectively; the CPU field and the Mem field of resDisListItem [ s ] are respectively stored into the CPU field and the memory Mem field of the 'value' of resDis;
3.5.3.5 if s < resDisListLen, let s be s +1, go to 3.5.3.4, otherwise go to 3.5.4;
3.5.4, the CPU domain of the performance monitoring index indication variable perfIndex is updated by the 'business-performance' model establishing module, the method is:
3.5.4.1 calling an Influxdb interface to query a user _ cpu table in a time sequence database, obtaining a query result in timeOffset seconds in the table, grouping the query result by an SFC name, an SFC position and a Pod, calling an Influxdb aggregation function derivative () to calculate a speed, calling an Influxdb aggregation function mean () to calculate an average value, wherein the final query result comprises: the name of the SFC, the position of the SFC, the name of the Pod and the CPU utilization rate are stored in a variable CPU state list cpuList, each element of the cpuList is four fields, namely the name SFC of the SFC, the position sfcIndex of the SFC, the name SFC of the Pod and the CPU utilization rate CPU, and the cpuListlen is the length of the list cpuList;
3.5.4.2 if cpuListLen <1, go to 3.5.9, otherwise go to 3.5.4.3;
3.5.4.3 let cpuList number t equal to 1;
3.5.4.4 obtaining that the t-th item of the cpuList is the cpuItem [ t ], respectively storing the SFC field, the sfcIndex field and the Pod field of the cpuItem [ t ] into the SFC field, the SFCIndex field and the Pod field of the perfIndex 'key', and storing the CPU field of the cpuItem [ t ] into the CPU field of the perfIndex 'value';
3.5.4.5 if t < cpulislen, let t be t +1, turn 3.5.4.4, otherwise turn 3.5.5;
3.5.5 "Business-Performance" model building Module updates the Mem field of the Performance monitoring index indicator variable perfIndex by:
3.5.5.1 calling an Influxdb interface to query a use _ memory table in a time sequence database, obtaining a query result in timeOffset seconds in the table, grouping the query result by an SFC name, an SFC position and a Pod, calling an Influxdb aggregation function derivative () to calculate a speed, then calling an Influxdb aggregation function mean () to calculate an average value, wherein the final query result comprises: the name of the SFC, the position of the SFC, the name of the Pod and the memory utilization rate are stored in a memory state list memList, each element of the memList is four fields, namely the name SFC of the SFC, the position sfcIndex of the SFC, the name SFC of the Pod and the memory utilization rate mem, and the memListlen is the length of the memList list;
3.5.5.2 if memListLen <1, switching to 3.5.9, otherwise, switching to 3.5.5.3;
3.5.5.3 let memList number u equal to 1;
3.5.5.4 obtaining the u-th item of the memList as memItem [ u ], updating the Mem field 'value' of the "key" of the memItem [ u ] of the variable perfIndex to the Mem field of the memItem [ u ];
3.5.5.5 if u < memListLen, making u equal to u +1, turning to 3.5.5.4, otherwise turning to 3.5.6;
3.5.6, a service-performance model building module updates a service monitoring index indicator vnfIndex, and the method comprises the following steps:
3.5.6.1 calling an infilfxdb interface to query a detail _ count table in a time sequence database, obtaining a query result within timeOffset seconds in the table, grouping the query result with an SFC name and a Pod, calling an infilfxdb aggregation function derivative () to calculate a speed, then calling an infilfxdb aggregation function mean () to calculate an average value, wherein the final query result comprises the SFC name, the Pod name and a service processing speed, and is stored in a service processing state list vnfList, each element of the vnfList comprises three fields, namely the name sfC SFC, the name Pod name, the service processing speed bpRate, and the vnfListLen is the length of the vnfList list;
3.5.6.2 if vnfListLen <1, go 3.5.9, otherwise go 3.5.6.3
3.5.6.3 make vnfList number v ═ 1;
3.5.6.4 obtaining the vnfList with the v-th item as vnfItem [ v ], respectively storing the sfC field and the Pod field of the vnfItem [ v ] into the SFC field and the Pod field of the vnfIndex 'key', and storing the bpRate field of the vnfItem [ x ] into the vnfIndex 'value' field;
3.5.6.5 if v < vnfListLen, let v ═ v +1, go 3.5.6.4, otherwise go 3.5.7.
3.5.7 resource allocation scheme generating module finds the warning of the over-range value and updates the resource allocation scheme resDis, the method is:
3.5.7.1, making the index of performance monitoring index indicator variable perfIndex with the serial number w being 1;
3.5.7.2 if the performance monitoring indicator indicates that the CPU field of the "value" of the w-th item of the variable perfIndexItem [ w ] exceeds the CPU SD, go to 3.5.7.3, otherwise go to 3.5.7.4;
3.5.7.3 the CPU field that records the "value" of perfIndexItem [ w ] is tsdCPU; marking the character string of 'SFC position' deleted by the 'key' of perfIndexItem [ w ] as Pod _ tsdCPU, and finding the Pod capable of balancing CPU resources from perfIndex by the following method:
3.5.7.3.1 let perfIndex number w1 be 1 and CPU field value comparison variable minCPU be 1.0 x 106
3.5.7.3.2 if the SFC field of perfIndexItem [ w ] is equal to the SFC field of perfIndex [ w1], and the SFCIndex field of perfIndexItem [ w ] is smaller than the SFCIndex field of perfIndex [ w1], and minCPU is smaller than the CPU field of perfIndex [ w1], go to 3.5.7.3.3, otherwise go to 3.5.7.3.4;
3.5.7.3.3 designates minCPU as the cpu field of perfIndex [ w1], deletes the "SFC position" character string from the "key" of perfIndex [ w1], and designates it as pod _ minCPU;
3.5.7.3.4 if w1< perfIndexLen, let w1 ═ w1+1, turn 3.5.7.3.2, otherwise turn 3.5.7.3.5;
3.5.7.3.5 if minCPU is not equal to 1.0 x 106And minCPU is less than CPU SD, go to step 3.5.7.3.6, otherwise, go to step 3.5.10
3.5.7.3.6 let the total CPU to be allocated cpuSum be tsdCPU + minCPU;
3.5.7.3.7 takes an entry in the list vnfIndex for which "key" is pod _ tsdCPU, let tsdCPUVNF be the "value" of the entry;
3.5.7.3.8 takes out the entry in the list vnfIndex with "key" as pod _ minCPU and minCPUVNF as the "value" of the entry;
3.5.7.3.9 update the CPU field of the resDis variable to pod _ tsdCPU "Key" corresponds to "value
Figure FDA0003001275670000161
3.5.7.3.10 update the CPU field of the resDis variable to the value corresponding to the pod _ minCPU key
Figure FDA0003001275670000162
3.5.7.4 if the Mem field of the "value" of perfIndexItem [ w ] exceeds memTSD, go to 3.5.7.5, otherwise go to 3.5.7.6;
3.5.7.5 the Mem field recording the "value" of perfIndexItem [ w ] is tsdMem; after deleting the character string of the SFC position by the key of the perfIndexItem [ w ], marking the character string as Pod _ tsdMem, and finding out Pod which can balance memory resources from the variable perfIndex by the method;
3.5.7.5.1 let perfIndex number w2 be 1 and internal memory value comparison variable minMem be 1.0 x 106
3.5.7.5.2 if the SFC field of perfIndexItem [ w ] is equal to the SFC field of perfIndex [ w2], and the SFCIndex field of perfIndexItem [ w ] is smaller than the SFCIndex field of perfIndex [ w2], and minMem is smaller than the Mem field of perfIndex [ w2], go to 3.5.7.5.3, otherwise go to 3.5.7.5.4;
3.5.7.5.3, let minMem be the Mem field of perfIndex [ w2], delete the "SFC position" character string from the "key" of perfIndex [ w2], and then mark it as pod _ minMem;
3.5.7.5.4 if w2< perfIndexLen, let w2 ═ w2+1, turn 3.5.7.5.2, otherwise turn 3.5.7.5.5;
3.5.7.5.5 if minMem is not equal to 1.0 x 106And minMem is less than memTSD, go to 3.5.7.5.6, otherwise, go to step 3.5.10;
3.5.7.5.6, making the total memory amount to be allocated memSum tsdMem + minMem;
3.5.7.5.7 takes an item in the list vnfndex with "key" pod _ tsdMem, let tsdmemf vnf be the "value" of the item;
3.5.7.5.8 takes an item in the list vnfIndex with "key" pod _ minMem, let minMemVNF be the "value" of the item;
3.5.7.5.9 update the pod _ tsdMem "key" of the resDis variable to "value" with the mem field
Figure FDA0003001275670000171
3.5.7.5.10 update the mem field of the podjminmem "key" corresponding to "value" of the resDis variable to
Figure FDA0003001275670000172
3.5.7.6 if w < perfIndexLen, let w be w +1, go to 3.5.7.2, otherwise go to 3.5.8;
3.5.8 resource allocation scheme generating module outputs resource allocation scheme resDis, the method is:
3.5.8.1 make the sequence number z of the item in resDis equal to 1;
3.5.8.2 obtaining the z-th item resDisItem [ z ] of resDis, outputting to the user the "key" and "value" of resDisItem [ z ];
3.5.8.3 if z < resDisLen, z ═ z +1, go to 3.5.8.2, otherwise go to 3.5.9;
3.5.9, recording the current system time t4, if t4-t3 is more than or equal to searchInterval, turning to 3.5.2, otherwise, turning to 3.5.9;
3.5.10 output "original resources are not enough, cannot be allocated between the Pod, and new resources need to be added to the SFC", and end.
2. The method of claim 1, wherein the cAdvisor version is V0.35 or more; the Prometheus version is V2.16 or greater; the time sequence database is created by using open source software InfluxDB, and the version of InfluxDB is V1.7.9 and above.
3. The method as claimed in claim 1, wherein the method for setting pwd _ node _ targets in the monitoringindex collection configuration file monConfig in the monitoriindex collection module is to configure "jobname" as one entry as "node" and configure "files" option of "node" lower entry "file _ sd _ configs" as storage location pwd _ node _ targets under "scrape _ configs" entry of the promethius configuration file; the method for setting pwd _ pod _ targets in the monConfig is to configure "jobname" as another entry as "pod" and configure "-files" option of "pod" lower entry "file _ sd _ configs" as storage location pwd _ pod _ targets under "script _ configs" entry in the configuration file of Prometheus.
4. The method for unified monitoring of virtual network functions as claimed in claim 1, wherein when the autoDisConfig is configured in step 2.1, the value of scheduleInterval frequency of interface call is set to 60 seconds; setting the value of the configuration file Config position k8sConfigPath of k8s to "/root/. kube/Config"; setting < port > in remote _ write to 8086 when the 2.2-step configuration is monConfig; configuring the URL of the time sequence database as http://127.0.0.1:8086 when configuring the resAdmConfig in the step 2.4; setting the frequency searchInterval of querying the database to 300 seconds; setting the time offset timeOffset of the query database to 600 seconds; setting the CPU warning threshold CPU utsd to 80%; the memory warning threshold memTSD is set to 80%.
5. The method according to claim 1, wherein the step 2.3 of configuring the service monitoring index configuration file vnfConfig of the service monitoring index generating module according to the type of the VNF service function unit on the pod _ < VNF > includes:
2.3.1 if the VNF service function unit on pod _ < VNF > is a packet filtering service unit, configure vnfConfig to the following values: configuring tags option as "[ 'rule _ id', 'sip', 'dip', 'sport', 'dport', 'protocol', 'npkt', 'nbytes' ]", configuring type option as "[ 'Double', 'String', 'Double' ]"; wherein, tags is as follows: rule _ ID is a rule ID of message filtering, sip is a source IP address of the message, dip is a destination IP address of the message, sport is a source port of the message, dport is a destination port of the message, protocol is a layer 4 protocol number of the message, npkt is the number of flow messages, and nbbytes is the number of bytes of the flow; types is type: double indicates that the content of the tag is of a numerical type, String indicates that the content of the tag is of a character type;
2.3.2 if the VNF traffic functional unit on pod _ < VNF > is a load balancing traffic unit, configure vnfConfig to the following values: configuring tags option as "[ 'rule _ id', 'dip', 'sip', 'nip', 'csnpkt', 'csnbyte', 'scnpkt', 'scnbyte' ]", configuring type option as "[ 'Double', 'String', 'Double' ]"; wherein, tags is as follows: rule _ ID is a rule ID of load balancing, dip is a destination IP address of a message, sip is a source IP address of the message, nip is the destination IP address of load balancing, csnpkt is the number of traffic messages sent by a client to a server, csnbyte is the number of bytes of the traffic messages sent by the client to the server, scnpkt is the number of the traffic messages sent by the server to the client, and scnbyte is the number of bytes of the traffic messages sent by the server to the client; types is type: double indicates that the content of the tag is of a numerical type, String indicates that the content of the tag is of a character type;
2.3.3 if the VNF business function on pod _ < VNF > is a Network Address Translation (NAT) business unit, configure vnfConfig to the following values: configuring tags option as "[ 'rule _ id', 'vip', 'sip', 'tip', 'csnpkt', 'csnbyte', 'scnpkt', 'scnbyte' ]", configuring type option as "[ 'Double', 'String', 'Double' ]"; wherein, tags is as follows: rule _ ID is NAT rule ID, vip is a target IP address before address conversion, sip is a source IP address of a message, tip is a target IP after address conversion, csnpkt is the number of flow messages sent by a client to a server, csnbyte is the number of bytes of the flow messages sent by the client to the server, scnpkt is the number of the flow messages sent by the server to the client, and scnbyte is the number of bytes of the flow messages sent by the server to the client; types is type: double indicates that the content of the tag is of a numerical type, String indicates that the content of the tag is of a character type;
2.3.4 if the VNF service functional unit on the pod _ < VNF > is a deep packet inspection service unit, configure vnfConfig to the following values: configuring tags option as "[ 'bytes _ send', 'bytes _ rcvd', 'host', 'ifid', 'packets _ send', 'packets _ rcvd', 'protocol', 'l 4 protocol',", configuring types option as "[ 'Double', 'String', 'Double', 'String' ]"; wherein, tags is as follows: bytes _ send is the number of bytes of the sent message, bytes _ rcvd is the number of bytes of the received message, host is the host name, ifid is the host network card name, packets _ send is the number of the sent message, packets _ rcvd is the number of the received message, protocol is the protocol name of the application layer, and l4proto is the transport layer protocol name; types is type: double indicates that the content of the tag is of a numerical type, String indicates that the content of the tag is of a character type;
2.3.5 if a VNF business functional unit on pod _ < VNF > is an intrusion prevention function business unit, configure vnfConfig to the value: tags options are configured as "[ 'action', 'rule _ info', 'sip', 'sport', 'dip', 'dport', 'protocol' ]", and type options are configured as "[ 'String', 'String' ]; wherein, tags is as follows: action is the action taken by the intrusion prevention function on the message matched with the rule, rule _ info is the description of the intrusion prevention function on the rule matched with the message, sip is the source IP address of the message, sport is the source port of the message, dip is the destination IP address of the message, dport is the destination port of the message, and protocol is the layer 4 protocol number of the message; types is type: string indicates that the content of the tag is of a character type;
2.3.6 if the VNF business function unit on pod _ < VNF > is a protocol identification business unit, configure vnfConfig to the following values: tags options are configured as "[ 'protocol name', 'appname', 'sip', 'dip', 'sport', 'dport', 'protocol' ]", type options are configured as "[ 'String', 'Double' ]"; wherein, tags is as follows: the protocol name is the identified protocol type, the appname is the identified application name, the sip is the source IP address of the message, the dip is the destination IP address of the message, the sport is the source port of the message, the dport is the destination port of the message, and the protocol is the layer 4 protocol number of the message; types is type: double indicates that the content of the tag is of a numerical type, String indicates that the content of the tag is of a character type;
2.3.7 if the VNF business function unit on pod _ < VNF > is a Web application protection business unit, configure vnfConfig to the following values: tags options are configured as "[ 'domain', 'method', 'attribute', 'attacktype', 'attacker', 'attackinfo', 'uri', 'scheme' ]", and types options are configured as "[ 'String', 'String' ]"; wherein, tags is as follows: domain is a domain name of web application attacked by waf, method is a request method of http used in the waf attack, attacktype is the type of the waf attack, attacker is a client ip address sending the waf attack, attackinfo is specific content of the waf attack, uri is url data of http, and scheme is http or https; types is type: string indicates that the content of the tag is of character type.
6. The method as claimed in claim 1, wherein the API version of k8s is client-java-API-8.0.0; the MetricFamilySamples function version of the API of Prometheus is io. Prometheus-0.8.0; the connect function version of the API interface of the infiuxDB is infixdb-java-2.19 or more.
7. The method according to claim 1, wherein the step 3.3.1 of initializing the mapping list metaDataMap of monitoring index metadata includes:
3.3.1.1 let the tag number p be 1;
3.3.1.2 obtaining item p of tags from tags, item p of type from type, taking tags p as "key", type p as "value" as "key-value" structure, adding "key-value" structure to metaDataMap;
3.3.1.3 if p < tagsLen, let p be p +1, change to 3.3.1.2, otherwise, end, get metaDataMap after initialization.
8. The method according to claim 1, wherein the method for buffering the service monitoring index metadata to the service monitoring index buffer variable vnf _ buffer _ count by the service index metadata parsing module in step 3.3.5 is as follows:
3.3.5.1, let serial number r of metaData be 1, let vnfTagContest be null;
3.3.5.2 obtaining the r-th item metaData [ r ] from metaData, merging the metaData [ r ] with vnfttagContest, and saving to vnfttagContest;
3.3.5.3 if r < metaDataLen, let r be r +1, go to 3.3.5.2, otherwise go to 3.3.5.4;
3.3.5.4, if vnf _ buffer _ count already contains "key" vnftagtest, adding 1 to the service monitoring index statistical value count of the original vnf _ buffer _ count "value", and replacing the timestamp field of the original "value" with the current timestamp of the system, and ending; if the vnf _ buffer _ count does not contain the "key" vnftagContest, the vnftagContest is used as a new "key" to be added to the vnf _ buffer _ count, the count field corresponding to the "value" is set to be 1, and the timestamp field is used as the timestamp of the current system, and the method is ended.
CN202110347584.XA 2021-03-31 2021-03-31 Unified monitoring method for virtual network functions Active CN112925649B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110347584.XA CN112925649B (en) 2021-03-31 2021-03-31 Unified monitoring method for virtual network functions

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110347584.XA CN112925649B (en) 2021-03-31 2021-03-31 Unified monitoring method for virtual network functions

Publications (2)

Publication Number Publication Date
CN112925649A true CN112925649A (en) 2021-06-08
CN112925649B CN112925649B (en) 2021-09-14

Family

ID=76176762

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110347584.XA Active CN112925649B (en) 2021-03-31 2021-03-31 Unified monitoring method for virtual network functions

Country Status (1)

Country Link
CN (1) CN112925649B (en)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113608900A (en) * 2021-10-08 2021-11-05 成都数之联科技有限公司 Method, device, equipment and medium for calling algorithm model
CN114826939A (en) * 2022-04-25 2022-07-29 神州灵云(北京)科技有限公司 Network traffic analysis method of K8S cluster
CN115333928A (en) * 2022-07-12 2022-11-11 中国电信股份有限公司 Network early warning method and device, electronic equipment and storage medium
CN116170341A (en) * 2022-12-23 2023-05-26 中国联合网络通信集团有限公司 Virtualization platform monitoring method, device, system and storage medium
WO2023098557A1 (en) * 2021-12-01 2023-06-08 International Business Machines Corporation Network interface controller aware placement of virtualized workloads

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105471649A (en) * 2015-12-03 2016-04-06 深圳市金立通信设备有限公司 Virtual network performance monitoring system and method, and related equipment
CN105873114A (en) * 2015-01-21 2016-08-17 中兴通讯股份有限公司 Virtualised network function performance monitoring method and a corresponding system
US20180349202A1 (en) * 2017-05-30 2018-12-06 Hewlett Packard Enterprise Development Lp Virtual Network Function Resource Allocation
CN109150567A (en) * 2017-06-19 2019-01-04 中兴通讯股份有限公司 Monitoring method, equipment and the readable storage medium storing program for executing of virtual network function module
CN112511339A (en) * 2020-11-09 2021-03-16 宝付网络科技(上海)有限公司 Container monitoring alarm method, system, equipment and storage medium based on multiple clusters

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105873114A (en) * 2015-01-21 2016-08-17 中兴通讯股份有限公司 Virtualised network function performance monitoring method and a corresponding system
CN105471649A (en) * 2015-12-03 2016-04-06 深圳市金立通信设备有限公司 Virtual network performance monitoring system and method, and related equipment
US20180349202A1 (en) * 2017-05-30 2018-12-06 Hewlett Packard Enterprise Development Lp Virtual Network Function Resource Allocation
CN109150567A (en) * 2017-06-19 2019-01-04 中兴通讯股份有限公司 Monitoring method, equipment and the readable storage medium storing program for executing of virtual network function module
CN112511339A (en) * 2020-11-09 2021-03-16 宝付网络科技(上海)有限公司 Container monitoring alarm method, system, equipment and storage medium based on multiple clusters

Non-Patent Citations (4)

* Cited by examiner, † Cited by third party
Title
HYUNSIK YANG ET AL: "Implementation of VNFC monitoring driver in the NFV architecture", 《 2017 INTERNATIONAL CONFERENCE ON INFORMATION AND COMMUNICATION TECHNOLOGY CONVERGENCE (ICTC)》 *
林凯: "基于CloudStack的云计算资源监控系统的研究与实现", 《中国优秀硕士学位论文全文数据库信息科技辑》 *
甘兵: "docker:快速构建容器监控系统cAdvisor+InfluxDB+Grafana", 《51CTO博客,HTTPS://WWW.BAIDU.COM/LINK?URL=XCBI-LIGQT5L3M8CJWMPO7X80AHK2KZRHALQMIL042XCLS0KJRNAZE0VZHLZ7R2C&WD=&EQID=E75BE5400000025100000006610B527A》 *
韩先超: "虚拟网络监控进阶之路-DeepFlow", 《CSDN博客,HTTPS://WWW.BAIDU.COM/LINK?URL=AZSN82DBAVNJ2JJJTRQX-PVGQRTFKGHQYAU8LR7XVWREYZ33CGPEQR1GZYL9YJP-ZTMUBXMDN5AHKRGE2LCDB4WPY_IQVAQNQDWUPAAS74C&WD=&EQID=86ABA665002BC4B300000006610B518D》 *

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113608900A (en) * 2021-10-08 2021-11-05 成都数之联科技有限公司 Method, device, equipment and medium for calling algorithm model
CN113608900B (en) * 2021-10-08 2021-12-10 成都数之联科技有限公司 Method, device, equipment and medium for calling algorithm model
WO2023098557A1 (en) * 2021-12-01 2023-06-08 International Business Machines Corporation Network interface controller aware placement of virtualized workloads
CN114826939A (en) * 2022-04-25 2022-07-29 神州灵云(北京)科技有限公司 Network traffic analysis method of K8S cluster
CN114826939B (en) * 2022-04-25 2024-04-23 神州灵云(北京)科技有限公司 Network traffic analysis method of K8S cluster
CN115333928A (en) * 2022-07-12 2022-11-11 中国电信股份有限公司 Network early warning method and device, electronic equipment and storage medium
CN116170341A (en) * 2022-12-23 2023-05-26 中国联合网络通信集团有限公司 Virtualization platform monitoring method, device, system and storage medium
CN116170341B (en) * 2022-12-23 2024-04-09 中国联合网络通信集团有限公司 Virtualization platform monitoring method, device, system and storage medium

Also Published As

Publication number Publication date
CN112925649B (en) 2021-09-14

Similar Documents

Publication Publication Date Title
CN112925649B (en) Unified monitoring method for virtual network functions
CN106100999B (en) Image network flow control methods in a kind of virtualized network environment
CN100456687C (en) Network failure real-time relativity analysing method and system
CA2287769C (en) Management system for a multi-level communication network
CN103235820B (en) Date storage method and device in a kind of group system
CN111522653A (en) Container-based network function virtualization platform
CN104219327B (en) Distributed cache system
CN102427410B (en) Method, device and system for processing fiber channel identity, and network equipment
CN103188112A (en) Network flow detection method and network flow detection device
CN103376772B (en) Method for monitoring performance of embedded type numerical control system
CN112769602B (en) System and method for unified configuration management of white box switch and network operating system
CN111984505B (en) Fortune dimension data acquisition device and acquisition method
Bruschi et al. Openvolcano: An open-source software platform for fog computing
US20240022507A1 (en) Information flow recognition method, network chip, and network device
US20060126534A1 (en) Method and mechanism for identifying an unmanaged switch in a network
KR102423039B1 (en) Real-time packet data storing method and apparatus for mass network monitoring
CN109831473A (en) Logistics service providing method and equipment
CN107528794B (en) Data processing method and device
CN109358820B (en) Data access method and device, electronic equipment and computer readable storage medium
CN116708266A (en) Cloud service topological graph real-time updating method, device, equipment and medium
CN101577660B (en) Method and device for acquiring label forwarding list item and forwarding message
KR102423038B1 (en) Real-time packet data collection method and apparatus for mass network monitoring
CN114500418A (en) Data statistical method and related device
CN113485749A (en) Data management and control system and data management and control method
Liao et al. Distributed and efficient network hypervisor for SDN virtualization

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant