CN114816914A

CN114816914A - Data processing method, equipment and medium based on Kubernetes

Info

Publication number: CN114816914A
Application number: CN202210425398.8A
Authority: CN
Inventors: 刘洋; 李煌东; 冯军; 徐彤; 周月侨; 刘恺
Original assignee: Alibaba China Co Ltd
Current assignee: Alibaba China Co Ltd
Priority date: 2022-04-21
Filing date: 2022-04-21
Publication date: 2022-07-29

Abstract

The application provides a data processing method, equipment and medium based on Kubernetes. The method comprises the following steps: monitoring calling events among target objects in a kernel of at least one slave node running an application program; analyzing the calling event to determine various initial calling related information corresponding to the calling event; associating the initial calling related information with the dimension of the target object to form various target calling related information; and storing the relevant information of each target call into a corresponding database. The probe for collecting the related information when the target object is called is compiled by using the programming language which can be executed by the kernel, so that the unification of the programming language can be realized; and the multiple initial call related information collected in the kernel is associated by the dimension of the target object to form multiple target call related information and stored, so that the target call related information can be obtained from the corresponding database, and the running state of the target object can be known in an all-around manner.

Description

Data processing method, equipment and medium based on Kubernetes

Technical Field

The present disclosure relates to cloud computing technologies, and in particular, to a data processing method, device, and medium based on Kubernetes.

Background

Kubernetes (K8 s for short) is an open source system for automatically deploying, extending, and managing containerized applications. The method has the characteristics of portability, expandability, automatic deployment and the like, so that the method is widely applied. However, as the kubernets architecture becomes more complex, the operation and maintenance of kubernets becomes more difficult.

At present, when the kubernets are operated and maintained, related information is acquired when target objects such as application programs or workloads and the like are called, when the related information is acquired, multiple monitoring control tools are generally deployed in each layer of a kubernets framework, a point-burying technology is adopted for acquiring corresponding data, so that the related information cannot be acquired comprehensively, programming languages of the point-burying technology are not uniform, the acquired related information cannot be effectively correlated, and finally the running state of the target objects cannot be acquired comprehensively.

Disclosure of Invention

The application provides a data processing method, equipment and medium based on Kubernets, which are used for solving the problems that the relevant information of a calling event cannot be comprehensively collected, the programming language of a buried point technology is not uniform, the collected relevant information cannot be effectively correlated, and finally the running state of a target object cannot be comprehensively known.

In a first aspect, the present application provides a data processing method based on Kubernetes, including:

monitoring calling events among target objects in a kernel of at least one slave node running an application program; the slave node is positioned in a Kubernetes cluster;

analyzing the calling event to determine various initial calling related information corresponding to the calling event;

associating the initial calling related information with the dimension of the target object to form various target calling related information;

and storing the relevant information of each target call into a corresponding database.

In a second aspect, the present application provides an electronic device comprising: a processor, and a memory communicatively coupled to the processor;

the memory stores computer-executable instructions;

the processor executes the computer execution instructions stored in the memory to realize the Kubernets-based data processing method.

In a third aspect, the present application provides a computer-readable storage medium, in which computer-executable instructions are stored, and when the computer-executable instructions are executed by a processor, the kubernets-based data processing method is implemented.

According to the data processing method, the data processing equipment and the data processing medium based on the Kubernetes, a calling event between target objects is monitored in an inner core of at least one slave node running an application program; the slave node is positioned in a Kubernetes cluster; analyzing the calling event to determine various initial calling related information corresponding to the calling event; associating the initial calling related information with the dimension of the target object to form various target calling related information; and storing the relevant information of each target call into a corresponding database. Monitoring calling events among the target objects in the kernel, and compiling a monitoring program by using a programming language which can be executed by the kernel to complete monitoring of the calling events among the target objects, so that the unification of the programming language is realized, and the cost of a buried point is reduced; monitoring calling events among the target objects in the kernel, and analyzing the calling events to comprehensively obtain various initial calling related information and uniformly collect various initial calling related information when the target objects are called as various calling related information passes through the kernel; the collected multiple kinds of initial calling related information are associated according to the dimension of the target object to form multiple kinds of target calling related information, and then the collected related information can be effectively associated to form multiple kinds of target calling related information; various target calling related information is stored in the database, so that the target calling related information can be acquired from the corresponding database, and the running state of the target object can be known in an all-around manner.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and together with the description, serve to explain the principles of the application.

Fig. 1 is a schematic view of an application scenario provided in the present application;

fig. 2 is a flowchart of a data processing method based on Kubernetes according to an embodiment of the present application;

fig. 3 is a flowchart of a data processing method based on Kubernetes according to a third embodiment of the present application;

fig. 4 is a schematic diagram illustrating a principle of a data processing method based on Kubernetes according to the present application;

fig. 5 is a flowchart of a data processing method based on Kubernetes according to a fifth embodiment of the present application;

FIG. 6 is a schematic diagram of another application scenario provided by the present application;

FIG. 7 is a topological diagram between the target objects provided herein;

FIG. 8 is a schematic diagram of displaying various target call related information and alarm information provided herein;

FIG. 9 is a schematic diagram of a warning message display provided by the present application;

FIG. 10 is a schematic illustration of key indicators provided herein;

fig. 11 is a schematic diagram illustrating display of target network invocation information provided by the present application;

fig. 12 is a schematic structural diagram of an electronic device according to an eighth embodiment of the present application.

With the above figures, there are shown specific embodiments of the present application, which will be described in more detail below. These drawings and written description are not intended to limit the scope of the inventive concepts in any manner, but rather to illustrate the inventive concepts to those skilled in the art by reference to specific embodiments.

Detailed Description

Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.

The terms referred to in this application are explained first:

kubernetes: abbreviated K8S is a container orchestration tool for managing applications running in containers. Kubernets works by breaking an application into logical units by grouping containers to facilitate management and discovery. Kubernets may run on bare machines, virtual machines, cloud instances, or cloud computing management platforms.

A master node: the English is called Master Node, which is the control panel or control plane of Kubernetes. Is where decisions about the clusters are made, such as scheduling, and probing/responding to cluster events, all control commands executed by the clusters are received and processed by the Master server. The master node in the cluster is mainly responsible for the following tasks: is responsible for managing all nodes (Node in English); the node is responsible for scheduling the Pod to run on which nodes; and is responsible for controlling all states in the cluster operation process.

The slave node: the operating node is the operating node in Kubernetes, and the operating node is responsible for operating the container and provides an operating environment of the Kubernetes. The following tasks are mainly responsible in the cluster: responsible for managing all containers (Container in english); and the monitoring control/report of the running state of all the Pod is responsible.

Pod: the system is the minimum unit which can be created and deployed in Kubernetes, is an application example in a Kubernetes cluster, and is always deployed on the same Node. The Pod includes one or more containers and also includes resources shared by the containers such as storage and network.

The prior art is described in detail below:

in the prior art, a point-buried technology is usually adopted to collect relevant information when a target object in each layer of a Kubernetes framework is called, because the relevant information when the target object in each layer is called is collected, the collected relevant information cannot be effectively associated, and a user needs to open a plurality of pages when looking up the relevant information when the target object is called, so that all the obtained relevant information collected when the target object is called can be looked up. Moreover, since the relevant information when the target object is called is collected by using the embedded point technology, the programming language of the embedded point technology to be used needs to be determined according to the programming language of the target object, so that the programming language using the embedded point technology is not uniform, the time cost for writing the embedded point program is high due to the non-uniform programming language using the embedded point technology, and the collected relevant information cannot be effectively associated. And monitoring control tools used for collecting relevant information when a target object is called are not uniform, such as various monitoring control tools like Prometheus, Zabbix, cloud monitoring control and the like, tool interfaces and data are not uniform, and one-stop monitoring control cannot be realized.

The inventor finds in research that, in order to unify the programming language of the embedded point technology, relevant information is collected in a kernel when a target object is called, and a monitor program can be written by the programming language which can be executed by the kernel to monitor calling events among the target objects, so that the unification of the programming language can be realized, and the embedded point cost is reduced. Since various calling related information passes through the kernel, the related information of the target object during calling is collected in the kernel, but the related information of the target object during calling is not collected in each layer, the collected related information of the target object during calling can be uniformly and comprehensively collected, the collected related information is correlated according to the dimension of the target object, the collected related information can be effectively correlated, various target calling related information is stored in a database, the related information of each target calling can be obtained from the corresponding database, and the running state of the target object can be comprehensively known.

Fig. 1 is a schematic view of an application scenario provided in the present application, and as shown in fig. 1, a network architecture corresponding to the present application includes: the user private cloud and the control private cloud are constructed by Kubernets. The user private cloud comprises a slave node1, a slave node 2, a slave node 3 and a master node, and the management and control private cloud comprises a database. The method comprises the steps that a monitor is deployed on each slave node, so that initial calling related information is obtained from a kernel of the node, the initial calling related information comprises initial network index information (whole English name: Metric), initial network calling information and the like (whole English name: Trace), and Log information (whole English name: Log) can be obtained, wherein the Metric, Trace and Log are abbreviated as MTL; as shown in fig. 1, the monitor may filter a probe for an extension packet, then associate the acquired MTL with a dimension of a target object to form various target call related information, and store the various target call related information in a database that manages and controls a private cloud; after the various target calling related information is stored in the database, the various target calling related information of the target object can be acquired from the database, and the running state of the target object can be obtained in an all-around manner.

The following describes the technical solutions of the present application and how to solve the above technical problems with specific embodiments. The following several specific embodiments may be combined with each other, and details of the same or similar concepts or processes may not be repeated in some embodiments. Embodiments of the present application will be described below with reference to the accompanying drawings.

Example one

Fig. 2 is a flow chart of a data processing method based on Kubernetes according to an embodiment of the present application, which is provided for a problem that a program language using a point burying technique is not uniform, and collected related information cannot be effectively associated with each other, which eventually results in that an operation state of a target object cannot be known in an all-around manner. The method in this embodiment is applied to a Kubernetes-based data processing apparatus, which may be located in an electronic device. Among other things, an electronic device may be a physical machine or a virtual machine that represents various forms. Such as local physical servers, virtual machines, cloud hosts, laptop computers, desktop computers, workstations, blade servers, mainframe computers, and other suitable computers.

As shown in fig. 2, the method comprises the following specific steps:

step S101, monitoring calling events among target objects in a kernel of at least one slave node running the application program.

Wherein the slave node is located in a kubernets cluster. In the embodiment of the application, the target object may be an application program in a Kubernetes cluster; the target object may also be a POD in a Kubernetes cluster.

The embodiment of the present application does not specifically limit the manner of listening for the call event in the kernel, and for example, the listening for the call event in the kernel may be implemented by using a listener or a listening function compiled by a kernel executable language.

And step S102, analyzing the calling event to determine various initial calling related information corresponding to the calling event.

The multiple kinds of initial call related information are information obtained by calling an event, and may include: caller information, callee information, call time, call data, etc. of the call event.

Specifically, the monitored information in the calling event is analyzed, and various information in the calling event is determined as various initial calling related information.

The method for analyzing the call event is not limited in the embodiment of the present application, and illustratively, the monitored information in the call event may be analyzed according to a format of a network protocol.

And step S103, associating the initial calling related information with the dimension of the target object to form various target calling related information.

In the embodiment of the application, the initial calling related information corresponding to the same target object is correlated, and the multiple kinds of initial calling related information correlated by the target object are determined to be multiple kinds of target calling related information. The initial call related information corresponding to the same target object may be the initial call related information of the target object as the caller or the callee, or may be the initial call related information of other target objects in the same link with the callee and the callee, which is not specifically limited in this embodiment of the present application.

Specifically, the way of associating the initial call related information with the dimension of the target object may be: and storing each piece of initial call related information in association with identification information of a target object, wherein the identification information of the target object is information representing a unique target object, and for example, the identification information of the target object may be the name of the target object.

And step S104, storing the relevant information of each target call into a corresponding database.

In the embodiment of the application, after various target calling related information is formed, the target calling related information associated with the dimensionality of the target object is stored in the corresponding database, so that the target calling related information corresponding to the target object can be conveniently obtained in the database.

In the embodiment of the application, calling events among target objects are monitored in a kernel of at least one slave node running an application program; the slave nodes are located in a Kubernetes cluster; analyzing the calling event to determine various initial calling related information corresponding to the calling event; associating the initial calling related information with the dimension of the target object to form various target calling related information; and storing the relevant information of each target call into a corresponding database. Monitoring calling events among the target objects in the kernel, and compiling a monitoring program by using a programming language which can be executed by the kernel to complete monitoring of the calling events among the target objects, so that the unification of the programming language is realized, and the cost of a buried point is reduced; monitoring calling events among the target objects in the kernel, and analyzing the calling events to comprehensively obtain various initial calling related information and uniformly collect various initial calling related information when the target objects are called as various calling related information passes through the kernel; the collected multiple kinds of initial calling related information are associated according to the dimension of the target object to form multiple kinds of target calling related information, and then the collected related information can be effectively associated to form multiple kinds of target calling related information; various target calling related information is stored in the database, so that the target calling related information can be acquired from the corresponding database, and the running state of the target object can be known in an all-around manner.

Example two

On the basis of the first embodiment, for listening to the calling event between the target objects in the kernel of the at least one slave node running the application program in step S101, the extended packet filtering probe eBPF may be used to listen to the calling event between the target objects in the kernel of the at least one slave node running the application program.

The Extended Packet filtering probe (English is called as Extended Berkeley Packet Filter, abbreviated as eBPF) can run a sandbox program in a Linux kernel without changing kernel source codes or loading kernel modules. A user can add a monitoring function in a kernel through the eBPF, and then can monitor calling events among target objects by adopting the eBPF, so that non-invasive monitoring of the calling events can be realized, and various initial calling related information can be determined.

Optionally, one specific implementation manner of using the extended packet filtering probe eBPF to listen to a call event between target objects in a kernel of at least one slave node running an application program includes the following steps:

step S201, configuring a hook function corresponding to the eBPF in a kernel of at least one slave node running the application program.

In the embodiment of the present application, the eBPF is run by the kernel when the call event is triggered, so it may be a function hook or an event-driven programming form. That is, the kernel runs a hook function to monitor the calling events between the target objects.

Step S202, a hook function is adopted to monitor calling events among target objects.

The embodiment of the present invention does not specifically limit the configured hook function, and the hook function corresponding to the eBPF may be written using a programming language that can be executed by a kernel, such as an eBPF script, and configures the hook function in an application program, and triggers the hook function through the configured hook point when a call event runs in the kernel, thereby implementing monitoring of the call event between target objects.

In the embodiment of the application, the extended packet filtering probe eBPF can be used to monitor the calling event between the target objects in the kernel of at least one slave node running the application program, the extended packet filtering probe eBPF can be written using the same programming language, so that the time cost for writing the probe program for monitoring the calling event between the target objects can be reduced, and the extended packet filtering probe eBPF is used to monitor the calling event between the target objects in the kernel without changing the kernel source code or loading the kernel module, so that the non-invasive monitoring of the calling time can be realized.

EXAMPLE III

Fig. 3 is a flowchart of a data processing method based on kubernets according to a third embodiment of the present application, where on the basis of any of the foregoing embodiments, the implementation of the present application relates to a step S102 of analyzing a call event to determine a refinement of an implementation manner of multiple types of initial call related information corresponding to the call event, and as shown in fig. 3, the implementation specifically includes the following steps:

and S301, determining the network protocol type of the calling event.

Specifically, after the call event is monitored, the network protocol type of the call event is analyzed according to the format of the network protocol. The format of the network protocol defines the information contained in the network protocol and the location of the information in the network protocol.

Illustratively, one alternative way to parse the network protocol type of the invocation event according to the format of the network protocol may be: and judging whether the information in the calling event is matched with the information defined by the format of the network protocol, and if the information in the calling event is matched with the information defined by the format of the network protocol, determining that the calling event is the network protocol type corresponding to the format of the network protocol. For example, if the format of the network protocol defines that the network protocol includes a Destination Port number (collectively referred to as Destination Port) and the call event also includes the Destination Port number, the information-Destination Port number in the call event matches the information-Destination Port number defined by the format of the network protocol.

Step S302, if the network protocol type is determined to be the preset network protocol, analyzing various initial calling related information from the calling event according to the format of the preset network protocol.

Wherein a plurality of initial call related information is stored in the cache area.

In the embodiment of the present application, the preset network protocol is a preset network protocol, for example, the preset network protocol may be an HTTP protocol.

In the embodiment of the present application, the manner of determining the network protocol type as the preset network protocol may be: and acquiring the format of the preset network protocol, and if the information in the calling event is matched with the format of the preset network protocol, determining that the network protocol type of the calling event is the preset network protocol. The mode for determining the network protocol type as the preset network protocol may further be: after the network protocol type of the calling event is determined, a preset network protocol is obtained, and whether the network protocol type of the calling event is the preset network protocol or not is judged.

Specifically, after the network protocol type is determined to be the preset network protocol, the information in the call event is acquired according to the position of the information defined by the format of the preset network protocol in the network protocol, and the information acquired from the call event is determined to be various initial call related information.

In the embodiment of the present application, as shown in fig. 1, an extended packet filtering probe may be used to obtain various initial call related information, fig. 4 is a schematic diagram of a data processing method based on Kubernetes provided in the present application, and as shown in fig. 4, a call event is analyzed to determine various initial call related information corresponding to the call event; and associating the initial call related information with the dimension of the target object to form various target call related information, wherein the whole process can be divided into two stages, one stage is a kernel state stage, and the other stage is a user state stage. The kernel mode phase is to monitor the calling event in the kernel and form the related information of each initial call, and the user mode phase is to form various target calling related information.

As shown in fig. 4, the call events that the extended packet filtering probe can listen to in the kernel-mode stage include: after monitoring the kernel calling event, the method can analyze the network protocol type corresponding to the calling event according to the format of the network protocol, and if the network protocol type is a preset network protocol, store various initial calling related information analyzed from the calling event in a ring buffer (all called ring buffer in English), that is, the buffer.

In the embodiment of the application, the network protocol type of the calling event is determined; if the network protocol type is determined to be the preset network protocol, analyzing various initial calling related information from the calling event according to the format of the preset network protocol; the method can acquire various calling events corresponding to the specified analyzable preset network protocol, and can analyze various initial calling related information matched with the format of the preset network protocol from the calling events; thus, various initial call related information corresponding to the call event can be determined.

Example four

On the basis of any of the above embodiments, the implementation of the present application relates to the step S103 of associating each piece of initial invocation related information with the dimension of the target object to form a refinement of the implementation manner of the various pieces of target invocation related information, which specifically includes:

and associating and storing the initial calling related information and the identification information of the target object to form various target calling related information so as to associate the initial calling related information belonging to the same target object through the identification information of the target object.

Specifically, identification information of a target object corresponding to various kinds of initial call related information is determined, the various kinds of initial call related information are associated with the identification information of the corresponding target object, and each piece of initial call related information is associated with the identification information of the corresponding target object for storage.

Optionally, on the basis of storing the initial call related information and the identification information of the target object in an associated manner, the initial call related information may be associated with the dimension of the target object in the user mode stage of the eBPF to form a plurality of target call related information.

In the embodiment of the application, the extended packet filtering probe eBPF can monitor the calling event at the kernel state stage, analyze the calling event to determine a plurality of corresponding initial calling related information, and store each initial calling related information in association with the identification information of the target object; in the user mode stage, the initial calling related information belonging to the same target object can be associated through the identification information of the target object.

Optionally, one implementation manner of storing each initial call related information in association with the identification information of the target object may include the following steps:

step S401, respectively converting the initial call related information and the identification information of the target object into a key-value pair format.

The embodiment of the present application does not specifically limit the manner of converting into the key-value pair format, and for example, if each piece of initial call related information includes a corresponding network protocol type and the network protocol is a DNS protocol, the type is "DNS" after converting into the key-value pair format.

For example, the conversion into the key-value pair format may be: and acquiring a preset key-value pair format, and putting the initial calling related information and the identification information of the target object into the preset key-value pair format so as to respectively convert the initial calling related information and the identification information of the target object into the key-value pair format.

And S402, respectively performing associated storage on the initial calling related information and the identification information of the target object according to a key value pair format to form various target calling related information.

In the embodiment of the application, each piece of initial call related information and the identification information of the target object can be respectively stored in an associated manner according to a key value pair format based on the mode of the label, that is, the label of each piece of initial call related information corresponding to the call event is marked with the identification information of the corresponding target object, so that the associated storage of each piece of initial call related information and the identification information of the target object is realized. Where a Label in k8s is a key-value pair of key and value, where key and value are specified by the user himself.

Illustratively, the associating and storing each initial call related information and the identification information of the target object according to the key-value pair format, and the forming of the multiple target call related information may include: name ═ ack ", type ═ DNS", and node ═ node 1.

Where "ack" is the name of the target object, "DNS" is the network protocol type, and "node 1" is the name of the slave node where the application is located.

Wherein, the Label can also be attached to various resource objects, and the resource objects can be: node, Pod, service, replication controller, etc. Therefore, the identification information of the target object and the corresponding resource object can be respectively associated and stored to form corresponding target call related information.

In the embodiment of the application, the initial calling related information and the identification information of the target object are respectively stored in an associated manner to form various target calling related information; the initial calling related information belonging to the same target object can be associated through the identification information of the target object, after each initial calling related information and the identification information of the target object are respectively associated and stored, various target calling related information corresponding to the target object can be directly obtained from the formed various target calling related information, and further, the monitoring control of the target object can be realized according to the various target calling related information corresponding to the target object.

EXAMPLE five

Fig. 5 is a flow chart of a data processing method based on kubernets according to a fifth embodiment of the present application, where the initial call related information in the embodiment of the present application may include: based on any one of the above embodiments, the embodiment of the present application relates to step S103 of associating each piece of initial invocation-related information with a dimension of a target object to form a refinement of an implementation manner of a plurality of pieces of target invocation-related information, and as shown in fig. 5, specifically includes the following steps:

step S501, identification information of the target object in the calling event is determined.

Specifically, a caller and a callee in the calling event are determined as target objects, and identification information of the caller and identification information of the callee in the calling event are determined, thereby determining identification information of all target objects in the calling event.

Step S502, acquiring resource meta-information of the corresponding target object from a service interface of a main node running the application program according to the identification information of the target object.

The resource meta information of the target object includes resource identification information of the target object, and the resource identification information of the target object may be identification information of a container corresponding to the target object and identification information of a pod corresponding to the target object. The resource meta information may further include performance index data of the resource corresponding to the target object, for example, information about a request rate, a delay duration, and a cpu usage of the container corresponding to the target object.

Fig. 6 is a schematic view of another application scenario provided by the present application, and as shown in fig. 6, the master node has a service interface, and the service interface is english and is called: the API Server in k8s provides an interface for modifying and checking various resource objects of k8s, and is a data bus and a data center of the whole system. The resource meta information of the corresponding target object can be acquired from the service interface of the master node. The embodiment of the present application does not specifically limit the manner of obtaining the resource metadata of the corresponding target object from the service interface of the master node according to the identification information of the target object.

Step S503, the target object identification information, the resource identification information and the initial network index information are correlated to form target network index information.

In the embodiment of the present application, the initial call related information includes initial network index information and initial network call information.

The initial network indicator may be indicator information of a target object analyzed from a call event, and exemplarily, the initial network indicator may include: key metrics and other network metrics. Key indicators may include: number of requests, number of errors, average response time, number of slow calls. Other network metrics may include: traffic, delay, bandwidth, etc.

Step S504, the target object identification information, the resource identification information and the initial network calling information are correlated to form target network calling index information.

The initial network invocation information may be information corresponding to a single invocation event, and includes: calling party target object identification information and called party target object identification information. It can be understood that, if there are more than the caller and the callee in the call link, and there is a forwarder responsible for forwarding in the middle, the initial network call information further includes forwarder identification information.

In the embodiment of the present application, as shown in fig. 4, in the user mode stage of the extension packet filtering probe, multiple kinds of initial call related information may be read from the ring buffer of the ring buffer, network index information in the initial call related information and network call information in the initial call related information are assembled respectively, respectively associating the network calling information and the network index information with the dimension of the target object to form target calling related information, and acquiring resource meta information of a corresponding target object from a service interface of a master node running an application program, associating target object identification information with resource identification information, updating target call related information, forming updated target network index information and target network call index information, and storing the updated target network index information and the target network calling index information in corresponding databases.

In the embodiment of the application, identification information of a target object in a calling event is determined; acquiring resource meta-information of a corresponding target object from a service interface of a main node running an application program according to identification information of the target object; associating the target object identification information, the resource identification information and the initial network index information to form target network index information; associating the target object identification information, the resource identification information and the initial network calling information to form target network calling index information; the target network calling index information and the target network index information can include resource identification information, the resource identification information can be obtained when the target network calling index information and the target network index information are obtained, the resource identification corresponding to the target object can be determined, and further, the running state of the resource corresponding to the target object can be obtained according to the resource identification.

Optionally, on the basis of the fourth embodiment, the step S104 of storing the relevant information of each target call in the corresponding database may specifically include:

storing the target network index information into an index database; and storing the target network calling information into a link tracking database.

In an embodiment of the present application, the database includes: the index database and the link tracking database can collect MTL monitored by the probes in the cluster through the collector, store target network index information into the index database through the collector, and store target network calling information into the link tracking database.

For example, as shown in fig. 6, the Collector may be an OTLP Collector, and target network index information in the multiple kinds of target call related information may be stored in an index database through the OTLP Collector; and collecting various target calling related information through an OTLP Collector, and storing target network calling information in the various target calling related information into a link tracking database. The opentelemetrology Protocol and OTLP Collector are collectors conforming to the opentelemetrology Protocol, and opentelemetrology is a set of standards and tools, and is intended to manage observation-type data, such as trace, metrics, logs, and the like. The OTLP Collector is used to collect MTL monitored by probes in the cluster, i.e., initial call information, initial index information, and log information, and is pulled to the back end by a system monitoring control and alarm system (e.g., Prometheus) for storage.

In the embodiment of the application, target network index information is stored in an index database; the target network calling information is stored in the link tracking database, can be acquired by accessing the index database when the target network index information is needed, and can be acquired by accessing the link tracking database when the target network calling information is needed, so that the data volume for acquiring the target network index information or searching the target network calling information can be reduced, the time for acquiring the target network index information or acquiring the target network calling information is reduced, and further, the time for constructing a topological graph among target objects and the time for displaying various target calling related information can be reduced.

EXAMPLE six

On the basis of any of the above embodiments, the embodiment of the present application may further obtain log information and alarm information of the target object, and specifically includes the following steps:

step S601, log information and alarm information collected from at least one slave node are obtained.

The embodiment of the present application does not specifically limit the manner of obtaining the log information of the slave node, and for example, the log collection agent may be run on the slave node to collect the log, and the log collection agent may be run using a container, and may access the directory where the log files of all application containers on the slave node are located.

The method for acquiring the alarm information in the embodiment of the present application is not specifically limited, and exemplarily, as shown in fig. 6, cAdvisor may be used to acquire performance index data related to a container and a POD, NodeExporter may be used to acquire host node performance index data, and when the performance index data related to the container and the POD or the host node performance index data exceeds a set threshold, the alarm information is generated; the acquired log information can be analyzed, and alarm information is generated according to the analyzed log information; the alarm information may also be obtained by parsing the call event. For example, if the call event is a call failure, generating alarm information, where the alarm information may include an alarm level, a time when the call event occurs, and initial call related information corresponding to the alarm information, such as a network protocol type, a call event type, identification information of a corresponding target object, request content of the call event, and the like. The cAdvisor and the nodeeporter are components in k8s, and the cAdvisor can perform real-time monitoring control and performance data acquisition on resources and containers on the node machine, including the use condition of a CPU (central processing unit), the use condition of a memory, the network throughput and the use condition of a file system. The NodeExporter can collect the operation indexes of the host such as CPU, memory, disk and other information.

Step S602, the log information and the alarm information are stored in association with the identification information of the corresponding target object, so as to associate the log information and the alarm information with the corresponding target object.

In the embodiment of the present application, the manner of storing the log information and the alarm information in association with the corresponding target object is similar to the manner of storing the initial call related information in association with the identification information of the target object in the fourth embodiment, and details are not repeated here. For example, the log information and the alarm information are stored in association with the identification information of the corresponding target object in the corresponding database.

In the embodiment of the application, the log information and the alarm information collected from at least one slave node are obtained, and the log information and the alarm information are stored in association with the identification information of the corresponding target object so as to associate the log information and the alarm information with the corresponding target object; after the log information, the alarm information and the identification information of the corresponding target object are stored in the corresponding database in an associated manner, the log information and the alarm information corresponding to the target object can be acquired from the database according to the identification information of the target object, and more comprehensive monitoring control on the target object is realized according to the log information and the alarm information.

EXAMPLE seven

On the basis of any of the above embodiments, the embodiment of the present application relates to an implementation method for building a topology map after storing information related to each target call in a corresponding database, and specifically includes the following steps:

step S701, responding to a topological graph construction request triggered by a user, and acquiring target network calling information corresponding to each target object in a link tracking database.

In this embodiment of the present application, a user may trigger a topology map construction request for one target object, or may trigger a topology map construction request for one cluster, which is not specifically limited in this embodiment of the present application.

Specifically, a target cluster corresponding to a topological graph construction request triggered by a user is determined, each target object and identification information of the target object in the target cluster are determined, and target network call information corresponding to each target object is obtained from a link tracking database according to the identification information of each target object.

Step S702, determining the identification information of the calling party target object and the identification information of the called party target object in the target network calling information.

Specifically, after the identification information of the calling party target object and the identification information of the called party target object in the target network calling information are determined, a topological graph between the calling party target object and the calling party target object is generated according to the identification information of the calling party target object and the identification information of the called party target object.

And step S703, constructing a topological graph among the target objects according to the identification information of the calling target object and the identification information of the called target object.

Specifically, an icon corresponding to each target object identification information is generated, a connection line is generated between the icon corresponding to the calling target object identification information and the icon corresponding to the called target object identification information, and a topological graph between the target objects is constructed.

Fig. 7 is a topological diagram between target objects provided in the present application, and exemplarily, as shown in fig. 7, target object identification information included in the link tracking database includes: A. b, C, D, E, F, G, H, I are provided. And generating icons corresponding to the identification information of each target object, and connecting the icons corresponding to the identification information of the calling target object in the calling information of the target network with the icons corresponding to the identification information of the called target object by using a dotted line, thereby constructing a topological graph between the target objects. For example, if the calling target object identification information in the target network calling information is a and the called target object identification information is I, the icon corresponding to a and the icon corresponding to I are connected by a dotted line as shown in fig. 7.

Optionally, a control for modifying the topology map may be further provided in the page of the topology map between the target objects. As shown in FIG. 7, controls may be provided to zoom in on the topology map, zoom out on the topology map, and rebuild the topology map.

In the embodiment of the application, target network calling information corresponding to each target object in a link tracking database is obtained in response to a topological graph construction request triggered by a user; determining calling party target object identification information and called party target object identification information in target network calling information; and constructing a topological graph among the target objects according to the identification information of the calling target object and the identification information of the called target object. And a topological graph among the target objects is constructed, so that a user can intuitively know the calling relation among the target objects in the cluster, and the target objects and the cluster in which the target objects are positioned are monitored and controlled.

Optionally, after constructing the topological graph between the target objects, multiple kinds of target invocation related information, log information, and alarm information may also be displayed, and a specific implementation manner may include the following steps:

step S704, responding to an information viewing request of any target object triggered by a user, and acquiring various target calling related information, log information and alarm information according to the identification information of the target object.

The multiple kinds of target invocation related information may include target network index information and target network invocation information.

Specifically, the target network index information may be obtained from an index database, and the target network invocation information may be obtained from a link tracking database. The log information and the alarm information corresponding to the target object can be obtained according to the identification information of the target object from an opposite database in which the log information and the alarm information are stored in association with the identification information of the corresponding target object.

The method for triggering the information viewing request of the target object in the embodiment of the application is not particularly limited, for example, the information viewing request of the target object may be triggered by clicking an icon corresponding to the target object, or the information viewing request of the target object may be triggered by moving a cursor above the icon corresponding to the target object.

Step S705, displaying the related information of the target calling, the log information and the alarm information according to a preset format.

The preset format may be a mode of displaying one or more of various target call related information, log information and alarm information in the same page.

Optionally, in this embodiment of the application, the information viewing request triggering the target object is in different manners, and the corresponding preset formats of the display are different. For example, by clicking an icon corresponding to a target object to trigger an information viewing request of the target object, target call related information, log information and alarm information corresponding to all the target objects can be displayed; one or more of the current target call related information may be displayed by moving the cursor over the icon corresponding to the target object.

Fig. 8 is a schematic diagram for displaying various target call related information and alarm information provided by the present application, and as shown in fig. 8, if an icon corresponding to a target object I is clicked, a topological graph may be displayed on the left side of a page, the name and key index of the target object I may be displayed on the right side of the page, and a component for viewing current alarm information and target network call information may be used. Illustratively, as shown in fig. 10, clicking on the view details under the target object name I, the key indicator page of the target object I may be displayed. Illustratively, as shown in FIG. 9, clicking on the current alert component may display the alert information for target object I. The DNS and HTTP are preset network protocols, which can be used as components for invoking information by the target network. Illustratively, as shown in fig. 11, clicking on the DNS or HTTP component may display the target network invocation information for the corresponding network protocol.

For example, as shown in fig. 9, after clicking the current warning, the warning information corresponding to the target object I may be displayed, for example, two warning information are displayed in fig. 9, and each displayed warning information may include a warning level, a target call related information message, a corresponding protocol Type protocol, a Type of the target object, a time when the warning information is generated, and the like.

In the embodiment of the application, the key indexes may be the number of requests, the number of errors, the average response time, and the number of slow calls corresponding to the target object at the current time, and may also include the number of requests, the number of errors, the average response time, and the number of slow calls corresponding to the target object within a certain time period. Illustratively, as shown in fig. 10, the current request number, error number, average response time, and slow call number of the target object may be displayed above the page. When a user triggers to view a specific key index, the number of requests, the number of errors, the average response time, and the number of slow calls corresponding to the target object in a certain time period may be displayed in a form of a line graph, and the number of requests, the number of errors, the average response time, the number of slow calls, and the like corresponding to the interface of the physical machine corresponding to each target object may be displayed in a form of a list, as in fig. 10,/tb/api,/a is an interface name. As shown in fig. 10, the maximum MAX, minimum Min, and average Avg of the request numbers may also be displayed, and the percentile response time within a certain time may also be displayed in the form of a broken line graph, such as displaying the P99 response time, the P95 response time, the P90 response time, the P50 response time, and the P99 response time, where the P95 response time indicates that the response time is arranged from small to large, and the response time at the 95% position in order is the P95 response time. The meaning of the P90 response time, the P50 response time and the P99 response time is similar to that of the P95 response time, and is not described in detail herein.

Specifically, clicking on HTTP in fig. 8 may display target network invocation information corresponding to the network protocol. As shown in fig. 11, the time, time consumption, status code, Trace ID, Trace source, etc. of the target network call information corresponding to the call event generated by the target object may be displayed in a preset format. Where Trace tracing is a transaction supported by distributed microservice collaboration. A link tracks each invocation event that includes the service provided by the transaction. The Trace ID is a unique identifier for a link Trace. The link tracking can be performed according to the Trace ID, and the request information and the response information in the initial call related information corresponding to each call event under the link tracking can be determined according to the Trace ID, so that the analysis of the request content and the return code in the request information and the response information is facilitated, and the specific fault target object and the specific fault interface can be known.

Exemplarily, in fig. 11, detailed target network invocation information corresponding to the invocation event may also be displayed, for example, if the invocation event generated when the generation time is selected to be 2021-12-1616: 52:15, the corresponding target network invocation information is displayed below the page, as shown in fig. 11, a source end and a destination end may be displayed, the source end is the invoked party, and the destination end is the invoking party; the IP addresses of the calling party and the called party can also be displayed, as shown in fig. 11, the IP address of the called party is specifically 192.168.0.99:17 ×, the identification information of the calling party can also be displayed as I, and the network protocol type HTTP corresponding to the calling event is displayed; the displayed target network calling information can also call request receiving, request processing and request responding execution results, time consuming and time axis in events, and the total time consuming and execution results of the calling events; corresponding request information and response information may also be displayed, as shown in fig. 11, the displayed request information may include a Uniform Resource Locator (URL), request Headers, and the like.

Optionally, after the topological graph between the target objects is constructed, the number of the warning information may be displayed on the icon of the target object according to the warning information corresponding to each target object. As shown in fig. 8 and 9, if there are two pieces of current alarm information corresponding to the target object I, a number 2 is displayed at the upper right corner of the icon corresponding to the target object I to prompt the user that the target object I has two pieces of current alarm information, so that the user can maintain or troubleshoot the target object I.

In the embodiment of the application, in response to an information viewing request of any target object triggered by a user, acquiring various target calling related information, log information and alarm information according to identification information of the target object; displaying related information of various target calls, log information and alarm information according to a preset format; the method can enable a user to acquire the running state of the target object in an all-around manner, and enable the user information to carry out monitoring control, maintenance and other operations on the target object according to the target calling related information, log information and alarm information.

Example eight

Fig. 12 is a schematic structural diagram of an electronic device according to an eighth embodiment of the present application. As shown in fig. 12, the electronic apparatus 100 includes: processor 1001, memory 1002, transceiver 1003, and computer executable instructions stored on memory 1002 and executable on processor 1001.

Wherein, the processor 1001, the memory 1002 and the transceiver 1003 are interconnected in circuit; the transceiver 1003 is used for transceiving data; the memory 1002 is used to store computer-executable instructions; the Kubernetes-based data processing method provided by any of the above method embodiments is implemented when the processor 1001 executes computer-executable instructions stored in the memory 1002.

The embodiment of the present application further provides a computer-readable storage medium, in which computer-executable instructions are stored, and when the computer-executable instructions are executed by a processor, the method provided by any one of the above method embodiments is implemented.

An embodiment of the present application further provides a computer program product, where the program product includes: the at least one processor of the channel service device may read the computer-executable instructions from the readable storage medium, and the at least one processor executes the computer-executable instructions to cause the channel service device to perform the method provided by any one of the above method embodiments.

Other embodiments of the present application will be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the invention following, in general, the principles of the application and including such departures from the present disclosure as come within known or customary practice within the art to which the invention pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the application being indicated by the following claims.

It will be understood that the present application is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the application is limited only by the appended claims.

Claims

1. A data processing method based on Kubernetes is characterized by comprising the following steps:

2. The method of claim 1, wherein listening for invocation events between target objects in a kernel of at least one slave node running an application comprises:

and adopting an extended packet filtering probe eBPF to listen to calling events among target objects in a kernel of at least one slave node running the application program.

3. The method according to claim 2, wherein the listening, with the extended packet filtering probe eBPF, for calling events between target objects in a kernel of at least one slave node running an application comprises:

configuring a hook function corresponding to eBPF in a kernel of at least one slave node running an application program;

and monitoring calling events among target objects by adopting the hook function.

4. The method of claim 1, wherein the parsing the call event to determine a plurality of types of initial call related information corresponding to the call event comprises:

determining a network protocol type of the calling event;

if the network protocol type is determined to be the preset network protocol, analyzing various initial calling related information from the calling event according to the format of the preset network protocol, wherein the various initial calling related information is stored in a cache region.

5. The method of claim 2, wherein associating each initial call related information with a dimension of a target object to form a plurality of target call related information comprises:

6. The method according to claim 5, wherein the associating and storing each initial call related information with the identification information of the target object to form a plurality of target call related information comprises:

respectively converting the initial calling related information and the identification information of the target object into a key-value pair format;

and respectively performing associated storage on the initial calling related information and the identification information of the target object according to the key value pair format to form various target calling related information.

7. The method of claim 5, wherein associating the initial call related information with a dimension of the target object to form a plurality of target call related information comprises:

and in the user mode stage of the eBPF, correlating the initial call related information with the dimension of the target object to form various target call related information.

8. The method of claim 5, wherein the initial call related information comprises: initial network index information and initial network call information;

the associating of the initial call related information with the dimension of the target object to form various target call related information comprises the following steps:

determining identification information of a target object in a calling event;

acquiring resource meta-information of a corresponding target object from a service interface of a main node running an application program according to the identification information of the target object; the resource meta-information comprises resource identification information;

associating the target object identification information, the resource identification information and the initial network index information to form target network index information;

and associating the target object identification information, the resource identification information and the initial network calling information to form target network calling index information.

9. The method of claim 7, wherein storing information related to each target call in a corresponding database comprises:

storing the target network index information into an index database;

and storing the target network calling information into a link tracking database.

10. The method of claim 1, further comprising:

acquiring log information and alarm information acquired from at least one slave node;

and performing association storage on the log information and the alarm information and the identification information of the corresponding target object so as to associate the log information and the alarm information with the corresponding target object.

11. The method of any one of claims 1-10, further comprising:

responding to a topological graph construction request triggered by a user, and acquiring target network calling information corresponding to each target object in a link tracking database;

determining calling party target object identification information and called party target object identification information in the target network calling information;

and constructing a topological graph among the target objects according to the identification information of the calling target object and the identification information of the called target object.

12. The method of claim 9, further comprising:

responding to an information viewing request of any target object triggered by a user, and acquiring various target calling related information, the log information and the alarm information according to the identification information of the target object;

and displaying the related information of the various target calls, the log information and the alarm information according to a preset format.

13. An electronic device, comprising: a processor, and a memory communicatively coupled to the processor;

the memory stores computer-executable instructions;

the processor executes computer-executable instructions stored by the memory to implement the method of any of claims 1-11.

14. A computer-readable storage medium having computer-executable instructions stored therein, which when executed by a processor, are configured to implement the method of any one of claims 1-11.