CN117950953A - EBPF and SKYWALKING technology-based distributed system performance tracking method - Google Patents

EBPF and SKYWALKING technology-based distributed system performance tracking method Download PDF

Info

Publication number
CN117950953A
CN117950953A CN202410154396.9A CN202410154396A CN117950953A CN 117950953 A CN117950953 A CN 117950953A CN 202410154396 A CN202410154396 A CN 202410154396A CN 117950953 A CN117950953 A CN 117950953A
Authority
CN
China
Prior art keywords
distributed system
preset
data
tracking
performance
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202410154396.9A
Other languages
Chinese (zh)
Inventor
乔献强
罗焕平
陈东杰
唐丽晶
李健新
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenzhen Mastercom Technology Corp
Original Assignee
Shenzhen Mastercom Technology Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenzhen Mastercom Technology Corp filed Critical Shenzhen Mastercom Technology Corp
Priority to CN202410154396.9A priority Critical patent/CN117950953A/en
Publication of CN117950953A publication Critical patent/CN117950953A/en
Pending legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/30Monitoring
    • G06F11/3003Monitoring arrangements specially adapted to the computing system or computing system component being monitored
    • G06F11/3006Monitoring arrangements specially adapted to the computing system or computing system component being monitored where the computing system is distributed, e.g. networked systems, clusters, multiprocessor systems
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/30Monitoring
    • G06F11/34Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
    • G06F11/3466Performance evaluation by tracing or monitoring
    • G06F11/3476Data logging

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Quality & Reliability (AREA)
  • General Physics & Mathematics (AREA)
  • Computing Systems (AREA)
  • Computer Hardware Design (AREA)
  • Mathematical Physics (AREA)
  • Debugging And Monitoring (AREA)

Abstract

The application discloses a distributed system performance tracking method based on eBPF and SKYWALKING technologies, and belongs to the technical field of information processing. According to the application, the real-time performance data of the distributed system is obtained, and the real-time performance data is subjected to structural transformation to obtain the standard performance data, wherein the standard performance data is used for displaying the tracking information contained in the real-time performance data, and the distributed system is tracked by analyzing the standard performance data into the tracking chart according to the preset SKYWALKING program, so that the technical problem of poor monitoring and performance tuning effects of the existing call chain tracking method on the distributed system is solved.

Description

EBPF and SKYWALKING technology-based distributed system performance tracking method
Technical Field
The present application relates to the field of information processing technologies, and in particular, to a server performance tracking method, device, electronic apparatus, and computer readable storage medium based on eBPF and SKYWALKING technologies.
Background
With the development of internet technology, distributed systems are widely used, so that computing, storage and network resources are virtualized, and flexible allocation and management of resources are realized, so that an asynchronous call chain needs to be monitored to ensure system performance and stability.
The existing method for monitoring the asynchronous call chain generally directly places the synchronous call chain tracking method into an asynchronous environment for use, but with the increase of the scale and complexity of a distributed system, the monitoring requirement on performance is rapidly increased, and the traditional method has the technical problems of high delay, low data transmission efficiency, high network bandwidth pressure and low processing efficiency on large-scale performance data in the asynchronous environment, so that the problems of low positioning and solving efficiency on system performance bottlenecks and poor capturing effect on global performance information occur, and therefore, the existing call chain tracking method has poor effects on monitoring and performance tuning of the distributed system.
The foregoing is provided merely for the purpose of facilitating understanding of the technical solutions of the present application and is not intended to represent an admission that the foregoing is prior art.
Disclosure of Invention
The application mainly aims to provide a distributed system performance tracking method, a distributed system performance tracking device and a distributed system performance tuning method, and aims to solve the technical problem that an existing call chain tracking method is poor in distributed system monitoring and performance tuning effects.
To achieve the above object, the present application provides a distributed system performance tracking method based on eBPF and SKYWALKING technologies, which includes:
acquiring real-time performance data of a distributed system;
performing structural transformation on the real-time performance data to obtain standard performance data, wherein the standard performance data is used for displaying tracking information contained in the real-time performance data;
according to a preset SKYWALKING program, the standard performance data is analyzed into a tracking chart to track the distributed system.
Optionally, the step of performing structural transformation on the real-time performance data to obtain standard performance data includes:
compressing the real-time performance data to obtain compression performance data, and storing the compression performance data into a preset data storage area;
Transmitting the compression performance data in the preset data storage area according to a preset transmission program and a preset time interval;
and decompressing the compression performance data into standard performance data under the standard data structure according to a standard data structure defined by a preset data mapping program.
Optionally, after the step of performing structural transformation on the real-time performance data to obtain standard performance data, the distributed system performance tracking method further includes:
Acquiring a communication mechanism supported by the preset data mapping program;
And transmitting the standard performance data to the preset SKYWALKING program according to the communication mechanism.
Optionally, the step of acquiring real-time performance data of the distributed system includes:
acquiring real-time process information of the distributed system;
Determining a data acquisition logic for acquiring data of the distributed system according to the real-time process information;
and acquiring real-time performance data generated under the real-time process information according to the data acquisition logic.
Optionally, before the step of acquiring real-time performance data of the distributed system, the distributed system performance tracking method further includes:
Acquiring a preset storage version and a preset data port for real-time tracking of the distributed system;
And performing environment deployment on a preset SKYWALKING program according to the preset storage version and the preset data port.
Optionally, the step of tracking the distributed system by parsing the standard performance data into a tracking graph according to a preset SKYWALKING program includes:
Acquiring a preset time index and a preset performance index;
determining a chart template for analyzing the standard performance data according to the preset time index and the preset performance index;
generating a tracking chart under the preset time index according to the chart template and the preset SKYWALKING program;
the distributed system is tracked by showing the tracking graph.
Optionally, after the step of tracking the distributed system by parsing the standard performance data into a tracking chart according to a preset SKYWALKING program, the distributed system performance tracking method further includes:
Acquiring a preset performance threshold;
and outputting a performance abnormality alarm when the standard performance data reaches the preset performance threshold.
In addition, to achieve the above object, the present application further provides a distributed system performance tracking device based on eBPF and SKYWALKING technologies, the distributed system performance tracking device including:
the acquisition module is used for acquiring real-time performance data of the distributed system;
The conversion module is used for carrying out structural conversion on the real-time performance data to obtain standard performance data, wherein the standard performance data is used for displaying tracking information contained in the real-time performance data;
And the tracking module is used for tracking the distributed system by analyzing the standard performance data into a tracking chart according to a preset SKYWALKING program.
Optionally, the conversion module is further configured to:
compressing the real-time performance data to obtain compression performance data, and storing the compression performance data into a preset data storage area;
Transmitting the compression performance data in the preset data storage area according to a preset transmission program and a preset time interval;
and decompressing the compression performance data into standard performance data under the standard data structure according to a standard data structure defined by a preset data mapping program.
Optionally, the distributed system performance tracking device is further configured to:
Acquiring a communication mechanism supported by the preset data mapping program;
And transmitting the standard performance data to the preset SKYWALKING program according to the communication mechanism.
Optionally, the acquiring module is further configured to:
acquiring real-time process information of the distributed system;
Determining a data acquisition logic for acquiring data of the distributed system according to the real-time process information;
and acquiring real-time performance data generated under the real-time process information according to the data acquisition logic.
Optionally, the distributed system performance tracking device is further configured to:
Acquiring a preset storage version and a preset data port for real-time tracking of the distributed system;
And performing environment deployment on a preset SKYWALKING program according to the preset storage version and the preset data port.
Optionally, the step of tracking the distributed system by parsing the standard performance data into a tracking graph according to a preset SKYWALKING program includes:
Acquiring a preset time index and a preset performance index;
determining a chart template for analyzing the standard performance data according to the preset time index and the preset performance index;
generating a tracking chart under the preset time index according to the chart template and the preset SKYWALKING program;
the distributed system is tracked by showing the tracking graph.
Optionally, the distributed system performance tracking device is further configured to:
Acquiring a preset performance threshold;
and outputting a performance abnormality alarm when the standard performance data reaches the preset performance threshold.
The application also provides an electronic device comprising: the video compression system comprises a memory, a processor and a distributed system performance tracking program stored on the memory and capable of running on the processor, wherein the video compression program is configured to realize the steps of the distributed system performance tracking method.
The present application also provides a readable storage medium, which is a computer readable storage medium having stored thereon a program for implementing a distributed system performance tracking method, the program for implementing the distributed system performance tracking method being executed by a processor to implement the steps of the distributed system performance tracking method as described above.
The application obtains the real-time performance data of the distributed system; performing structural transformation on the real-time performance data to obtain standard performance data, wherein the standard performance data is used for displaying tracking information contained in the real-time performance data; according to a preset SKYWALKING program, the standard performance data is analyzed into a tracking chart so as to track the distributed system, so that the real-time performance data of the distributed system is converted into tracking information containing the distributed system through compression operation, and the analyzed standard performance data can be directly analyzed, so that the tracking chart corresponding to the standard performance data is generated according to the preset performance monitoring system, and the distributed system is tracked.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the application and together with the description, serve to explain the principles of the application.
In order to more clearly illustrate the embodiments of the application or the technical solutions of the prior art, the drawings which are used in the description of the embodiments or the prior art will be briefly described, and it will be obvious to a person skilled in the art that other drawings can be obtained from these drawings without inventive effort.
FIG. 1 is a flowchart of a distributed system performance tracking method according to an embodiment of the present application;
FIG. 2 is an algorithm frame diagram of a distributed system performance tracking method according to an embodiment of the present application;
FIG. 3 is a flowchart of a distributed system performance tracking method according to a second embodiment of the present application;
FIG. 4 is a schematic block diagram illustrating a performance tracking device of a distributed system according to a third embodiment of the present application;
Fig. 5 is a schematic diagram of an apparatus structure of a hardware operating environment related to a distributed system performance tracking method according to a fourth embodiment of the present application.
The achievement of the objects, functional features and advantages of the present application will be further described with reference to the accompanying drawings, in conjunction with the embodiments.
Detailed Description
It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the application.
Example 1
EBPF (Extended Berkeley PACKET FILTER ) data acquisition: eBPF technology is a mechanism that enables custom code to be injected into a kernel without modifying the kernel source code. Its flexibility and efficiency make it an ideal choice for capturing kernel-level information. Particularly in the aspect of network data packet capturing, the eBPF technology can realize low-overhead real-time monitoring, thereby providing a new idea of tracking a call chain in an asynchronous environment. Using eBPF register Tracepoint (trace point) in kernel space, a system call entry point is selected, eBPF mapping is defined by trace_ execve (trace execution) function to enable trace and performance data collection for critical system calls.
LZ4 (Lempel-Ziv-4, lossless digital compression) compression algorithm: when compressing and decompressing, the two parties can generate identical dictionaries with data according to the compressing or decompressing process, and then record the matched data before by adopting offset and matching length to replace the original data, thereby achieving the effect of compressing the data. An LZ4 compression algorithm is introduced into eBPF programs to compress the acquired performance data in real time, so that efficient transmission of the performance data is realized, and the transmission delay and the network bandwidth requirement are reduced.
SKYWALKING (observable platform) visualization: SKYWALKING as an open source distributed tracking system provides a powerful tool for monitoring and tuning. The system can track the calling relation among all components of the distributed system in real time, and provide the overall performance of the visual interface display system, so that a developer can easily understand the running condition of the system, and accelerate the problem positioning and solving process. The SKYWALKING user program can periodically obtain compressed data from eBPF kernel mapping, decompress and restore the compressed data into original performance data by using LZ4 for asynchronous call chain tracking and performance monitoring. And the tracking result of the calling chain is displayed in real time through a visual interface, so that visual system performance monitoring and analysis are provided.
Based on this, the present application proposes a distributed system performance tracking method based on eBPF and SKYWALKING technologies of the first embodiment, which includes:
Step S10, acquiring real-time performance data of a distributed system;
Step S20, carrying out structural transformation on the real-time performance data to obtain standard performance data, wherein the standard performance data is used for displaying tracking information contained in the real-time performance data;
step S30, according to a preset SKYWALKING program, the standard performance data is analyzed into a tracking chart so as to track the distributed system.
It should be noted that, the distributed system is used for representing a system needing to perform asynchronous call chain tracking and performance monitoring, specifically may be a distributed call chain tracking system, the real-time performance data is used for representing performance data generated by a system process executed in real time by the current distributed system, specifically may include operation efficiency, memory occupation condition, network flow and the like of a central processing unit, the preset SKYWALKING program is used for representing a monitoring program capable of regularly acquiring and interpreting compressed data, the standard performance data is used for representing data readable by the obtained preset SKYWALKING program after compressing and decompressing the real-time performance data, and specifically may be a data structure matched with the skywalking program.
Additionally, it should be noted that the trace chart is used for representing the presentation form of the standard performance data according to the performance index and the asynchronous call chain in the distributed system, specifically, a visual chart, so that the user can quickly locate the problem point in the distributed system according to the visual trace chart, thereby realizing real-time trace and visual presentation of the asynchronous call chain to the distributed system.
In one possible implementation manner, the acquiring real-time performance data of the distributed system may specifically select an entry point for tuning in the distributed system through a eBPF program, and define and initialize a mapping space for storing system data, so as to start acquiring a real-time process of the distributed system, so as to acquire performance data obtained in the process.
Additionally, in one possible implementation, in SKYWALKINGAGENT (observable platform agency), the eBPF receiver is configured to support both gRPC and HTTP protocols, so that the data receiving service of SKYWALKINGAGENT can be started and a firm connection is established with the back end of SKYWALKING, ensuring that the agency layer can accurately parse the data collected from eBPF programs and perform efficient processing in the back end system. And opening skywalkingweb an interface to observe asynchronous call chain tracking data, and observing data collected by the interface tracking eBPF, so that the reliability of the real-time monitoring and tracking functions of the system is ensured.
For example, to facilitate understanding of the technical concept or technical principle of the present application, referring to fig. 2, an algorithm framework diagram is provided in fig. 2, and in the eBPF procedure, tracepoint (tracking point) is taken as a system call entry point, so as to implement tracking and performance data collection of a key system call. the trace execve function defines eBPF mapping to securely store performance data in kernel space. And an LZ4 algorithm is introduced to compress the acquired data in real time, so that the transmission delay and the network bandwidth requirement are effectively reduced. The compressed data is stored as a bridge in eBPF maps, connecting efficient data transfer in kernel space and user space. In user space, SKYWALKING programs periodically obtain compressed data from eBPF kernel maps, which are quickly decompressed using the LZ4 library. The decompressed data maintains the original format and structure for SKYWALKING asynchronous call chain tracking and performance monitoring. Through the integrated design, efficient performance data transmission and comprehensive system monitoring are realized, and the real-time performance and accuracy of monitoring are improved.
As an example, steps S10 to S30 include: and acquiring real-time performance data of the distributed system, and converting the real-time performance data into standard performance data by compressing the real-time performance data, wherein the standard performance data is used for displaying tracking information of the distributed system, and tracking the distributed system by analyzing the standard performance data into a tracking chart according to a preset SKYWALKING program.
The application obtains the real-time performance data of the distributed system; converting the real-time performance data into standard performance data by compressing the real-time performance data, wherein the standard performance data is used for displaying tracking information of the distributed system; according to a preset SKYWALKING program, the distributed system is tracked by analyzing the standard performance data into a tracking chart, the real-time performance data of the distributed system is converted into the tracking information containing the distributed system through compression operation, and the analyzed standard performance data can be directly analyzed, so that the tracking chart corresponding to the standard performance data is generated according to the preset performance monitoring system, and the distributed system is tracked.
The step of performing structural transformation on the real-time performance data to obtain standard performance data comprises the following steps:
Step A10, compressing the real-time performance data to obtain compression performance data, and storing the compression performance data into a preset data storage area;
step A20, transmitting the compression performance data in the preset data storage area according to a preset transmission program and a preset time interval;
and step A30, decompressing the compression performance data into standard performance data under the standard data structure according to a standard data structure defined by a preset data mapping program.
It should be noted that, the compression performance data is used to represent data obtained after compressing the real-time performance according to a preset compression algorithm, where the preset compression algorithm may be specifically an LZ4 compression algorithm, the preset data storage area may be specifically a storage area mapped in the kernel space by eBPF, the preset transmission program is used to represent a program for acquiring and transmitting the compression performance data, and specifically an API (Application Programming Interface ) mapping method, and the preset time interval may be specifically an interval of every 1 hour, every 2 hours, or every half day, and the compression performance data will be periodically acquired in the kernel space mapped by eBPF by the API mapping method and then transmitted into the user space mapped by eBPF.
Additionally, it should be noted that, the standard data structure is used to represent a data structure that meets the data analysis standard, that is, a data structure that can be directly transmitted with the preset SKYWALKING program and directly analyzed by the preset SKYWALKING program, specifically, the real-time performance data may be converted into a data structure that includes core call information and other related call details, so that the preset SKYWALKING program, that is, the external monitoring system skywalking, may directly convert the standard performance data into the trace chart.
In one possible implementation manner, after the real-time performance data is collected, an LZ4 compression algorithm may be invoked to compress the real-time performance data, and the compressed real-time performance data is converted into compressed performance data and stored in a preset eBPF map to reduce the amount of data stored and transferred in kernel space. Then, the data in the mapping of the kernel space eBPF is periodically acquired in the user space by using a preset data acquisition statement, so that efficient information acquisition is realized on the kernel layer by eBPF technology, comprehensive system performance analysis is ensured, meanwhile, modification of kernel source codes is avoided according to low invasiveness and high efficiency of eBPF technology, the cost of system maintenance and upgrading is reduced, and the risk of system stability is reduced.
As an example, steps a10 to a30 include: compressing real-time performance data to obtain compression performance data, storing the compression performance data into a preset data storage area, transmitting the compression performance data in the preset data storage area according to a preset transmission program and a preset time interval, decompressing the compression performance data into standard performance data under a standard data structure according to a standard data structure defined by a preset data mapping program, wherein the eBPF program has technical defects of certain network bandwidth waste, transmission delay increase and the like in terms of data transmission and processing, so that the collaborative optimization strategy combining eBPF and LZ4 compression algorithm is used, and the LZ4 algorithm is embedded into eBPF program according to the advantages of high compression speed and low delay of LZ4, so that efficient real-time compression can be realized while data acquisition is realized, thereby reducing network flow, improving data transmission speed, effectively reducing system resource burden, and improving the overall processing efficiency of network data.
After the step of performing structural transformation on the real-time performance data to obtain standard performance data, the distributed system performance tracking method further includes:
step B10, acquiring a communication mechanism supported by the preset data mapping program;
And step B20, transmitting the standard performance data to the preset SKYWALKING program according to the communication mechanism.
It should be noted that, the communication mechanism is used to indicate a protocol capable of performing data communication, and may specifically include a communication protocol such as HTTP (Hypertext Transfer Protocol ) or gRPC (Google Remote Procedure Call, google remote procedure call), and in particular, in SKYWALKINGAGENT, the receiver mapped by eBPF may be configured to support both gRPC and HTTP protocols, so that skywalking has a variety of communications, and skywalking has a certain adaptability to various distributed systems.
In one possible implementation, the collected data is read from the kernel space by writing a eBPF mapping program, that is, a preset mapping program, in the user space, using the eBPF mapping API, and the decompressed data packet in the LZ4 algorithm database is called in the user space, then the compression performance data is decompressed according to the preset data structure, and the decompressed compression performance data is sent to the external monitoring system skywalking for real-time visualization. Specifically, by embedding an LZ4 decompression library in eBPF mapping programs written in a user space, accurate decompression of compressed tracking data is realized. Meanwhile, a data structure conforming to SKYWALKING standard is defined, which may include core call information such as service call, operation name, start time and end time, and other related call details, so that a data structure conforming to SKYWALKING is constructed with high readability and versatility. Further, by adding a mechanism supporting both HTTP and gRPC communication protocols, dual transmission of data to SKYWALKINGAGENT is achieved.
As an example, steps B10 to B20 include: and acquiring a communication mechanism supported by the preset data mapping program, and transmitting standard performance data to the preset SKYWALKING program according to the communication mechanism, so that monitoring of data tracked and collected by the eBPF program is realized according to asynchronous call chain tracking data observed at a skywalking webpage interface, and the reliability of real-time monitoring and tracking functions of a call chain in a distributed system is ensured.
The step of acquiring the real-time performance data of the distributed system comprises the following steps:
Step C10, acquiring real-time process information of the distributed system;
Step C20, determining data acquisition logic for acquiring data of the distributed system according to the real-time process information;
and step C30, acquiring real-time performance data generated under the real-time process information according to the data acquisition logic.
It should be noted that, the real-time process information is used to represent a process generated or invoked by the distributed system in the running process, and the data acquisition logic is used to represent a data acquisition mode corresponding to the real-time process information, that is, for different real-time process information, the executed data acquisition logic is not used, so that the current performance data can be determined by identifying the real-time process information executed by the tape tracking system, and thus, the current performance data acquisition logic is executed according to the real-time process information, so as to realize acquisition of various types of performance data.
In one possible implementation, by writing eBPF a program for retrieving data generated by the distributed system during operation, using Tracepoint as an entry point for system calls, a map of eBPF is defined and eBPF is initialized for data collection operations, at which point the map will store the collected data in kernel space while a preset system call function will be used to register Tracepoint. In the logic of data collection, the ID (Identity document, identity) of the current process is called by using a preset programming language, and the current performance data collection logic is executed, which may specifically relate to the acquisition of various information, such as calling time or system calling parameters, and meanwhile, the collected data calling information is saved in the defined eBPF map.
As an example, steps C10 to C30 include: acquiring real-time process information of the distributed system, determining data acquisition logic for acquiring data of the distributed system according to the real-time process information, and acquiring real-time performance data generated under the real-time process information according to the data acquisition logic.
Example two
In another embodiment of the present application, the same or similar content as that of the first embodiment may be referred to the description above, and will not be repeated. On this basis, referring to fig. 3, before the step of acquiring the real-time performance data of the distributed system, the distributed system performance tracking method further includes:
Step D10, acquiring a preset storage version and a preset data port for real-time tracking of the distributed system;
and step D20, performing environment deployment on a preset SKYWALKING program according to the preset storage version and the preset data port.
It should be noted that, the preset storage version is used to represent a configuration environment for storing real-time tracking data, and may specifically represent an environment version capable of analyzing and storing performance data in eBPF program environment, and the environment deployment represents an environment setup for use of SKYWALKING program, so that the SKYWALKING program may perform operations such as data reading and chart displaying in the environment.
In one possible implementation, when setting up eBPF the environment, it is necessary to confirm that the Linux (operating system kernel) environment supports the kernel version of the eBPF latest feature, and then, after satisfying the kernel version, configure yum (Yellow Dog Updater, modified, package manager) source installation dependencies and tools thereof.
Additionally, in one possible implementation, the SKYWALKING program environment is built, typically involving deploying components such as backend OPA (Open Policy Agent, general policy engine), agents, UI (USER INTERFACE), and data stores of es (search server), selecting the corresponding storage version and usage version. Then, the configuration file of the OPA is modified, and meanwhile, a storage configuration item can be modified at the user side, a storage selector configuration item is used to set a specifically used memory, for example, an elastesearch7. X version is set to be used as the memory, additionally, the configuration file in SkyWalkingUI is modified, default port information in the environment is modified to prevent port conflict between multiple middleware, and finally, the probe configuration of each configuration file in the proxy environment is also required to be modified, and the configuration item customized by the user is acquired.
As an example, steps D10 to D20 include: and acquiring a preset storage version and a preset data port for real-time tracking of the distributed system, and performing environment deployment on a preset SKYWALKING program according to the preset storage and the preset data port.
Wherein the step of tracking the distributed system by parsing the standard performance data into a tracking chart according to a preset SKYWALKING program includes:
step E10, acquiring a preset time index and a preset performance index;
step E20, determining a chart template for analyzing the standard performance data according to the preset time index and the preset performance index;
Step E30, generating a tracking chart under the preset time index according to the chart template and the preset SKYWALKING program;
and E40, tracking the distributed system by showing the tracking chart.
It should be noted that, the preset performance index is used to represent index information for classifying or displaying each operation performance data acquired on each system to be tracked according to conditions, specifically, performance data such as CPU (Central Processing Unit ) usage rate, memory occupation, network flow and the like are selected, and display modes or display contents of drinks are determined, and the chart template is used to represent a template for visualizing standard performance data, specifically, a time chart or a flowchart of an asynchronous call chain, specifically, standard performance data can be analyzed according to the chart template according to a preset display mode, so that the standard performance data can be visually displayed.
Additionally, the preset time index is used to represent a time-dependent change index of the asynchronous call chain, and specifically may be a change condition of the asynchronous call chain every hour, every half hour, every minute, or the like. Specifically, all performance indexes obtained by the eBPF program are converted into a tracking chart under the corresponding preset time index by acquiring the preset time index indicated by the user, wherein the tracking chart is a visual chart.
In one possible implementation, to implement asynchronous standard performance data visualization, a new dashboard may be created at SKYWALKING, dedicated to exposing eBPF performance metrics and asynchronous call chains. By selecting an appropriate chart type for the performance index eBPF, and simultaneously acquiring the index of the display change along with time selected by the user, correspondingly, a time line diagram or a flow chart can be selected for the asynchronous call chain. The comprehensive performance insight of the distributed system is displayed to the user, and the comprehensive performance insight can comprise CPU utilization rate, memory occupation, network flow and the like, so that potential performance problems or bottlenecks can be found conveniently and rapidly.
As an example, steps E10 to E40 include: acquiring a preset time index and a preset performance index, determining a corresponding chart template, generating a tracking chart changing along with time according to the chart template and a preset SKYWALKING program, and displaying the tracking chart. Therefore, through the visualized chart, the user can more quickly locate the problem point in the system, the fault removal time can be shortened, the usability and stability of the system can be improved, and continuous improved power can be provided for the user.
Wherein after the step of tracking the distributed system by parsing the standard performance data into a tracking chart according to the preset SKYWALKING program, the distributed system performance tracking method further includes:
step F10, acquiring a preset performance threshold;
and F20, outputting a performance abnormality alarm when the standard performance data reach the preset performance threshold.
It should be noted that, the preset performance threshold may specifically be a maximum value or a minimum value that can be reached by each standard performance data in the tracking chart, that is, when the standard performance data reaches or exceeds the preset performance threshold of the maximum value or the minimum value, it will be determined that the standard performance data is abnormal, at this time, the performance data corresponding to the standard performance data is located, and a performance abnormality alarm is sent to the user side.
As an example, steps F10 to F20 include: the method comprises the steps of obtaining a preset performance threshold, comparing all standard performance data in a tracking chart with the preset performance threshold, combining a rule engine and a performance analysis plug-in when the standard performance data exceeds the preset performance threshold, positioning performance problems corresponding to the standard performance data, and setting an alarm notification, so that complexity of problem positioning is reduced through automatic monitoring and positioning, and automation level of system tracking and monitoring is improved.
According to the embodiment, a new instrument board is created in SKYWALKING and is specially used for displaying eBPF performance indexes and an asynchronous call chain, a SKYWALKING visual interface is integrated, so that a user can select and display indexes changing along with time, a time line diagram or a flow chart is selected for the asynchronous call chain, real-time tracking and visual presentation of the asynchronous call chain are realized, potential performance problems can be timely found and solved through comprehensive asynchronous call chain tracking and real-time performance analysis and combination of the visual interface, stability of a system is improved, and simultaneously, functions of real-time monitoring, visual display, user-friendly interface design, automatic rule setting and the like are provided, so that user experience is smoother, a user is helped to manage and optimize a distributed system more quickly and effectively, operation and maintenance efficiency is improved, labor cost is reduced, and finally, an automatic system to be tracked can be monitored and positioned, and alarm notification is set when an abnormality occurs in the system to be tracked.
Example III
The embodiment of the application also provides a distributed system performance tracking device based on eBPF and SKYWALKING technologies, referring to fig. 4, the distributed system performance tracking device includes:
an acquisition module 101, configured to acquire real-time performance data of the distributed system;
The conversion module 102 is configured to perform structural conversion on the real-time performance data to obtain standard performance data, where the standard performance data is used to display tracking information included in the real-time performance data;
The tracking module 103 is configured to track the distributed system by parsing the standard performance data into a tracking chart according to a preset SKYWALKING program.
Optionally, the conversion module 102 is further configured to:
compressing the real-time performance data to obtain compression performance data, and storing the compression performance data into a preset data storage area;
Transmitting the compression performance data in the preset data storage area according to a preset transmission program and a preset time interval;
and decompressing the compression performance data into standard performance data under the standard data structure according to a standard data structure defined by a preset data mapping program.
Optionally, the distributed system performance tracking device is further configured to:
Acquiring a communication mechanism supported by the preset data mapping program;
And transmitting the standard performance data to the preset SKYWALKING program according to the communication mechanism.
Optionally, the obtaining module 101 is further configured to:
acquiring real-time process information of the distributed system;
Determining a data acquisition logic for acquiring data of the distributed system according to the real-time process information;
and acquiring real-time performance data generated under the real-time process information according to the data acquisition logic.
Optionally, the distributed system performance tracking device is further configured to:
Acquiring a preset storage version and a preset data port for real-time tracking of the distributed system;
And performing environment deployment on a preset SKYWALKING program according to the preset storage version and the preset data port.
Optionally, the tracking module 103 is further configured to:
Acquiring a preset time index and a preset performance index;
determining a chart template for analyzing the standard performance data according to the preset time index and the preset performance index;
generating a tracking chart under the preset time index according to the chart template and the preset SKYWALKING program;
the distributed system is tracked by showing the tracking graph.
Optionally, the distributed system performance tracking device is further configured to:
Acquiring a preset performance threshold;
and outputting a performance abnormality alarm when the standard performance data reaches the preset performance threshold.
The distributed system performance tracking device provided by the application adopts the distributed system performance tracking method in the first embodiment or the second embodiment, and can solve the technical problem that the tracking method of the existing call chain has poor effects on distributed system monitoring and performance tuning. Compared with the prior art, the beneficial effects of the distributed system performance tracking device provided by the embodiment of the application are the same as those of the distributed system performance tracking method provided by the embodiment, and other technical features in the distributed system performance tracking device are the same as those disclosed in the method of the previous embodiment, so that details are not repeated.
Example IV
The embodiment of the application provides electronic equipment, which comprises: at least one processor; and a memory communicatively coupled to the at least one processor; the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the distributed system performance tracking method of the first embodiment.
Referring now to fig. 5, a schematic diagram of an electronic device suitable for use in implementing embodiments of the present disclosure is shown. The electronic devices in the embodiments of the present disclosure may include, but are not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (PortableApplicationDescription: tablet computers), PMPs (PortableMediaPlayer: portable multimedia players), car terminals (e.g., car navigation terminals), and the like, and fixed terminals such as digital TVs, desktop computers, and the like. The electronic device shown in fig. 5 is merely an example and should not be construed to limit the functionality and scope of use of the disclosed embodiments.
As shown in fig. 5, the electronic apparatus may include a processing device 1001 (e.g., a central processor, a graphics processor, etc.) that may perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM) 1002 or a program loaded from a storage device 1003 into a Random Access Memory (RAM) 1004. In the RAM1004, various programs and data required for the operation of the electronic device are also stored. The processing device 1001, the ROM1002, and the RAM1004 are connected to each other by a bus 1005. An input/output (I/O) interface 1006 is also connected to the bus. In general, the following systems may be connected to the I/O interface 1006: input devices 1007 including, for example, a touch screen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, and the like; an output device 1008 including, for example, a liquid crystal display (LCD: liquidCrystalDisplay), a speaker, a vibrator, and the like; storage device 1003 including, for example, a magnetic tape, a hard disk, and the like; and communication means 1009. The communication means 1009 may allow the electronic device to communicate with other devices wirelessly or by wire to exchange data. While electronic devices having various systems are shown in the figures, it should be understood that not all of the illustrated systems are required to be implemented or provided. More or fewer systems may alternatively be implemented or provided.
In particular, according to embodiments of the present disclosure, the processes described above with reference to flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method shown in the flowcharts. In such an embodiment, the computer program may be downloaded and installed from a network through a communication device, or installed from the storage device 1003, or installed from the ROM 1002. The above-described functions defined in the method of the embodiment of the present disclosure are performed when the computer program is executed by the processing device 1001.
The electronic equipment provided by the application adopts the distributed system performance tracking method in the embodiment, and can solve the technical problem that the tracking method of the existing call chain has poor effects on distributed system monitoring and performance tuning. Compared with the prior art, the electronic device provided by the embodiment of the application has the same beneficial effects as the distributed system performance tracking method provided by the embodiment, and other technical features in the electronic device are the same as the features disclosed by the method of the previous embodiment, and are not described in detail herein.
It should be understood that portions of the present disclosure may be implemented in hardware, software, firmware, or a combination thereof. In the description of the above embodiments, particular features, structures, materials, or characteristics may be combined in any suitable manner in any one or more embodiments or examples.
The foregoing is merely illustrative of the present application, and the present application is not limited thereto, and any person skilled in the art will readily recognize that variations or substitutions are within the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.
Example five
An embodiment of the present application provides a computer readable storage medium having computer readable program instructions stored thereon for performing the distributed system performance tracking method of the first embodiment.
The computer readable storage medium according to the embodiments of the present application may be, for example, a usb disk, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, or device, or a combination of any of the foregoing. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM: erasableProgrammableReadOnlyMemory or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In this embodiment, a computer-readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, or device. Program code embodied on a computer readable storage medium may be transmitted using any appropriate medium, including but not limited to: electrical wires, fiber optic cables, RF (radio frequency), and the like, or any suitable combination of the foregoing.
The above-described computer-readable storage medium may be contained in an electronic device; or may exist alone without being assembled into an electronic device.
The computer-readable storage medium carries one or more programs that, when executed by an electronic device, cause the electronic device to: acquiring real-time performance data of a distributed system; performing structural transformation on the real-time performance data to obtain standard performance data, wherein the standard performance data is used for displaying tracking information contained in the real-time performance data; according to a preset SKYWALKING program, the standard performance data is analyzed into a tracking chart to track the distributed system.
Computer program code for carrying out operations of the present disclosure may be written in one or more programming languages, including an object oriented programming language such as Java, smalltalk, C ++ and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of remote computers, the remote computer may be connected to the user's computer through any kind of network, including a local area network (LAN: localAreaNetwork) or a wide area network (WAN: wideAreaNetwork), or may be connected to an external computer (for example, through the Internet using an Internet service provider).
The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The modules described in the embodiments of the present disclosure may be implemented in software or hardware. Wherein the name of the module does not constitute a limitation of the unit itself in some cases.
The readable storage medium provided by the application is a computer readable storage medium, and the computer readable storage medium stores computer readable program instructions for executing the distributed system performance tracking method, so that the technical problem that the tracking method of the existing call chain has poor effects on distributed system monitoring and performance tuning can be solved. Compared with the prior art, the beneficial effects of the computer readable storage medium provided by the embodiment of the present application are the same as those of the distributed system performance tracking method provided by the first embodiment or the second embodiment, and are not described herein.
Example six
Embodiments of the present application also provide a computer program product comprising a computer program which, when executed by a processor, implements the steps of a distributed system performance tracking method as described above.
The computer program product provided by the application can solve the technical problem that the tracking method of the existing call chain has poor effects on monitoring and performance tuning of the distributed system. Compared with the prior art, the beneficial effects of the computer program product provided by the embodiment of the present application are the same as those of the distributed system performance tracking method provided by the first embodiment or the second embodiment, and are not described herein.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or system. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or system that comprises the element.
The foregoing embodiment numbers of the present application are merely for the purpose of description, and do not represent the advantages or disadvantages of the embodiments.
From the above description of the embodiments, it will be clear to those skilled in the art that the above-described embodiment method may be implemented by means of software plus a necessary general hardware platform, but of course may also be implemented by means of hardware, but in many cases the former is a preferred embodiment. Based on such understanding, the technical solution of the present application may be embodied essentially or in a part contributing to the prior art in the form of a software product stored in a storage medium (e.g. ROM/RAM, magnetic disk, optical disk) as described above, comprising instructions for causing a terminal device (which may be a mobile phone, a computer, a server, or a network device, etc.) to perform the method according to the embodiments of the present application.
The foregoing description is only of the preferred embodiments of the present application, and is not intended to limit the scope of the application, but rather is intended to cover any equivalents of the structures or equivalent processes disclosed herein or in the alternative, which may be employed directly or indirectly in other related arts.

Claims (10)

1. A distributed system performance tracking method based on eBPF and SKYWALKING technologies, the distributed system performance tracking method comprising:
acquiring real-time performance data of a distributed system;
performing structural transformation on the real-time performance data to obtain standard performance data, wherein the standard performance data is used for displaying tracking information contained in the real-time performance data;
according to a preset SKYWALKING program, the standard performance data is analyzed into a tracking chart to track the distributed system.
2. The method for tracking performance of a distributed system according to claim 1, wherein the step of performing structural transformation on the real-time performance data to obtain standard performance data comprises:
compressing the real-time performance data to obtain compression performance data, and storing the compression performance data into a preset data storage area;
Transmitting the compression performance data in the preset data storage area according to a preset transmission program and a preset time interval;
and decompressing the compression performance data into standard performance data under the standard data structure according to a standard data structure defined by a preset data mapping program.
3. The distributed system performance tracking method based on eBPF and SKYWALKING techniques of claim 2, wherein after said step of structurally transforming said real-time performance data to obtain standard performance data, said distributed system performance tracking method further comprises:
Acquiring a communication mechanism supported by the preset data mapping program;
And transmitting the standard performance data to the preset SKYWALKING program according to the communication mechanism.
4. The distributed system performance tracking method based on eBPF and SKYWALKING techniques of claim 1, wherein the step of acquiring real-time performance data of the distributed system comprises:
acquiring real-time process information of the distributed system;
Determining a data acquisition logic for acquiring data of the distributed system according to the real-time process information;
and acquiring real-time performance data generated under the real-time process information according to the data acquisition logic.
5. The distributed system performance tracking method based on eBPF and SKYWALKING techniques of claim 1, wherein prior to the step of obtaining real-time performance data of the distributed system, the distributed system performance tracking method further comprises:
Acquiring a preset storage version and a preset data port for real-time tracking of the distributed system;
And performing environment deployment on a preset SKYWALKING program according to the preset storage version and the preset data port.
6. The distributed system performance tracking method based on eBPF and SKYWALKING techniques of claim 1, wherein tracking the distributed system by parsing the standard performance data into a tracking graph according to a preset SKYWALKING program includes:
Acquiring a preset time index and a preset performance index;
determining a chart template for analyzing the standard performance data according to the preset time index and the preset performance index;
generating a tracking chart under the preset time index according to the chart template and the preset SKYWALKING program;
the distributed system is tracked by showing the tracking graph.
7. The distributed system performance tracking method based on eBPF and SKYWALKING techniques of claim 1, wherein after the step of tracking the distributed system by parsing the standard performance data into tracking charts according to a preset SKYWALKING program, the distributed system performance tracking method further comprises:
Acquiring a preset performance threshold;
and outputting a performance abnormality alarm when the standard performance data reaches the preset performance threshold.
8. A distributed system performance tracking device based on eBPF and SKYWALKING techniques, the distributed system performance tracking device comprising:
the acquisition module is used for acquiring real-time performance data of the distributed system;
The conversion module is used for carrying out structural conversion on the real-time performance data to obtain standard performance data, wherein the standard performance data is used for displaying tracking information contained in the real-time performance data;
And the tracking module is used for tracking the distributed system by analyzing the standard performance data into a tracking chart according to a preset SKYWALKING program.
9. An electronic device, the electronic device comprising: a memory, a processor, and a distributed system performance tracking program stored on the memory and executable on the processor, the distributed system performance tracking program configured to implement the steps of the distributed system performance tracking method of any one of claims 1 to 7.
10. A computer readable storage medium, characterized in that it has stored thereon a distributed system performance tracking program based on eBPF and SKYWALKING technology, which when executed by a processor, implements the steps of the method of distributed system performance tracking according to any one of claims 1 to 7.
CN202410154396.9A 2024-02-04 2024-02-04 EBPF and SKYWALKING technology-based distributed system performance tracking method Pending CN117950953A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202410154396.9A CN117950953A (en) 2024-02-04 2024-02-04 EBPF and SKYWALKING technology-based distributed system performance tracking method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202410154396.9A CN117950953A (en) 2024-02-04 2024-02-04 EBPF and SKYWALKING technology-based distributed system performance tracking method

Publications (1)

Publication Number Publication Date
CN117950953A true CN117950953A (en) 2024-04-30

Family

ID=90795949

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202410154396.9A Pending CN117950953A (en) 2024-02-04 2024-02-04 EBPF and SKYWALKING technology-based distributed system performance tracking method

Country Status (1)

Country Link
CN (1) CN117950953A (en)

Similar Documents

Publication Publication Date Title
CN111475298B (en) Task processing method, device, equipment and storage medium
CN111625473B (en) Interface test case generation method and device, storage medium and electronic equipment
CN111274760B (en) Rich text data processing method and device, electronic equipment and computer storage medium
CN111897740B (en) User interface testing method and device, electronic equipment and computer readable medium
CN111367516B (en) Application interface generation method and device and electronic equipment
CN111198859B (en) Data processing method, device, electronic equipment and computer readable storage medium
CN111813641B (en) Method, device, medium and equipment for collecting crash information
CN110489323B (en) Visual RPC API debugging method, device, medium and equipment
CN111221572B (en) Method, device, medium and equipment for automatically adapting to running environment
CN111338944B (en) Remote Procedure Call (RPC) interface testing method, device, medium and equipment
CN112596738B (en) Method and device for determining front-end page to be tested, storage medium and electronic equipment
CN112291121B (en) Data processing method and related equipment
CN112954056B (en) Method and device for processing monitoring data, electronic equipment and storage medium
CN111198853B (en) Data processing method, device, electronic equipment and computer readable storage medium
CN117950953A (en) EBPF and SKYWALKING technology-based distributed system performance tracking method
CN112379967B (en) Simulator detection method, device, equipment and medium
CN115454956A (en) Log generation method and device, electronic equipment and storage medium
CN111625326B (en) Task pipeline execution method and device and electronic equipment
CN111399902B (en) Client source file processing method and device, readable medium and electronic equipment
CN117251355A (en) Performance test method, device, equipment, computer readable storage medium and product
CN111367791B (en) Method, device, medium and electronic equipment for generating test case
CN111382556B (en) Data conversion method, device, equipment and storage medium
CN111381813B (en) Front-end page debugging method and device, computer equipment and storage medium
CN116382597B (en) Vehicle operation data storage method, apparatus and computer medium
CN112688863B (en) Gateway data processing method and device and electronic equipment

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination